JP7720599B2 - Computer program, method, and server device - Google Patents

Description

The technology disclosed herein relates to a computer program, method, and server device used to distribute game-related videos from a distributor's terminal to each viewer's terminal.

A service called Mirrativ (Non-Patent Document 1) is known as a service used to distribute game-related videos from a distributor's device to each viewer's device.

With this service, a broadcaster can use a terminal to broadcast videos including avatar animations generated based on motion data related to the broadcaster's movements, and video related to a game generated based on operation data related to the broadcaster's operations, to multiple viewers' terminals. Non-Patent Document 1 is incorporated herein by reference in its entirety.

"Mirrativ," [online], August 16, 2019, Mirrativ, Inc., [Retrieved December 23, 2019], Internet (https://apps.apple.com/jp/app/mirrativ-Mirrativ-Easy-Game-Streaming-on-Smartphone/id1028944599)

Here, let's consider a case where a video is streamed featuring avatars from multiple streamers (e.g., a first streamer and a second streamer) appearing together, or a video where the multiple streamers play a game together. When the first streamer streams together with the second streamer, the number of viewers may be higher than when the first streamer streams alone. In this case, if the second streamer who streams together with the first streamer stops streaming and leaves, the number of viewers may decrease due to the impact of the departure, which could reduce the interest of the game when the first streamer later streams the game. Therefore, it is desirable to have a configuration that minimizes the interest of the game that will be streamed later by the first streamer, even if the second streamer who streams together with the first streamer stops streaming and leaves.

This disclosure therefore provides a computer program, method, and server device that reduces the possibility of the game's enjoyment being diminished.

In one embodiment, the computer program causes the processor to function as follows: broadcasting a first video, including motion data related to the movements of a first broadcaster or an animation of the first broadcaster's avatar generated based on the motion data, and a second video related to a game generated using a received web page based on operation data related to the first broadcaster's operations, to a viewer terminal of at least one viewer via a communication line; storing first data related to at least one user who jointly broadcasted with the first broadcaster before the game was played; determining whether a first condition is met after the first broadcaster and the at least one user jointly broadcast; and, if it is determined that the first condition is met, extracting second data related to at least one of the at least one user based on the first data; and, during or after the game is played, changing at least one current parameter of the first broadcaster's avatar based on the second data; or displaying the first broadcaster's avatar and at least one of the at least one user's avatar in the second video based on the first data.

A method according to one aspect includes the steps of: by executing instructions by a processor, broadcasting a first video including action data related to the actions of a first broadcaster or an animation of the first broadcaster's avatar generated based on the action data, and a second video related to a game generated using a received web page based on operation data related to the first broadcaster's operations, to a viewer terminal of at least one viewer via a communication line; storing first data related to at least one user who jointly broadcasted with the first broadcaster before the game was played; after the first broadcaster and the at least one user jointly broadcast, determining whether a first condition is met; if it is determined that the first condition is met, extracting second data related to at least one of the at least one user based on the first data; and, during or after the game is played, changing at least one current parameter of the first broadcaster's avatar based on the second data, or displaying the first broadcaster's avatar and at least one of the at least one user's avatar in the second video based on the second data.

A method according to one aspect includes the steps of: executing instructions by a processor to receive, via a communication line, a first video including motion data related to the motions of a first broadcaster or an animation of the first broadcaster's avatar generated based on the motion data; and a second video related to a game generated using a received web page based on operation data related to the first broadcaster's operations; determining whether a first condition is met after the first broadcaster and the at least one user have jointly broadcast; and, if it is determined that the first condition is met, receiving a first video in which at least one current parameter of the first broadcaster's avatar has been changed based on second data extracted related to at least one of the at least one users based on first data related to at least one user who jointly broadcast with the first broadcaster before the game was played; or receiving a second video displaying the first broadcaster's avatar and an avatar of at least one of the at least one user based on the second data.

In one embodiment, a server device has a processor that transmits a first video, including action data related to the actions of a first broadcaster or an animation of the first broadcaster's avatar generated based on the action data, and a second video related to a game generated using a received webpage based on operation data related to the first broadcaster's operations, to the viewer terminal of at least one viewer via a communication line; stores first data related to at least one user who jointly broadcasted with the first broadcaster before the game is played; determines whether a first condition is met after the first broadcaster and the at least one user jointly broadcast; and if it is determined that the first condition is met, extracts second data related to at least one of the at least one user based on the first data; and, during or after the game is played, changes at least one current parameter of the first broadcaster's avatar based on the second data, or displays the first broadcaster's avatar and at least one of the at least one user's avatar in the second video based on the second data.

In one embodiment, the server device transmits a web page including a computer program via a communication line to the broadcaster terminal of the broadcaster, configured such that a processor broadcasts a first video including motion data related to the motions of a first broadcaster or an animation of the first broadcaster's avatar generated based on the motion data, and a second video related to a game generated based on operation data related to the first broadcaster's operations using the received web page to the viewer terminal of at least one viewer via a communication line, and the computer program broadcasts the first broadcaster's broadcaster terminal to at least one viewer who jointly broadcasted with the first broadcaster before the game was played. and stores first data related to the users; after the first broadcaster and the at least one user jointly broadcast, determines whether a first condition is met; and if it is determined that the first condition is met, extracts second data related to at least one of the at least one users based on the first data; and, during or after the game is played, changes at least one current parameter of the first broadcaster's avatar based on the second data, or causes the processor to display the first broadcaster's avatar and an avatar of at least one of the at least one user in the second video based on the second data.

FIG. 1 is a block diagram showing an example of the configuration of a video distribution system 1 according to an embodiment. FIG. 2 is a block diagram schematically illustrating an example of the hardware configuration of the terminal 10 (server device 20) shown in FIG. FIG. 3 is a block diagram schematically illustrating an example of the functions of the terminal 10 shown in FIG. FIG. 4 is a block diagram schematically illustrating an example of the functions of the server device 20 shown in FIG. FIG. 5A is a flow diagram showing an example of the operation performed in the video distribution system 1 shown in FIG. FIG. 5B is a flow diagram showing an example of the operation performed in the video distribution system 1 shown in FIG.

Various embodiments of the present disclosure will now be described with reference to the accompanying drawings. Note that common components in the drawings are designated by the same reference numerals. Please also note that components depicted in one drawing may be omitted in another drawing for the sake of clarity. Furthermore, please note that the accompanying drawings are not necessarily drawn to scale.

1. Configuration of a Video Distribution System Briefly, the video distribution system of the present disclosure enables a user who distributes content (hereinafter referred to as a "distributor") to use his/her terminal to distribute, via a communication line, a first video including animation of an avatar (avatar object) generated based on motion data related to the distributor's motions, and a second video related to a game generated based on operation data related to the distributor's operations, to the terminals of users who watch the video (hereinafter referred to as "viewers"). The first example will be described below with reference to Figures 1 to 5B.

Figure 1 is a block diagram showing an example of the configuration of a video distribution system according to one embodiment. As shown in Figure 1, the video distribution system 1 can include multiple terminals 10 connected to a communication line (communication network) (not shown), and one or more server devices 20 connected to the communication line. Each terminal 10 can be connected to one or more server devices 20 via the communication line.

Note that while Figure 1 shows multiple terminals 10, such as a first broadcaster terminal 11A, a second broadcaster terminal 11B, a viewer terminal 12A, a viewer terminal 12B, and a viewer terminal 12C, one or more other terminals 10 may be used as well. Furthermore, in the following description, the first broadcaster terminal 11A and the second broadcaster terminal 11B may be collectively referred to as broadcaster terminals 11, and the viewer terminals 12A-12C may be collectively referred to as viewer terminals 12. Furthermore, while Figure 1 shows server devices 20A and 20B as one or more server devices 20, one or more other server devices 20 may be used as well. Furthermore, communication lines (not shown) may include, but are not limited to, a mobile phone network, a wireless LAN, a fixed-line telephone network, the Internet, an intranet, and/or Ethernet (registered trademark), etc.

Furthermore, in one embodiment, the video distribution system 1 may also include one or more studio units 30 connected to a communications network. Each studio unit 30 may be connected to one or more server devices 20 or multiple terminals 10 via a communications line. Note that while FIG. 1 shows studio units 30A and 30B, for example, as the one or more studio units 30, one or more other studio units 30 may also be used. Each studio unit 30 may have the same functions as the terminal 10 or server device 20, which will be described later.

For simplicity's sake, the following description focuses on the case where one or more server devices 20 transmit and receive video and other data between each terminal 10. Alternatively or in addition to this, as described below, one or more server devices 20 can transmit and receive video and other data between each studio unit 30. Furthermore, alternatively or in addition to this, as described below, the studio unit 30 can transmit and receive video and other data between each terminal 10.

1-1. Terminal 10
The multiple terminals 10 may include, for example, multiple broadcaster terminals 11 (e.g., a first broadcaster terminal 11A and a second broadcaster terminal 11B) and one or more viewer terminals 12 (e.g., viewer terminals 12A-12C). The broadcaster corresponding to the first broadcaster terminal 11A will be referred to as the "first broadcaster," and the broadcaster corresponding to the second broadcaster terminal 11B will be referred to as the "second broadcaster." In the following description, if the first broadcaster terminal 11A of the multiple broadcaster terminals 11 is broadcasting a game solely, the second broadcaster terminal 11B does not need to function as a terminal that broadcasts a game, and may, for example, be configured to function in the same way as the viewer terminal 12.

Each terminal 10 has a common configuration and can function as either a terminal for distributing videos (distributor terminal 11) or a terminal for viewing videos (viewer terminal 12).

Furthermore, when each terminal 10 operates as a terminal for distributing videos (distributor's terminal), it executes an installed video distribution application (which may be middleware or a combination of an application and middleware; the same applies below) to acquire motion data related to the distributor's movements, and transmits a first video, including animation of a virtual character (avatar object) modified according to the acquired data, to server device 20 (distribution server system 20A1) via a communication line. Such first video is distributed by server device 20 (distribution server system 20A1) via a communication line to a terminal for receiving videos (viewer terminal) 12. Such viewer terminal 12 can receive the video by executing an installed video viewing application (which may be middleware or a combination of an application and middleware; the same applies below).

Furthermore, when operating as a distributor's terminal, each terminal 10 can receive a web page (HTML document) from the server device 20 (web server system 20B) and execute a game program included in the web page by executing a browser function built into the installed video distribution application and/or by executing an installed web browser application. By executing this game program, each terminal 10 can generate a second video in which at least one game object is operated based on operation data related to the distributor's operation.

Furthermore, when each terminal 10 operates as a distributor's terminal, it can transmit the second video generated in this manner to the server device 20 (distribution server system 20A) via a communication line by executing the video distribution application. Such second video is also distributed by the server device 20 (distribution server system 20A) via a communication line to a terminal (viewer terminal) 10 for receiving videos that is executing an installed video viewing application.

Furthermore, when each terminal 10 operates as a distributor's terminal, it can execute the video distribution application to receive, via the server device 20 (distribution server system 20A), gift information indicating that a gift will be given to the distributor and/or comment information indicating that a comment will be sent to the distributor from a viewer terminal 12 viewing the second video (distributed by the distributor terminal 11).

On the other hand, when each terminal 10 operates as a terminal for viewing videos (viewer terminal), it can receive the first video and/or the second video distributed by the distributor terminal 11 via the server device 20 (distribution server system 20A) by executing the installed video viewing application.

Furthermore, when operating as a viewer terminal, each terminal 10 can execute a video viewing application to send gift information indicating that a gift will be given to the broadcaster and/or comment information indicating that a comment will be sent to the broadcaster to the first broadcaster terminal 11A via the server device 20 (distribution server system 20A).

The video distribution application and video viewing application described above can be installed and executed on each terminal 10 as a single integrated application, or as separate applications.

Each terminal 10 is any terminal capable of performing such operations, and may include, but is not limited to, a smartphone, tablet, mobile phone (feature phone), and/or personal computer. Furthermore, each terminal 10 is configured to be capable of, for example, collaborative distribution (joint distribution or collaborative distribution).

In the following description, collaborative streaming refers to, for example, multiple streamers jointly streaming using their respective devices. Such collaborative streaming can include, for example, video streaming using the avatar of a first streamer and the avatars of other streamers, and game streaming conducted by the first streamer and other streamers. The game streamed through such game streaming is, for example, a live game. Furthermore, in a game streamed through collaborative streaming, users streaming and/or watching the game can input operation information and/or voice into their own devices 10, and transmit that input information to devices 10 operated by other users, enabling users to communicate with each other.

Furthermore, when a collaborative broadcast is conducted, for example, each of the multiple broadcasters may be treated separately. For example, one of the multiple broadcasters may be treated as, for example, the host broadcaster, and the remaining broadcasters may be treated as, for example, guests. Specifically, for example, when two broadcasters (a first broadcaster and a second broadcaster) are conducting a collaborative broadcast, the first broadcaster may be treated as the host broadcaster, and the second broadcaster may be treated as a guest. In this case, when the collaborative broadcast ends, the first broadcaster continues the broadcast, and the second broadcaster is no longer able to continue the broadcast. Note that, even if the second broadcaster is no longer able to continue the broadcast, for example, the broadcast may be configured to be viewable as a viewer.

Furthermore, when multiple broadcasters are treated separately, for example, the display content displayed on the display screen of the device operated by the host broadcaster may be configured to be different from the display content displayed on the display screen of the device operated by the guest.Furthermore, when multiple broadcasters are treated separately, for example, the number of commands that can be operated by the host broadcaster may be configured to be greater than the number of commands that can be operated by the guest.

The above describes an example in which multiple broadcasters are treated separately when performing collaborative broadcasting, but this is not limited to this example. When performing collaborative broadcasting, multiple broadcasters may be treated without distinction.

1-2. Server device 20
As shown in FIG. 1, the server device 20 includes, for example, a distribution server system 20A and a web server system 20B.

The distribution server system 20A can distribute the first video and/or the second video transmitted by each terminal (each broadcaster terminal) 10 to each terminal (each viewer terminal) 10. The distribution server system 20A can also transmit to the broadcaster terminal 11 gift information transmitted by each terminal (each viewer terminal) 10 indicating that a gift will be given to a certain broadcaster and/or comment information indicating that a comment will be sent to a certain broadcaster.

To perform such operations, in one embodiment, the distribution server system 20A may include a distribution server device 20A1, a database (DB) server device 20A2, a game API server device 20A3, and a collaboration server device 20A4, which are connected to each other via communication lines (including wireless and/or wired lines, not shown).

The distribution server device 20A1 can distribute the first video and/or the second video transmitted by each broadcaster terminal 11 to each terminal 10. Furthermore, the distribution server device 20A1 can store gift information and/or comment information transmitted by each viewer terminal 12 in the DB server device 20A2.

The DB server device 20A2 stores gift information and/or comment information received from the distribution server device 20A1, and can read and transmit gift information and/or comment information queried by the game API server device 20A3 to the game API server device 20A3.

When the game API server device 20A3 receives an inquiry about gift information and/or comment information from each broadcaster terminal 11, it can read the gift information and/or comment information for the broadcaster terminal 11 from the DB server device 20A2 and send it to that broadcaster terminal 11. In addition, to determine users with whom to perform collaborative broadcasts, the game API server device 20A3 acquires information on candidate users other than the first broadcaster and stores this information in the DB server device 20A2. Here, "candidate users other than the first broadcaster" are, for example, the second broadcaster or viewers.

For example, during collaborative distribution, the collaboration server device 20A4 communicates audio data and/or motion data with each terminal 10. For example, if a first broadcaster makes a sound to the first broadcaster terminal 11A during collaborative distribution, the collaboration server device 20A4 receives data related to the sound from the first broadcaster terminal 11A and transmits the data related to the sound to the second broadcaster terminal 11B and/or the viewer terminal 12. Also, for example, if a second broadcaster makes a sound to the second broadcaster terminal 11B during collaborative distribution, the collaboration server device 20A4 receives data related to the sound from the second broadcaster terminal 11B and transmits the data related to the sound to the first broadcaster terminal 11A and/or the viewer terminal 12.

In the example shown in Figure 1, for purposes such as load distribution, distribution server system 20A is shown as having four server devices: distribution server device 20A1, DB server device 20A2, game API server device 20A3, and collaboration server device 20A4. However, at least one of these four server devices may be integrated with at least one of the remaining server devices.

As shown in FIG. 1, the web server system 20B includes, for example, a game server device 20B1 and a content server device 20B2.

The game server device 20B1 transmits a web page (HTML, for example, a document created in accordance with HTML5) incorporating a game program to each accessing distributor terminal 11. This game program allows the terminal 10 that receives and decodes it to execute various operations related to video distribution, including the operations exemplified in (a) to (c) below.
(a) An operation of executing a game; (b) An operation of transmitting a game screen generated in association with the execution of the game to the distribution server system 20A (distribution server device 20A1); and (c) an operation of acquiring gift information and/or comment information sent to this distributor from the distribution server system 20A (game API server device 20A3).

For example, when a game is distributed through collaborative distribution, the game server device 20B1 manages the operation information for the game and performs game logic calculations. Furthermore, for example, when a game is distributed through collaborative distribution, the game server device 20B1 transmits and receives game operation information and/or calculation information to and from terminals 10 operated by users who distribute and/or watch the game. During collaborative distribution, the game server device 20B1 stores data related to the second broadcaster and/or viewers of the game distributed through collaborative distribution. The data stored by the game server device 20B1 can be used to determine the content of the specified processing at S324, described below.

The content server device 20B2 is a device that stores, for example, game-related data (e.g., live game content) and provides game rendering data to terminals 10 operated by users who broadcast and/or watch the games.

1-3. Studio Unit 30
The studio unit 30 may be placed in a studio, room, hall, or the like where a performer (distributor) distributes a video. As described above, the studio unit 30 is capable of performing the same functions as the terminal 10 and/or the server device 20. In FIG. 1 , dashed lines indicate that data can be communicated between the studio unit 30 and other devices, and solid lines indicate that data can be communicated between other devices.

2. Hardware Configuration of Each Device Next, an example of the hardware configuration of each of the terminal 10, the server device 20, and the studio unit 30 will be described.

2-1. Hardware Configuration of Terminal 10 An example of the hardware configuration of each terminal 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram that schematically shows an example of the hardware configuration of the terminal 10 (server device 20) shown in Fig. 1 (note that in Fig. 2, reference numerals in parentheses are those described in relation to the server device 20, as will be described later).

As shown in FIG. 2, each terminal 10 may mainly include a central processing unit 14, a main memory device 15, an input/output interface device 16, an input device 17, an auxiliary memory device 18, and an output device 19. These devices are connected to each other by a data bus and/or a control bus.

The central processing unit 14, also known as the "CPU," performs calculations on the instructions and data stored in the main memory 15 and stores the results of these calculations in the main memory 15. Furthermore, the central processing unit 14 can control the input device 17, auxiliary memory device 18, output device 19, and the like via the input/output interface device 16. The terminal 10 can include one or more such central processing units 14.

The main memory device 15 is also referred to as "memory" and stores instructions and data received from the input device 17, auxiliary memory device 18, and communication line 30 (such as server device 20) via the input/output interface device 16, as well as the results of calculations by the central processing unit 14. The main memory device 15 can include, but is not limited to, RAM (random access memory), ROM (read-only memory), and/or flash memory.

The auxiliary storage device 18 is a storage device with a larger capacity than the main storage device 15. It can store the instructions and data (computer programs) that make up the specific applications (video distribution applications, video viewing applications, etc.) and web browser applications described above. Furthermore, the auxiliary storage device 18 is controlled by the central processing unit 14, and can transmit these instructions and data (computer programs) to the main storage device 15 via the input/output interface device 16. The auxiliary storage device 18 can include, but is not limited to, a magnetic disk device and/or an optical disk device.

The input device 17 is a device that inputs data from the outside and includes, but is not limited to, a touch panel, buttons, a keyboard, a mouse, and/or sensors. As described below, sensors may include, but are not limited to, one or more cameras and/or one or more microphones.

The output device 19 may include, but is not limited to, a display device, a touch panel, and/or a printer device.

In this hardware configuration, the central processing unit 14 sequentially loads instructions and data (computer programs) constituting a specific application stored in the auxiliary storage device 18 into the main storage device 15, and by operating on the loaded instructions and data, controls the output device 19 via the input/output interface device 16, or transmits and receives various information to and from other devices (e.g., a server device 20, other terminals 10, and/or a studio unit 30, etc.) via the input/output interface device 16 and the communication line 2.

As a result, the terminal 10 can execute at least one of the following operations (a) to (d) by executing the installed specific application.
(a) Operations necessary for distributing the first video and/or the second video; (b) Operations necessary for receiving gift information and/or comment information sent to the terminal 10 by another terminal 10; (c) Operations necessary for receiving the first video and/or the second video distributed by another terminal 10; (d) Operations necessary for transmitting gift information and/or comment information to the other terminal 10.

In addition, the terminal 10 may include one or more microprocessors and/or graphics processing units (GPUs) instead of or in addition to the central processing unit 14.

2-2. Hardware Configuration of Server Device 20 An example of the hardware configuration of each server device 20 will be described again with reference to FIG. 2. The hardware configuration of each server device 20 (distribution server device 20A1, DB server device 20A2, game API server device 20A3, game server device 20B1, and content server device 20B2) may be the same as the hardware configuration of each terminal 10 described above. Therefore, reference numerals for the components of each server device 20 are shown in parentheses in FIG. 2.

As shown in FIG. 2, each server device 20 may mainly include a central processing unit 24, a main memory device 25, an input/output interface device 26, an input device 27, an auxiliary memory device 28, and an output device 29. These devices are connected to each other via a data bus and/or a control bus.

The central processing unit 24, main memory device 25, input/output interface device 26, input device 27, auxiliary memory device 28, and output device 29 can be substantially the same as the central processing unit 14, main memory device 15, input/output interface device 16, input device 17, auxiliary memory device 18, and output device 19 included in each of the terminals 10 described above.

In this hardware configuration, the central processing unit 24 sequentially loads instructions and data (computer programs) constituting a specific application stored in the auxiliary storage device 28 into the main storage device 25, and by performing operations on the loaded instructions and data, controls the output device 29 via the input/output interface device 26, or transmits and receives various information to and from other devices (such as terminals 10 and/or studio units 30) via the input/output interface device 26 and communication line 2.

This allows the server device 20 to perform operations necessary to distribute the first video and/or second video transmitted by each broadcaster terminal 11 to each viewer terminal 12, and operations necessary to transmit gift information and/or comment information transmitted by each viewer terminal 12 to a certain broadcaster terminal 11 to that broadcaster terminal 11 (including various operations described in detail below).

In addition, the server device 20 may include one or more microprocessors and/or graphics processing units (GPUs) instead of or in addition to the central processing unit 24.

2-3. Hardware Configuration of Studio Unit 30 As described above, the studio unit 30 is capable of performing the same functions as the terminal 10 and/or the server device 20. Therefore, the studio unit 30 can have the same hardware configuration as the terminal 10 or the server device 20 described above.

3. Functions of Each Device Next, an example of the functions of each of the terminal 10, the server device 20, and the studio unit 30 will be described.

3-1 Functions of the Terminal 10 An example of the functions of the terminal 10 will be described with reference to Fig. 3. Fig. 3 is a block diagram that schematically shows an example of the functions of the terminal 10 shown in Fig. 1.

As shown in FIG. 3, the terminal 10 may mainly include a first video generation unit 100, a second video generation unit 110, a communication unit 120, a display unit 130, a memory unit 140, a user interface unit 150, and a gift processing unit 160.

(1) First moving image generator 100
The first moving image generating unit 100 can generate a first moving image including animation of an avatar object of the broadcaster based on motion data related to the broadcaster's motion. To achieve this, the first moving image generating unit 100 can include, for example, an acquiring unit 102 and a processing unit 104.

The acquisition unit 102 may include one or more first sensors 102a (not shown) that acquire data related to the streamer's body, and one or more second sensors 102b (not shown) that acquire audio data related to speaking and/or singing uttered by the streamer.

In a preferred embodiment, the first sensor 102a may include an RGB camera that captures visible light and a near-infrared camera that captures near-infrared light. Such a camera may be, for example, the one included in the iPhone X (registered trademark) TrueDepth camera. The second sensor 102b may include a microphone that records audio.

First, with regard to the first sensor 102a, the acquisition unit 102 captures an image of the streamer's body using the first sensor 102a, which is placed close to the streamer's body. This allows the acquisition unit 102 to generate data (e.g., an MPEG file) in which images captured by the RGB camera are associated with a time code (a code indicating the time of capture) and recorded over a unit of time. Furthermore, the acquisition unit 102 can generate data (e.g., a TSV file) in which a predetermined number (e.g., 51) of depth-indicating values (e.g., floating-point values) captured by the near-infrared camera are associated with the time code and recorded over a unit of time. A TSV file is a file format in which multiple pieces of data are recorded, separated by tabs.

Regarding the near-infrared camera, specifically, a dot projector emits an infrared laser that forms a dot pattern onto the performer's body, and the near-infrared camera captures the infrared dots projected onto and reflected from the performer's body, generating an image of the captured infrared dots. The acquisition unit 102 compares an image of the dot pattern emitted by a pre-registered dot projector with an image captured by the near-infrared camera, and can calculate the depth of each point (feature point) (e.g., the distance between each point/feature point and the near-infrared camera) using the positional deviation of each point (feature point) (e.g., each of 51 points/feature points) in both images. The acquisition unit 102 can associate the numerical value indicating the calculated depth with the time code as described above, and generate data recorded over a unit time.

Next, with regard to the second sensor 102b, the acquisition unit 102 acquires audio related to the speaking and/or singing uttered by the streamer using the second sensor 102b, which is positioned close to the streamer's body. This allows the acquisition unit 100 to generate data (e.g., an MPEG file) recorded over a unit time period associated with a time code. In one embodiment, the acquisition unit 102 can acquire audio data related to the speaking and/or singing uttered by the streamer using the second sensor 102b, while simultaneously acquiring data related to the streamer's body using the first sensor 102a. In this case, the acquisition unit 102 can generate data (e.g., an MPEG file) recorded over a unit time period associated with the same time code, including the image acquired by the RGB camera and the audio data related to the speaking and/or singing uttered by the streamer using the second sensor 102b.

The acquisition unit 102 can output the data generated in this manner related to the broadcaster's body (MPEG files, TSV files, etc.) and/or audio data related to the broadcaster's speech and/or singing (MPEG files, etc.) to the processing unit 104.

Here, we have described a case where the first sensor 102a includes an RGB camera and an infrared camera, but the first sensor 102a can also include, for example, any of the following (A) to (C):
(A) A camera that detects infrared light emitted by an infrared laser and reflected off the performer's face. (B) Multiple RGB cameras that capture visible light. (C) A single camera that captures visible light.

In the case of (A) above, the acquisition unit 102 can calculate the depth of each feature point on the streamer's body using the same method as described above. In the case of (B) above, the acquisition unit 102 can calculate the depth of each feature point on the streamer's body using multiple images captured by multiple RGB cameras. In the case of (C) above, the acquisition unit 102 can calculate the depth of each feature point on the streamer's body from images captured by a single camera using deep learning, etc. Furthermore, in the case of (C) above, the acquisition unit 102 may calculate the depth of each feature point on the performer's body by performing analytical processing (image processing, etc.) on images captured by a single camera.

The processing unit 104 can generate video including animation of a virtual character (the broadcaster's avatar object) based on data related to the broadcaster's body from the acquisition unit 102. Regarding the video of the virtual character itself, the processing unit 104 can generate the video of the virtual character by having a rendering unit (not shown) perform rendering using various information (e.g., geometry information, bone information, texture information, shader information, blend shape information, etc.) stored in a character data storage unit (not shown).

Furthermore, the processing unit 104 can use data about the streamer's body from the acquisition unit 102 (data about the depth of each feature point on the performer's body) to generate a video in which the facial expression of the avatar object is changed (for example, a video in which the facial expression of the avatar object changes in synchronization with the movement of the performer's mouth and eyes, i.e., a video in which the facial expression of the avatar object changes in lip-sync and eye-tracking relative to the performer's face) using a variety of well-known techniques.

In addition, any other well-known technology may be used to generate a first video including animation of the broadcaster's avatar object based on motion data relating to the broadcaster's movements.

(2) Second moving image generator 110
The second moving image generating unit 110 can generate a second moving image related to the game based on operation data related to the distributor's operation using the web page received from the web server system 20B.

Specifically, the second video generation unit 110 executes a game program embedded in a web page received from the web server system 20B. This enables the second video generation unit 110 to render a second video (game video) relating to a game in which at least one game object operates, based on operation data indicating the distributor's operations input by the user interface unit 150.

Furthermore, the second video generation unit 110 can control at least one game object to be displayed in the second video based on gift information generated by the viewer terminal of each viewer watching the second video and transmitted to the user (broadcaster) of that terminal 10. Note that the gift information is information indicating that a gift will be given to the broadcaster.

(3) Communication unit 120
The communication unit 120 can communicate various information required for video distribution and/or viewing between the distribution server system 20A and the web server system 20B. For example, when the terminal 10 operates as a broadcaster terminal 11, the communication unit 120 can access the web server system 20B and receive a web page (HTML document) containing a game program required for game execution, etc. Furthermore, when the terminal 10 operates as a broadcaster terminal, the communication unit 120 can transmit the first video and/or the second video to the distribution server system 20A and receive gift information and/or comment information sent to the broadcaster from the distribution server system 20A.

Furthermore, when the terminal 10 operates as a viewer terminal, the communication unit 120 can receive the first video and/or the second video transmitted by the broadcaster terminal 11 from the distribution server system 20A, and transmit gift information and/or comment information for the broadcaster to the distribution server system 20A.

(4) Display section 130
The display unit 130 can display various information required for distribution and/or viewing of videos. For example, the display unit 130 can display the first video and/or the second video to be distributed and/or the first video and/or the second video that has been received.

(5) Storage unit 140
The storage unit 140 can store various information required for distribution and/or viewing of videos.

(6) User Interface Unit 150
The user interface unit 150 can input various information required for streaming and/or viewing of videos through user operations. For example, the user interface unit 150 can input operation data from the streamer indicating the content of the streamer's operations when playing a game, and output the operation data to the second video generation unit 110.

(7) Gift Processing Unit 160
The gift processing unit 160 can process gift information transmitted and received in connection with the distribution and/or viewing of a video. For example, when the terminal 10 operates as a distributor terminal 11, the gift processing unit 160 can process gift information sent to the distributor. When the terminal 10 operates as a viewer terminal 12, the gift processing unit 160 can process gift information to be sent by the viewer.

3-2. Functions of Server Device 20 An example of the functions of server device 20 will be described with reference to Fig. 4. Fig. 4 is a block diagram schematically showing an example of the functions of server device 20 shown in Fig. 1. As shown in Fig. 4, server device 20 includes, for example, a communication unit 200, a storage unit 210, a gift processing unit 220, a determination unit 230, and an extraction unit 240.

(1) Communication unit 200
The communication unit 200 can communicate various information required for video distribution and/or viewing between the distributor terminal 11 and/or the viewer terminal 12. For example, when the server device 20 operates as a distribution server system 20A, it can receive the first video and/or the second video from each distributor terminal 11 and distribute it to each viewer terminal 12. When the server device 20 operates as a web server system 20B, it can transmit a web page incorporating a game program to each accessing distributor terminal 11.

(2) Storage unit 210
The storage unit 210 can store various information required for streaming and/or viewing of videos. The storage unit 210 stores, for example, data indicating the relationship between a first streamer and other streamers and/or viewers. The storage unit 210 stores, for example, data related to the streaming history of collaborative streams conducted by the first streamer with other streamers. The data stored in the storage unit 210 is updated, for example, each time the terminal 10 performs a stream and/or views the stream. Note that, among the data stored in the storage unit 210, data related to at least one user who has performed a collaborative stream with the first streamer may be referred to as "first data." The first data is, for example, data related to users who have previously streamed games together with the first streamer and/or users who have streamed videos using their own avatars together with the avatar of the first streamer.

The data stored in the memory unit 210 includes, for example, information regarding at least one of the following: "collaboration information," "broadcast time/viewing time," "game progress," "match history," "number of followers," "special skills," and "gift information."

"Collaboration information" includes, for example, information indicating parameters of games previously streamed or viewed by other streamers who have streamed games in collaboration with the first streamer. Furthermore, "collaboration information" may include, for example, information regarding video streams using the first streamer's avatar and the other streamer's avatar. "Streaming time/viewing time" refers to the amount of time that users operating each terminal 10 have streamed or viewed games in the past. The storage unit 210 stores, for example, the cumulative amount of time that each user has streamed and/or viewed each game in the past.

"Game progress" indicates the progress of games that a broadcaster has previously broadcast. "Match history" indicates, for example, the number of wins, losses, and draws for users who participated in the game, if the game being played is a competitive game. "Match history" may also be, for example, a ranking of users who participated in the game, if the game being played is a competitive game. "Match history" may be a numerical value calculated by assigning a predetermined weight to each win, loss, draw, and ranking, and may be used as a score calculated from this.

"Number of followers" refers to the number of other users who follow a user. The more followers a user has, the greater the level of interest they receive from other users. "Special skills" refers to skills (special moves) possessed by the streamer's avatar. If the game being played is a competitive game, special skills include, for example, special moves that can be used in the game to inflict damage on the opponent. Note that these special moves inflict greater damage on the opponent than a normal attack would. Furthermore, these special moves have a greater effect than a normal attack would have. The memory unit 210 stores, for example, the name of the special skill, the conditions for using the special skill, etc.

"Gift information" refers to, for example, the number of gifts, gift value, gift type, gift level, etc., quantified according to a predetermined table, and the total value of each gift is listed. Furthermore, for example, "gift information" may include at least one of the time, frequency, and number of times gifts were given (sent) from the viewer terminal 12 to the broadcaster terminal 11. The number of gifts refers to the number of gifts given by viewers to the first broadcaster. Here, "gift value" refers to, for example, the price of a gift if it is paid, or the points consumed for that gift if it is free. Furthermore, "gift value" may be determined based on, for example, the usable time of the gift, the display time of the gift, and/or the number of people who can simultaneously use the gift, or the total number of gifts available in the game. A shorter usable time of the gift indicates a higher value of the gift. A longer usable time of the gift indicates a lower value of the gift. A longer display time of the gift indicates a higher value of the gift. A shorter display time of the gift indicates a lower value of the gift. Furthermore, the fewer the number of people who can use a gift at the same time, the higher the value of the gift. Furthermore, the more the number of people who can use a gift at the same time, the lower the value of the gift. Furthermore, the smaller the total number of gifts available in the game, the higher the value of the gift. Furthermore, the larger the total number of gifts available in the game, the higher the value of the gift.

(3) Gift Processing Unit 220
The gift processing unit 220 can process gift information sent from each viewer to each broadcaster. Note that, when the server device 20 operates as the web server system 20B, the gift processing unit 220 may be omitted.

(4) Determination unit 230
The determination unit 230 determines various conditions, such as whether a condition for starting a collaborative distribution has been met (S314 described below), whether a first condition has been met (S318 described below), and whether a predetermined time has elapsed (S326 described below). Note that the first condition is a condition for determining whether to end the collaborative distribution.

(5) Extraction unit 240
The extraction unit 240 extracts second data used to determine the processing content in S324 (described later) during or after the collaborative distribution. After the collaborative distribution ends, the extraction unit 240 extracts second data related to at least one user among at least one user related to the first broadcaster, based on first data related to at least one user who has previously performed a collaborative distribution with the first broadcaster.

3-3. Functions of the Studio Unit 30 As described above, the studio unit 30 is capable of performing the same functions as the terminal 10 and/or the server device 20. Therefore, the studio unit 30 is capable of having the functions of the terminal 10 described with reference to Fig. 3 and/or the functions of the server device 30 described with reference to Fig. 4.

4. Overall Operation of Video Distribution System 1 Next, the overall operation of the video distribution system 1 having the above configuration will be described with reference to Figures 5A and 5B. Figures 5A and 5B are flow diagrams showing an example of the operation performed in the video distribution system 1 shown in Figure 1. Note that in this embodiment, for example, the following description will be given assuming that there is a single user (second broadcaster) who collaborates with the first broadcaster on a broadcast; however, there may be multiple users who collaborate with the first broadcaster on a broadcast.

First, referring to FIG. 5A, in S302, in accordance with the operation of the distributor (first distributor), the terminal 10 (first distributor terminal 11A) launches and executes a video distribution application.

At S304, the first broadcaster terminal 11A generates a first video including animation of the first broadcaster's avatar object based on the motion data related to the first broadcaster's motion. As a result, the first video including animation of the first broadcaster's avatar object is displayed on the display unit 130 of the first broadcaster terminal 11A. The first broadcaster terminal 11A then transmits the generated first video to the distribution server device 20A1 of the distribution server system 20A. The distribution server device 20A1 distributes the first video received from the first broadcaster terminal 11A to each viewer terminal 12 running a video viewing application. As a result, the first video is also displayed on the display unit 130 of each viewer's viewer terminal 12.

At S306, when the first distributor taps or otherwise performs an action such as tapping on the "Game" icon 406 (not shown) displayed on the display unit 130 of the first distributor terminal 11A, the first distributor terminal 11A executes the browser function included in the video distribution application. As a result, at S308, the first distributor terminal 11A, while executing the video distribution application (i.e., having distributed the first video to each viewer terminal 12), can access the web server system 20B and receive an HTML5 document incorporating a game program.

At S310, the first broadcaster terminal 11A executes the game program embedded in the received HTML5 document to generate a video related to the game (second video).

In S312, the first broadcaster terminal 11A transmits the second video (game screen) generated in S310 to the distribution server device 20A1 of the distribution server system 20A. The distribution server device 20A1 distributes the second video received from the first broadcaster terminal 11A to each viewer terminal 12 running a video viewing application. This allows the second video to be displayed on the display unit 130 of each viewer terminal 12. For example, after the processing of S312 is executed, for example, after a predetermined time has elapsed, the processing of S314 is executed.

In S314, the server device 20 determines whether the conditions for starting collaborative distribution have been met. If the server device 20 determines in S314 whether the conditions for starting collaborative distribution have been met (YES in S314), the process proceeds to S316. On the other hand, if the server device 20 determines in S314 that the conditions for starting collaborative distribution have not been met (NO in S314), the process of S314 is performed again.

Note that the processing entity for S314 to S326 is, for example, game API server device 20A3 or game server device 20B1 among server devices 20, but is not limited to this and may be, for example, another device among server devices 20.

The conditions for starting a collaborative broadcast are met, for example, when the first broadcaster terminal 11A is broadcasting with the collaboration broadcast mode displayed on the display unit 130 enabled (ON), another terminal 10 sends data related to a collaborative broadcast application to the first broadcaster terminal 11A via the server device 20, and the first broadcaster approves the application by operating the first broadcaster terminal 11A. The first broadcaster can switch the collaboration broadcast mode between enabled (ON) and disabled (OFF) by operating the first broadcaster terminal 11A.

At S316, the server device 20 starts the collaborative distribution. After the collaborative distribution has started, for example, when a predetermined time has elapsed, the processing of S318 is performed. By starting the collaborative distribution at S316, a second video including animation of the avatar object of the first broadcaster and animation of the avatar object of a user other than the first broadcaster (e.g., a second broadcaster) is displayed on the display unit 130 of the terminal 10 operated by the user who is broadcasting and/or watching the game through the collaborative distribution.

In addition, after the server device 20 starts the collaborative distribution in S316, users who are broadcasting and/or watching the game through the collaborative distribution can input operation information and/or voice into their own terminal 10 to change the display content on the display screen of the terminal 10 and/or communicate voice to users operating other terminals 10. Also, after the server device 20 starts the collaborative distribution in S316, the terminals 10 of users who are broadcasting and/or watching the game through the collaborative distribution can read data such as live game content stored in the content server device 20B2.

Furthermore, in S316, after the server device 20 starts the collaborative distribution, operation information and/or voice input to the user's own terminal 10 who is broadcasting and/or viewing the game through the collaborative distribution may be stored in the server device 20 (e.g., game server device 20B1). This stored data may be used to determine the content of the specified processing in S324, which will be described later.

Furthermore, in S316, after the server device 20 starts the collaborative distribution, the server device 20 may configure the display content on the display unit 130 of the terminal 10 operated by the user who is distributing and/or watching the game through the collaborative distribution so that the display content is the same, or may configure the display content on the display unit 130 of the terminal 10 operated by the user who is distributing and/or watching the game through the collaborative distribution so that the display content is different. Note that if the display content on the display unit 130 of multiple broadcaster terminals 11 is configured to be different, a viewer who is watching the game through the collaborative distribution using a terminal 10 may configure the display content on the display unit 130 of their own terminal 10 to display the display content related to any one of the multiple broadcasters.

Furthermore, in S316, an example was described in which the first broadcaster and the second broadcaster jointly broadcast a game, but this is not limited to this, and any broadcast may be made jointly between the first broadcaster and another broadcaster. For example, in S316, instead of the first broadcaster and the second broadcaster jointly broadcasting a game, a video in which the avatar of the first broadcaster and the avatar of the second broadcaster appear together may be broadcast.

In S318, the server device 20 determines whether the first condition has been met. If the server device 20 determines in S318 that the first condition has been met (YES in S318), the collaborative distribution ends and S322 described in FIG. 5B is executed. On the other hand, if the server device 20 determines in S318 that the first condition has not been met (NO in S318), the processing of S318 is repeated.

The first condition may be, for example, a condition that is satisfied when third data, which is data requesting the end of the collaboration broadcast, is transmitted from a device 10 operated by a user who is performing and/or viewing the collaboration broadcast, or a condition that is satisfied when a predetermined time (first time) has passed since the start of the collaboration broadcast, or a condition that corresponds to the satisfaction of a condition in a game played through the collaboration broadcast. Here, the above-mentioned "request to end the collaboration broadcast" refers, for example, to a request made by the first broadcaster (host broadcaster) or the second broadcaster (guest) when a collaboration broadcast is being conducted using the avatar of the first broadcaster and the avatar of the second broadcaster. Furthermore, the above-mentioned "condition in the game" refers, for example, to a condition such as defeating a character other than the character being controlled in a stage provided in the game, obtaining a predetermined item in the stage, moving the character being controlled in the stage to a predetermined position, or the parameters of the character being controlled in the stage being above or below a predetermined value. Defeating a character other than the controlled character in a stage provided in the game, obtaining a specified item in that stage, or moving the controlled character to a specified position in that stage may be considered to have cleared that stage. Furthermore, the controlled character's parameters in that stage exceeding or falling below a specified value may be considered to have died.

Furthermore, when the determination in S318 is YES and the collaborative broadcast ends, for example, the first broadcaster terminal 11A is configured to allow the first broadcaster to continue broadcasting, and the second broadcaster terminal 11B is configured to prevent the second broadcaster from broadcasting. In this case, the second broadcaster terminal 11B may be configured to allow the second broadcaster to continue viewing the broadcast content related to the broadcast as a viewer.

At S322, the server device 20 extracts second data related to users other than the first broadcaster based on the first data. The first data is data related to at least one user who has previously performed a collaborative broadcast with the first broadcaster, and is stored, for example, in the storage unit 210. The second data includes, for example, at least one of data indicating parameters of games previously broadcast or viewed by users other than the first broadcaster, and data indicating the relationship between the first broadcaster and users other than the first broadcaster.

At S324, the server device 20 executes a predetermined process based on the second data extracted from the first data. Specifically, at S324, the server device 20 changes at least one current parameter of the first broadcaster's avatar based on the second data, or displays the first broadcaster's avatar and avatars of users other than the first broadcaster in the second video based on the second data. More specifically, for example, the server device 20 changes the parameters of the first broadcaster's avatar to the parameters of the second broadcaster's avatar based on the second data related to the second broadcaster, or displays the first broadcaster's avatar and the second broadcaster's avatar in the second video instead of displaying only the first broadcaster's avatar in the second video.

In S324, when the current parameters of the first broadcaster's avatar are changed to predetermined parameters (referred to as first parameters) based on the second data, these first parameters are parameters that are more advantageous in terms of game progression than the current parameters in the game. Specifically, when the second data indicates parameters related to a specific user, and the parameters of this specific user's avatar are more advantageous in terms of game progression than the parameters of the first broadcaster, the current parameters of the first broadcaster's avatar are changed to the parameters of this specific user's avatar. In addition, rather than changing the current parameters of the first broadcaster's avatar to the parameters of this specific user's avatar, some of the parameters of this specific user's avatar may be assigned to the first broadcaster's avatar.

Furthermore, in S324, when the avatar of the first broadcaster and the avatar of a user other than the first broadcaster are displayed in the second video based on the second data, the user in question is the user who has broadcast games together with the first broadcaster the most number of times, the user who has broadcast games together with the first broadcaster the longest cumulative time, the user who has watched games previously broadcast by the first broadcaster the most number of times, the user who has watched games previously broadcast by the first broadcaster the longest cumulative time, or the user with the highest level in the game.

Furthermore, in S324, when the avatar of the first broadcaster and the avatars of users other than the first broadcaster are displayed in the second video based on the second data, for example, the first broadcaster terminal 11A broadcasts the second video including the avatar of the first broadcaster and the avatars of users other than the first broadcaster to each viewer terminal 12. Note that the specific processing is the same as S304.

Next, in S326, the server device 20 determines whether or not to continue the predetermined processing executed in S324. If the server device 20 determines in S326 to continue the predetermined processing executed in S324 (YES in S326), the processing of S326 is repeated. If the server device 20 determines in S326 not to continue the predetermined processing executed in S324 (NO in S326), the series of processing shown in Figures 5A and 5B ends. Note that if the result in S326 is NO, the series of processing shown in Figures 5A and 5B may not end, and one of the processes described in Figures 5A and 5B (for example, the processing of S314) may be performed.

As described above, according to the present disclosure, first data related to at least one user who jointly streamed with a first streamer is stored before the game is played; after the first streamer and the at least one user jointly stream, it is determined whether the first condition is met; and if it is determined that the first condition is met, second data related to at least one of the at least one user is extracted based on the first data. During or after the game is played, at least one current parameter of the first streamer's avatar can be changed based on the second data, or the first streamer's avatar and at least one of the at least one user's avatars can be displayed in a second video based on the second data. Therefore, even if a second streamer who streams videos with the first streamer stops streaming and leaves, the interest of a game subsequently streamed by the first streamer is less likely to be diminished. In particular, the benefits of processing S324 are expected to be even greater when viewers are more interested in the second streamer who jointly streamed with the first streamer than in the first streamer.

5. Modifications In S324, if at least one current parameter of the first broadcaster's avatar is changed based on the second data, or if the first broadcaster's avatar and an avatar of a user other than the first broadcaster are displayed in the second video based on the second data, parameters of the user's (helper's) avatar may be changed. This parameter change may refer, for example, to parameters that are more advantageous in terms of game progression than the current parameters. Alternatively, instead of changing the parameters of the user's avatar, a gift may be given to the user. In this way, the user may be rewarded for being selected in S322 and S324. The degree of change in the parameters and the content of the gift may be determined based on the user's (helper's) level of contribution. For example, the greater the user's (helper's) level of contribution, the greater the degree of change in the parameters, so as to be more advantageous in terms of game progression. Furthermore, for example, the higher the degree of contribution of the user (helper), the greater the number of gifts, the higher the value of the gifts, the greater the variety of gifts, and/or the higher the level of the gifts.

In addition, while the example has been described in which, if S318 returns YES, S322 is executed and then S324 is executed, the server device 20 may count the number of viewers of the game streamed by the first broadcaster alone when S318 returns YES and before S322 is executed, or after S322 is executed and before S324 is executed. The server device 20 may then determine whether the number of viewers of the game streamed by the first broadcaster alone is equal to or greater than a predetermined number, and execute the subsequent process (S322 or S324) only if it is below the predetermined number. Alternatively, the server device 20 may determine whether the number of viewers of the game streamed by the first broadcaster alone is lower than the number of viewers during the collaborative broadcast by at least a predetermined number, and execute the subsequent process (S322 or S324) only if it is lower than the predetermined number. The number of viewers during the collaborative broadcast may be, for example, the maximum number of viewers, minimum number of viewers, or average number of viewers during the collaborative broadcast.

In addition, in the above explanation, an example was described in which the predetermined processing described in S324 is performed based on the second data when a second broadcaster who was broadcasting together with the first broadcaster stops broadcasting and leaves in S316, and the first broadcaster is broadcasting alone. However, this is not limited to this. For example, if there are multiple broadcasters (guests) broadcasting together with the first broadcaster in S316, and at least one of these guests stops broadcasting and leaves, and the first broadcaster and the remaining guests are both broadcasting a game, the processing of S322 and S324 may be performed.

Furthermore, in the above explanation, an example was described in which steps S302 to S318 shown in Figure 5A are executed in order, but this is not limited to this, and at least one of the processes from S304 to S312 may be omitted.

Furthermore, in the various embodiments described above, the distributor terminal 11 has been described as receiving an HTML (particularly HTML5) document incorporating a game program from the web server system 20B in order to generate a video related to a game (second video). However, instead of receiving an HTML document incorporating a game program from the web server system 20B, the distributor terminal 11 can also generate a second video including a screen related to a game by executing an installed video distribution application and receiving necessary information from a data server device (not shown) (or some other server device (not shown) included in the distribution server system 20A).

Furthermore, in the various embodiments described above, the broadcaster terminal 11 may be a smartphone used by a general user, or a dedicated terminal installed in a studio or the like. In this case, at least one of the server devices included in the broadcast server system 20A, namely the broadcast server device 20A1, the DB server device 20A2, and the game API server device 20A3 (and further the web server system 20B), may be installed alongside the dedicated terminal in the studio or the like.

Furthermore, in the various embodiments described above, the broadcaster terminal 11 generates a video including an animation that includes an avatar object of the broadcaster based on motion data related to the broadcaster's movements and audio data related to the broadcaster's voice, and transmits this video to the server device 20. However, the broadcaster terminal 11 may transmit motion data related to the broadcaster's movements and audio data related to the broadcaster's voice to the server device 20, and the server device 20 may generate a video based on this motion data and audio data and transmit it to each viewer terminal 12. This reduces the amount of information transmitted from the broadcaster terminal 11 to the server device 20.

Alternatively, the broadcaster terminal 11 may transmit motion data related to the broadcaster's movements and audio data related to the broadcaster's voice to a separately provided server device, which may then generate a video based on the motion data and audio data and return it to the broadcaster terminal 11, which may then transmit the received video to the server device 20. This may reduce the amount of information transmitted from the broadcaster terminal 11 to the separately provided server device 20.

In addition, the broadcaster terminal 11 may transmit motion data related to the broadcaster's movements and audio data related to the broadcaster's voice to each viewer terminal 12 via server device 20 or the separately provided server device, and each viewer terminal 12 may generate (render) and display a video based on the received motion data and audio data. This reduces the amount of information transmitted from server device 20 or the separately provided server device to each viewer terminal 12.

In addition, in either of the above cases, the generation of a video based on the movement data and audio data transmitted by the distributor terminal 11 may be shared among at least two of the server device 20, the separate server device, and each viewer terminal 12. In either case, the device responsible for generating the video may receive and store information (images, etc.) about the avatar object to be operated, for example, from the server device 20, etc.

Furthermore, in the various embodiments described above, the operations of each terminal 10 described above can also be performed by a studio unit 30 having a configuration similar to that of the terminal 10. In this case, the studio unit 30A illustrated in FIG. 1 can have the same functions as the terminal 10 for distributing videos. Similarly, the studio unit 30B illustrated in FIG. 1 can have the same functions as the terminal 10 for viewing videos. Furthermore, the operations of each server device 20 described above can also be performed by a studio unit 30 having the same configuration as that of the server device 20.

Furthermore, the various embodiments described above can be used in combination with each other as long as no contradictions or inconveniences arise.

The technology disclosed herein may also be applicable to, for example, role-playing games, simulation games, shooting games, action games, card games, mahjong, Go, Othello, puzzle games, quiz games, sports games, and/or board games. Here, the user (helper) associated with the second data extracted in S324 may assist the first broadcaster when playing a game, for example, as follows. For example, when the first broadcaster plays a role-playing game, simulation game, and/or shooting game, the user may demonstrate to the first broadcaster how to play the game. For example, when the first broadcaster plays an action game and engages in a battle with a specific character in the action game, the user may participate in the battle to support the first broadcaster. For example, when the first broadcaster plays a card game, the first broadcaster may be able to use special skills possessed by the user. For example, when the first broadcaster plays a sports game and engages in a match with a specific opponent (team) in the sports game, the user may participate in the match to support the first broadcaster. Furthermore, for example, if the first broadcaster plays a role-playing game, simulation game, shooting game, action game, card game, mahjong, Go, Othello, puzzle game, quiz game, sports game, and/or board game, the user provides information such as hints to help the first broadcaster play the game to their advantage.

In addition, in the second example, the first broadcaster terminal 11A of the first broadcaster executes the installed game application and accesses the distribution server system 20A, which also functions as a game server device, thereby broadcasting the screen of the game being played to each viewer's terminal device 10 via this distribution server system 20A. In these first and second examples, the first broadcaster terminal 11A of the first broadcaster is also able to generate a second video using viewing parameters received from the distribution server system 20A, as in the various embodiments described above.

Furthermore, in a third example, in the first or second example above, the first broadcaster terminal 11A of the first broadcaster can transmit the first broadcaster's operation data to the web server device 20B or the distribution server system 20A without generating a game screen. In this case, the web server device 20B or the distribution server system 20A can generate a game screen using the operation data, and the distribution server system 20A can distribute a second video including the generated game screen to each viewer's terminal device 10. In this third example, when the web server device 20B generates the game screen, the web server device 20B can use the viewing parameters received from the distribution server system 20A. When the distribution server system 20A generates the game screen, the distribution server system 20A can use the viewing parameters that it manages.

6. Various Aspects A computer program according to a first aspect "causes the processor to function as follows: when executed by at least one processor, the computer program broadcasts, via a communication line, a first video including action data related to actions of a first broadcaster or an animation of an avatar of the first broadcaster generated based on the action data, and a second video related to a game generated using a received web page based on operation data related to operations of the first broadcaster, to a viewer terminal of at least one viewer; before execution of the game, stores first data related to at least one user who jointly broadcasted with the first broadcaster; after the first broadcaster and the at least one user jointly broadcast, determines whether a first condition is met, and if it is determined that the first condition is met, extracts second data related to at least one of the at least one users based on the first data; during or after execution of the game, changes at least one current parameter of the avatar of the first broadcaster based on the second data; or causes the processor to display the avatar of the first broadcaster and an avatar of at least one of the at least one user in the second video based on the second data."
The computer program of the second aspect is the same as the first aspect described above, in which "the first data is data relating to at least one of at least one user who collaborated with the first broadcaster to broadcast a game, and at least one user who collaborated with the first broadcaster to broadcast using the first broadcaster's avatar and their own avatar."
The computer program of the third aspect is, in the first or second aspect, "causing the processor to function so that the second data includes at least one of data indicating parameters of games that the at least one user has previously streamed or watched, and data indicating the relationship between the at least one user and the first streamer."
A computer program according to a fourth aspect is, in any of the first to third aspects, that "when the avatar of the first broadcaster and the avatar of at least one of the at least one user are displayed in the second video based on the second data, at least one of the at least one user is the user who has jointly broadcast with the first broadcaster the most number of times, the user who has jointly broadcast with the first broadcaster the longest cumulative time, the user who has watched the most broadcasts by the first broadcaster, the user who has watched the longest cumulative time watching broadcasts by the first broadcaster, or the user with the highest level in the game."
The computer program of the fifth aspect is, in any of the first to fourth aspects, that "causes the processor to function so that the first condition is a condition indicating that third data has been transmitted from a broadcaster terminal of the first broadcaster or a broadcaster terminal of a broadcaster other than the first broadcaster, or a condition indicating that a first time has elapsed since the first broadcaster and the at least one user jointly started broadcasting."
The computer program of the sixth aspect is the same as that of the fifth aspect above, in which "the third data causes the processor to function so as to include data requesting that an avatar of at least one of the at least one user be displayed in the second video together with an avatar of the first broadcaster."
The computer program of the seventh aspect is, in any of the first to sixth aspects, "causing the processor to function to change parameters related to the at least one user's avatar when at least one current parameter of the first broadcaster's avatar is changed based on the second data, or when the first broadcaster's avatar and an avatar of at least one of the at least one user are displayed in the second video based on the second data."
The computer program of the eighth aspect is, in any of the first to seventh aspects, "causing the processor to function such that when the current parameter is changed to a first parameter based on the second data, the first parameter is a parameter that is more advantageous in the progression of the game than the current parameter."
A computer program according to a ninth aspect causes the processor to function as follows: "When executed by at least one processor, the computer program causes the processor to: receive, via a communication line, a first video including motion data related to the motions of a first broadcaster or an animation of an avatar of the first broadcaster generated based on the motion data; and a second video related to a game generated using the received web page based on operation data related to the operation of the first broadcaster; after the first broadcaster and the at least one user jointly broadcast, determine whether a first condition is met; and if it is determined that the first condition is met, receive a first video in which at least one current parameter of the avatar of the first broadcaster is changed based on second data extracted related to at least one of the at least one users based on first data related to the at least one user who jointly broadcast with the first broadcaster before the game was played; or receive a second video displaying the avatar of the first broadcaster and an avatar of at least one of the at least one user based on the second data."
A computer program according to a tenth aspect is the computer program according to any one of the first to ninth aspects, wherein "the communication line includes the Internet."
The computer program according to the eleventh aspect is any one of the first to tenth aspects, wherein "the at least one processor includes a central processing unit (CPU), a microprocessor, and/or a graphics processing unit (GPU)."
The computer program of the twelfth aspect is one in which "the at least one processor is installed in a server device, a smartphone, a tablet, a mobile phone, or a personal computer" in any of the first to eleventh aspects.
A method according to a thirteenth aspect is a method executed by at least one processor that executes computer-readable instructions, the method including the steps of: distributing, by the processor executing the instructions, a first video including action data related to the action of a first broadcaster or an animation of an avatar of the first broadcaster generated based on the action data; and a second video related to a game generated based on operation data related to the operation of the first broadcaster using a received web page, to a viewer terminal of at least one viewer via a communication line; and storing first data related to at least one user who jointly broadcasted the game with the first broadcaster before the game was played. a step of determining whether a first condition is met after the first broadcaster and the at least one user have jointly broadcast, and if it is determined that the first condition is met, a step of extracting second data related to at least one of the at least one users based on the first data, and a step of changing at least one current parameter of an avatar of the first broadcaster based on the second data during or after the game is played, or displaying an avatar of the first broadcaster and an avatar of at least one of the at least one user in the second video based on the second data."
The method according to the fourteenth aspect is "a method executed by at least one processor that executes computer-readable instructions, the method including the steps of: receiving, via a communication line, a first video including motion data related to a motion of a first broadcaster or an animation of an avatar of the first broadcaster generated based on the motion data, and a second video related to a game generated using the received web page based on operation data related to an operation of the first broadcaster; determining whether a first condition is met after the first broadcaster and the at least one user have jointly broadcasted; and, if it is determined that the first condition is met, receiving a first video in which at least one current parameter of the avatar of the first broadcaster has been changed based on second data extracted related to at least one of the at least one users based on first data related to the at least one user who jointly broadcasted with the first broadcaster before the game was played, or receiving a second video displaying the avatar of the first broadcaster and an avatar of at least one of the at least one user based on the second data."
A method according to a fifteenth aspect is the method according to the thirteenth or fourteenth aspect, wherein "the communication line includes the Internet."
The method according to the 16th aspect is any one of the 13th to 15th aspects, wherein "the at least one processor includes a central processing unit (CPU), a microprocessor, and/or a graphics processing unit (GPU)."
A server device according to a seventeenth aspect of the present invention includes at least one processor, wherein the processor transmits a first video, which includes action data related to the actions of a first broadcaster or an animation of an avatar of the first broadcaster generated based on the action data, and a second video related to a game, which is generated using the received web page based on operation data related to the operation of the first broadcaster, to a viewer terminal of at least one viewer via a communication line; stores first data related to at least one user who jointly broadcasted with the first broadcaster before the game is played; determines whether a first condition is met after the first broadcaster and the at least one user jointly broadcast, and if it is determined that the first condition is met, extracts second data related to at least one of the at least one user based on the first data; and, during or after the game is played, changes at least one current parameter of the avatar of the first broadcaster based on the second data, or displays the avatar of the first broadcaster and an avatar of at least one of the at least one user in the second video based on the second data.
The server device according to the eighteenth aspect is described as including at least one processor, wherein the processor transmits a web page including a computer program to a broadcaster terminal of the broadcaster configured to broadcast, via a communication line, a first video including action data related to the action of a first broadcaster or an animation of an avatar of the first broadcaster generated based on the action data, and a second video related to a game generated based on operation data related to the operation of the first broadcaster using the received web page, to a viewer terminal of at least one viewer, and the computer program controls the broadcaster terminal of the first broadcaster to broadcast the game in collaboration with the first broadcaster before the game is played. and storing first data relating to at least one user selected from the first broadcaster and the at least one user, determining whether a first condition is met after the first broadcaster and the at least one user have jointly broadcast, and if it is determined that the first condition is met, extracting second data relating to at least one of the at least one users based on the first data, and during or after the game is played, modifying at least one current parameter of an avatar of the first broadcaster based on the second data, or causing the processor to display an avatar of the first broadcaster and an avatar of at least one of the at least one user in the second video based on the second data.
The server device according to a nineteenth aspect is the server device according to the seventeenth or eighteenth aspect, wherein "the communication line includes the Internet."
The server device according to the twentieth aspect is any one of the seventeenth to nineteenth aspects, in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and/or a graphics processing unit (GPU)."

In this way, the technology disclosed herein can provide a computer program, method, and server device that reduces the possibility of the game's enjoyment being diminished.

1 Video distribution system 10 Terminal 11 Distributor terminal 11A First distributor terminal 11B Second distributor terminal 12, 12A, 12B, 12C Viewer terminal 20 Server device 20A Distribution server system 20A1 Distribution server device 20A2 DB server device 20A3 Game API server device 20A4 Collaboration server device 20B Web server system 20B1 Game server device 20B2 Content server device 30, 30A, 30B Studio unit

Claims (15)

When executed by at least one processor,
storing first data relating to at least one user who jointly broadcasted the game with the first user before the game was played;
a video relating to a game using a character object of the first user and a character object of at least one third user who is cooperating with the first user to distribute the game is made viewable on a second user terminal of at least one second user via a communication line;
determining whether or not a condition that the at least one third user has finished the game distribution is satisfied, and if it is determined that the condition is satisfied, extracting second data related to at least one user of the at least one user;
causing the processor to function to display a character object of at least one of the users in the video based on the second data;
Computer program. the first data is data relating to at least one of at least one user who has collaborated with the first user to stream a game, and at least one user who has collaborated with the first user to stream using the first user's character object and their own character object;
2. The computer program of claim 1 . causing the processor to function so that the second data includes at least one of data indicating parameters of a game that the at least one user has previously distributed or viewed, and data indicating a relationship between the at least one user and the first user;
2. The computer program of claim 1. causing the processor to function so as to change parameters relating to the character object of the at least one user when the character object of the at least one user is displayed in the video based on the second data;
2. The computer program of claim 1. When executed by at least one processor,
storing first data relating to at least one user who jointly broadcasted the game with the first user before the game was played;
receiving, via a communication line, a video relating to a game using a character object of the first user and a character object of at least one third user who is cooperating with the first user in game distribution ;
determining whether or not a condition that the at least one third user has finished the game distribution is satisfied, and if it is determined that the condition is satisfied, receiving a video displaying a character object of the at least one user based on second data extracted in relation to at least one user of the at least one user;
Computer program. The communication line includes the Internet.
6. A computer program according to claim 1 or 5 . the at least one processor comprises a central processing unit (CPU), a microprocessor, and/or a graphics processing unit (GPU);
6. A computer program according to claim 1 or 5 . the at least one processor:
It is installed in a server device, a smartphone, a tablet, a mobile phone, or a personal computer.
6. A computer program according to claim 1 or 5 . 1. A method performed by at least one processor executing computer readable instructions, comprising:
The processor executes the instructions to
storing first data relating to at least one user who jointly broadcasted the game with the first user before the game was played;
a step of making a video relating to a game using a character object of the first user and a character object of at least one third user who is jointly distributing the game with the first user viewable on a second user terminal of at least one second user via a communication line;
determining whether the at least one third user has completed the game distribution;
extracting second data related to at least one user of the at least one user when it is determined that the condition is satisfied;
and displaying a character object of at least one of the users in the video based on the second data.
method. 1. A method performed by at least one processor executing computer readable instructions, comprising:
The processor executes the instructions to
storing first data relating to at least one user who jointly broadcasted the game with the first user before the game was played;
receiving, via a communication line, video of a game using a character object of the first user and a character object of at least one third user who is cooperating with the first user in game distribution ;
determining whether or not a condition that the at least one third user has finished the game distribution is satisfied, and if it is determined that the condition is satisfied, receiving a video displaying a predetermined character object based on second data extracted in relation to at least one user of the at least one third user,
method. The communication line includes the Internet.
11. The method according to claim 9 or 10 . the at least one processor comprises a central processing unit (CPU), a microprocessor, and/or a graphics processing unit (GPU);
11. The method according to claim 9 or 10 . at least one processor;
the processor:
storing first data relating to at least one user who jointly broadcasted the game with the first user before the game was played;
transmitting a video relating to a game using the character object of the first user and the character object of at least one third user who is cooperating with the first user to distribute the game to a second user terminal of at least one second user via a communication line;
determining whether or not a condition that the at least one third user has finished the game distribution is satisfied, and if it is determined that the condition is satisfied, extracting second data related to at least one user of the at least one user;
displaying a character object of at least one of the users in the video based on the second data;
Distribution server. The communication line includes the Internet.
The distribution server according to claim 13 . the at least one processor comprises a central processing unit (CPU), a microprocessor, and/or a graphics processing unit (GPU);
The distribution server according to claim 13 .