JP7606776B2 - Distributor terminal, distribution server, distribution method, and program - Google Patents

Description

The present invention relates to technology for live streaming in VR space.

In recent years, services that allow users to become computer graphic characters (hereafter referred to as "avatars") in a virtual reality (VR) space and live stream the experience have become widespread.

With this service, streamers wear a head-mounted display (HMD) and hold controllers in both hands, and their own movements are reflected in an avatar as they live stream.

Viewers can not only watch live broadcasts, but also participate in the live broadcast by wearing an HMD. Viewers can add their own avatars to the broadcaster's VR space and communicate with the broadcaster.

JP 2018-7828 A JP 2018-94326 A

To liven up a live stream, viewers can have their viewer avatars perform cheering actions, but streamers cannot respond to all viewers. If viewers do not receive a reaction from the streamer, they may stop cheering, which could result in the stream losing its excitement.

Patent Document 1 describes how the player's avatar is placed in the audience in a VR space as one of many fans who have come to the idol character's live stage, and the player sends their gaze to cheer on the idol character. In Patent Document 1, the idol character provides special fan service to the avatar of a player who meets the activation conditions, such as points awarded for cheering. Patent Document 1 livens up the live stage by having players compete with each other to earn points. The special fan service is only provided to one player in the audience, and cheering does not necessarily result in a reaction from the idol character.

The present invention was made in consideration of the above, and aims to make it easier for viewers to receive reactions from broadcasters.

The broadcaster terminal of the present invention is a broadcaster terminal used for communication between a broadcaster avatar and a viewer avatar that exist in the same virtual space, and the broadcaster terminal and viewer terminal communicate with each other via a server, and is characterized in that it comprises a cheer detection unit that detects that a cheer signal indicating that the broadcaster avatar has been cheered on from a first viewer terminal , and a reaction unit that, when the broadcaster avatar reacts to the cheer, differentiates the reaction data transmitted between the first viewer terminal that transmitted the cheer signal and a second viewer terminal that did not transmit the cheer signal .

The present invention makes it easier for viewers to receive reactions from broadcasters.

FIG. 1 is a diagram illustrating an example of an overall system configuration for providing a live distribution service. FIG. 1 is a diagram illustrating a device for operating an avatar. FIG. 2 is a functional block diagram showing an example configuration of each device for providing a live distribution service. 11 is a sequence diagram showing a process flow in which a distributor starts live distribution and a viewer participates in the program. FIG. A diagram showing the state of a VR space in which a viewer participates. FIG. 13 is a sequence diagram showing the flow of a process for replacing a normal motion with a reaction motion when a viewer supports a distributor. A diagram showing a VR space in which a broadcaster reacts to viewers. FIG. 2 is a functional block diagram showing an example configuration of each device for providing a live distribution service. 13 is a flowchart showing a process flow in which a viewer terminal replaces a normal motion with a reaction motion. FIG. 13 is a sequence diagram showing the flow of a process for replacing a normal motion with a reaction motion when a viewer supports a distributor. FIG. 2 is a functional block diagram showing an example configuration of each device for providing a live distribution service. 13 is a flowchart showing a process flow in which a distributor terminal replaces a normal motion with a reaction motion. FIG. 13 is a sequence diagram showing the flow of a process for replacing a normal motion with a reaction motion when a viewer supports a distributor.

The following describes an embodiment of the present invention with reference to the drawings.

[First embodiment]
An example of the overall system configuration of a live distribution service according to a first embodiment will be described with reference to FIG.

The live streaming service of this embodiment is a live streaming service in which a broadcaster can become an avatar in a VR space and broadcast a live program. Furthermore, with this live streaming service, viewers can also participate in the program as viewer avatars in the same VR space as the broadcaster. The live streaming service is provided using a server 1, a broadcaster terminal 3, and a viewer terminal 5 connected to a network. In FIG. 1, the number of viewer terminals 5 is five, but in reality the number of viewer terminals 5 is more, and the number of viewer terminals 5 is arbitrary.

The server 1 receives live streaming video in the VR space from the distributor terminal 3 via the network, and distributes the live streaming video to the viewer terminal 5. Specifically, the server 1 receives motion data of the distributor avatar from the distributor terminal 3, and distributes the motion data to the viewer terminal 5. The viewer terminal 5 reflects the received motion data in the distributor avatar to render the VR space. When the VR space is rendered and displayed on the viewer terminal 5, the viewer terminal 5 holds model data necessary for rendering the VR space. For example, the viewer terminal 5 may receive model data such as avatars present in the VR space from the server 1, or may hold the model data in advance.

When a viewer joins a program, the server 1 receives motion data of the viewer avatar from the viewer terminal 5 and distributes the motion data to the distributor terminal 3 and other viewer terminals 5. The distributor terminal 3 and other viewer terminals 5 reflect the received motion data in the viewer avatar to render the VR space. The model data of the viewer avatar may be received from the server 1 when the viewer joins, or may be stored in advance by the distributor terminal 3 and viewer terminal 5.

Viewers who join the program can support the broadcaster through their viewer avatars. The broadcaster's avatar will respond to the viewer avatar who is supporting them.

When the broadcaster avatar reacts to a cheer, the server 1 makes the motion that the broadcaster avatar performs differently for the cheering viewer avatar and the non-cheering viewer avatar. Specifically, the server 1 distributes motion data in which part of the broadcaster avatar's motion has been changed to a reaction motion to the viewer terminal 5 of the cheering viewer, and distributes normal motion data to the viewer terminal 5 of the non-cheering viewer. In other words, when the broadcaster avatar reacts, the motion of the broadcaster avatar seen by the cheering viewer is different from that seen by the non-cheering viewer.

The distributor terminal 3 is a terminal used by the distributor for live distribution. For example, a personal computer connected to an HMD can be used as the distributor terminal 3.

As shown in FIG. 2, the broadcaster wears the HMD 100 and holds the controller 101 in both hands to control the broadcaster avatar.

The HMD 100 detects the streamer's head movements. The head movements detected by the HMD 100 are reflected in the streamer's avatar. The streamer can look around the VR space by moving their head. The HMD 100 renders the VR space in the direction the streamer is facing. The HMD 100 creates parallax between the image for the right eye and the image for the left eye, allowing the streamer to view a three-dimensional VR space.

The controller 101 detects the distributor's hand movements. The hand movements detected by the controller 101 are reflected in the distributor's avatar.

The distributor terminal 3 transmits motion data to the server 1, which reflects the detected movements of the distributor in the distributor avatar. The motion data is distributed by the server 1 to the viewer terminal 5. The distributor terminal 3 also receives motion data of viewer avatars who have participated in the distributor's program from the server 1. The distributor terminal 3 reflects the received motion data in the viewer avatars and renders the VR space.

The controller 101 is equipped with an operating means such as a button. When the broadcaster operates the button, the broadcaster avatar can be made to perform a predetermined movement. An example of an operation related to this embodiment is an operation to specify the timing of a reaction to a viewer's cheering. When a viewer avatar cheers for a broadcaster avatar, and the broadcaster specifies the timing of the reaction, the broadcaster terminal 3 transmits a reaction signal to the server 1. In response to the reaction signal, the server 1 makes the broadcaster avatar react to the cheering by making the reaction motion for the cheering viewer avatar different from the motion for the non-cheering viewer avatar.

The viewer terminal 5 is a terminal that a viewer uses to watch a live broadcast. As with the broadcaster terminal 3, the viewer terminal 5 can be a personal computer connected to an HMD.

Viewers can participate in a broadcaster's program. Specifically, when a viewer performs an operation to participate in a program, a viewer avatar that the viewer can control appears in the VR space in which the broadcaster avatar exists. As shown in FIG. 2, the viewer also wears the HMD 100 and holds the controller 101 in both hands to control the viewer avatar. The viewpoint of the viewer who participates in the program becomes the viewpoint of the viewer avatar in the VR space. In other words, the HMD 100 renders the VR space from the viewpoint of the viewer avatar.

Viewers can cheer on the broadcaster by operating the controller 101 or the like. When a viewer cheers on the broadcaster, the viewer terminal 5 transmits a cheering signal to the server 1.

Next, referring to FIG. 3, we will explain example configurations of the server 1, the distributor terminal 3, and the viewer terminal 5.

The server 1 shown in FIG. 3 includes a distribution unit 11, a reaction unit 12, a cheer detection unit 13, and a reaction data storage unit 14.

The distribution unit 11 transmits and receives data necessary for the live distribution service. Specifically, the distribution unit 11 receives the voice of the distributor and motion data of the distributor avatar from the distributor terminal 3, and distributes the voice and motion data to the viewer terminal 5. The distribution unit 11 also receives motion data of the viewer avatar from the viewer terminal 5 of the viewer who participated in the program, and distributes the motion data to the distributor terminal 3 and the viewer terminal 5. The distribution unit 11 may receive and distribute the voice of the viewer. The distribution unit 11 may receive and distribute comments (text information) entered by the viewer.

The reaction unit 12 replaces the normal motion sent to the viewer terminal 5 of the cheering viewer with a reaction motion at a predetermined timing. The normal motion is a motion according to the movement of the broadcaster detected by the HMD 100 and the controller 101. The reaction motion is a reaction motion stored in the reaction data storage unit 14 described later. The reaction unit 12 reads the reaction motion from the reaction data storage unit 14, converts it into an individual reaction motion for the cheering viewer avatar, and transmits it to the viewer terminal 5 of the cheering viewer. For example, an example of a reaction performed by the broadcaster avatar is a motion in which the broadcaster avatar turns its gaze (face) toward the viewer avatar and winks. Since the VR space is rendered from the viewer avatar's line of sight, the viewer can feel that the broadcaster avatar has turned its gaze toward them in response to their cheering. When the broadcaster avatar turns its gaze toward the viewer avatar, an effect such as a beam or a heart mark may be displayed in the direction of the gaze.

Other examples of reactions include turning the body of the broadcaster avatar towards the viewer avatar, or moving the broadcaster avatar closer to the viewer avatar. Reactions may be changed depending on the amount of support from the viewer (e.g., the amount of gifts sent). Reaction motions are selected from motion data stored in the reaction data storage unit 14, which will be described later. During reaction motions, the broadcaster avatar may not face towards viewer avatars that are not supporting.

The reaction unit 12 determines the timing for making the distributor avatar react according to a reaction signal received from the distributor terminal 3. Alternatively, without depending on a reaction signal, the reaction unit 12 may determine the reaction timing to be the timing when a predetermined time has elapsed after the detection of a cheer. The reaction unit 12 may determine the reaction timing according to the movement of the distributor avatar. For example, the reaction timing may be when the distributor avatar raises both hands up.

The distributor may also transmit reaction motions. In this case, the reaction motions received from the distributor terminal 3 are transmitted to the viewer terminals 5 of the viewers who are supporting the user, and the reaction motions are replaced with normal motions and transmitted to the viewer terminals 5 of the viewers who are not supporting the user.

The cheer detection unit 13 detects a viewer who has cheered for a broadcaster, and notifies the reaction unit 12 to make a reaction motion towards the broadcaster avatar of the broadcaster who cheered for the viewer. For example, the cheer detection unit 13 detects a viewer's cheering by receiving a cheering signal from the viewer terminal 5. The cheer detection unit 13 may detect cheering for the broadcaster based on the movement of the viewer avatar. For example, the cheer detection unit 13 detects that a viewer has cheered for a broadcaster when the viewer avatar waves its hand widely towards the broadcaster avatar. Alternatively, the cheer detection unit 13 may detect that a viewer has cheered for a broadcaster when cheering by the viewer is detected.

The reaction data storage unit 14 stores data on reaction motions. The reaction data storage unit 14 may store multiple types of reaction motions.

The distributor terminal 3 has a VR function unit 31 and a reaction instruction unit 32.

The VR function unit 31 has functions necessary for live streaming in a VR space, such as a function for rendering the VR space and a function for reflecting the streamer's movements in a streamer avatar in the VR space. For example, the VR function unit 31 determines the motion of the streamer avatar based on the streamer's head movements detected by the HMD 100 and the streamer's hand movements detected by the controller 101, and transmits the motion data to the server 1.

The reaction instruction unit 32 accepts a reaction operation from the broadcaster and transmits a reaction signal to the server 1. For example, when a button on the controller 101 is operated, a reaction signal is transmitted to the server 1. If the server 1 determines the timing to respond to support, the broadcaster terminal 3 does not need to accept a reaction operation from the broadcaster.

The viewer terminal 5 has a VR function unit 51 and a cheering instruction unit 52.

The VR function unit 51 has functions necessary for viewing live broadcasts in a VR space, such as the function of rendering the VR space and the function of reflecting the viewer's movements in the viewer's avatar in the VR space.

The cheering instruction unit 52 accepts cheering operations from the viewer and transmits cheering signals to the server 1. The cheering instruction unit 52 causes the viewer avatar to perform a cheering motion as necessary.

A viewer may show support by, for example, sending a gift or waving. When a viewer sends a gift, a support signal is sent from the support instruction unit 52 to the server 1. A support operation may be assigned to a button on the controller 101, and when the viewer operates that button, a support signal may be sent and the viewer avatar may be made to perform a support motion.

Each part of the server 1, the distributor terminal 3, and the viewer terminal 5 may be configured as a computer equipped with an arithmetic processing unit, a storage device, etc., and the processing of each part may be executed by a program. This program is stored in a storage device equipped in the server 1, the distributor terminal 3, and the viewer terminal 5, and can be recorded on a recording medium such as a magnetic disk, an optical disk, or a semiconductor memory, or can be provided via a network.

Next, we will explain how to start a live broadcast and how viewers join, with reference to the sequence diagram in Figure 4.

When the broadcaster performs an operation to start live broadcasting using the broadcaster terminal 3, the broadcaster terminal 3 notifies the server 1 that live broadcasting is to start (step S11).

When the live broadcast begins, the broadcaster terminal 3 detects the broadcaster's operations and movements, and transmits motion data for controlling the broadcaster avatar to the server 1 (step S12). The server 1 transmits the motion data to the viewer terminals 5A-5C that are watching the live broadcast. During the live broadcast, the broadcaster terminal 3 continues to transmit the motion data to the server 1, and the server 1 continues to transmit the motion data to the viewer terminals 5A-5C.

When viewer A performs an operation to participate in the program using viewer terminal 5A, viewer terminal 5A notifies server 1 that he or she will be participating in the program (step S13).

When viewer A joins the program, viewer terminal 5A detects the viewer's operations and movements, and transmits motion data for controlling the viewer's avatar to server 1 (step S14). Server 1 distributes the motion data to distributor terminal 3 and other viewer terminals 5B and 5C. While viewer A is participating in the program, the transmission of motion data from viewer terminal 5A to server 1 and the distribution of motion data from server 1 to distributor terminal 3 and viewer terminals 5B and 5C continue.

As in the case of viewer A, when viewer B performs an operation to participate in the program using viewer terminal 5B, viewer terminal 5B notifies server 1 that he or she will participate in the program (step S15) and starts transmitting motion data for controlling viewer B's viewer avatar to server 1 (step S16).

When viewer C performs an operation to participate in the program using viewer terminal 5C, viewer terminal 5C notifies server 1 that he or she will participate in the program (step S17) and starts transmitting motion data for controlling viewer C's viewer avatar to server 1 (step S18).

Figure 5 shows the VR space in which viewers are participating. As shown in the figure, in the VR space, not only is there a distributor avatar 300 on the stage, but below the stage there are also viewer avatars 500A-500E who have participated in the program. Viewers can see not only the distributor avatar 300, but also the viewer avatars 500A-500E of viewers A-E who have participated in the program.

The distributor can view the VR space from the position of distributor avatar 300 as a viewpoint. Viewers A to E can view the VR space from the positions of each of viewer avatars 500A to 500E as a viewpoint. In other words, the distributor terminal 3 renders the VR space based on the facial position and facial direction of distributor avatar 300, and the viewer terminal 5 renders the VR space based on the facial position and facial direction of each of viewer avatars 500A to 500E. In the case of Figure 5, viewers A to E will each view distributor avatar 300 from different positions and different angles.

The viewpoint of viewers who are not participating in the program is, for example, the position of a virtual camera set by the distributor at a position of their choice in the VR space.

Next, we will explain the support from viewers and the reactions from the broadcaster with reference to the sequence diagram in Figure 6.

When viewer A operates viewer terminal 5A to cheer on the broadcaster, viewer terminal 5A transmits a cheering signal to server 1, and server 1 notifies broadcaster terminal 3 that viewer A has cheered (step S21). At this time, it is assumed that viewers B and C are not cheering on the broadcaster.

When the broadcaster confirms that viewer A has cheered for him/her, he/she operates the broadcaster terminal 3 to send a reaction signal to the server 1 (step S22). The broadcaster performs a reaction operation to send a reaction signal, but the broadcaster does not need to react himself/herself. The broadcaster terminal 3 sends a normal motion according to the broadcaster's movement to the server 1 (step S23). Note that if the reaction timing is determined by the server 1, the process of sending a reaction signal in step S22 is not necessary.

When the server 1 receives the reaction signal, it replaces the normal motion with a reaction motion and transmits it to the viewer terminal 5A that transmitted the cheering signal (step S24), and transmits the normal motion to the viewer terminals 5B and 5C that did not transmit the cheering signal (steps S25 and S26). If there are multiple viewer terminals 5 that transmitted cheering signals, it transmits a dedicated reaction motion to each of the viewer terminals 5 that transmitted cheering signals.

Figure 7 shows the VR space where the distributor reacts. In this figure, viewers A and E (viewer avatars 500A and 500E) are seen cheering.

When the distributor avatar 300 responds to the cheering, the distributor avatar 300 reacts to each of the cheering viewer avatars 500A and 500E individually. Specifically, when the distributor avatar 300 reacts, the server 1 transmits a reaction motion dedicated to the viewer avatar 500A to the viewer terminal 5A of the viewer avatar 500A, and transmits a reaction motion dedicated to the viewer avatar 500E to the viewer terminal 5E of the viewer avatar 500E. A common normal motion is transmitted to the viewer terminals 5B, 5C, and 5D of the viewer avatars 500B, 500C, and 500D. In the VR space seen by viewer A, the distributor avatar is in the state of distributor avatar 300A approaching viewer avatar 500A. In the VR space seen by viewer E, the distributor avatar is in the state of distributor avatar 300E approaching viewer avatar 500E. In the VR space seen by viewers B, C, and D, the broadcaster avatar is in the state of broadcaster avatar 300 located in the center of the stage. In other words, when the broadcaster avatar reacts, viewer A sees broadcaster avatar 300A making a reaction motion dedicated to viewer avatar 500A, viewer E sees broadcaster avatar 300E making a reaction motion dedicated to viewer avatar 500E, and viewers B, C, and D see broadcaster avatar 300 in a normal motion. When the broadcaster avatar finishes reacting, the normal motion of the common broadcaster avatar is broadcast to viewer terminals 5A to 5E.

Note that the reaction motion does not have to involve moving the position of the distributor avatar 300, and may involve a motion such as directing one's gaze or waving at each of the viewer avatars 500A, 500E. In this case as well, a dedicated reaction motion is sent to each of the cheering viewer avatars 500A, 500E.

In this way, when the distributor avatar 300 reacts to a cheer, the distributor avatar 300A, 300E uses a dedicated reaction motion for the cheering viewer avatar 500A, 500E, which is different from the motion it uses for the non-cheering viewer avatars 500B, 500C, 500D, so that each of the viewers A, E can get a reaction from the distributor avatar.

Second Embodiment
In the second embodiment, a viewer terminal performs a process of replacing a normal motion of a distributor avatar with a reaction motion. The overall system configuration is similar to that of the first embodiment.

With reference to FIG. 8, we will explain an example configuration of the server 1, distributor terminal 3, and viewer terminal 5 used in the live streaming service of the second embodiment.

The server 1 includes a distribution unit 11. The distribution unit 11 transmits and receives data required for the live distribution service, similar to the distribution unit 11 of the server 1 in the first embodiment.

The distributor terminal 3 includes a VR function unit 31. The VR function unit 31 has the functions necessary for live distribution in a VR space, similar to the VR function unit 31 of the distributor terminal 3 in the first embodiment.

The viewer terminal 5 includes a VR function unit 51, a cheering instruction unit 52, a reaction unit 53, and a reaction data storage unit 54.

The VR function unit 51 has the functions necessary for live distribution in a VR space, similar to the VR function unit 31 of the viewer terminal 5 in the first embodiment. Furthermore, the VR function unit 51, in response to instructions from the reaction unit 53, replaces the normal motion of the distributor avatar received from the server 1 with a reaction motion, and renders the VR space.

The cheering instruction unit 52 accepts cheering operations from the viewer and notifies the reaction unit 53. The cheering instruction unit 52 causes the viewer avatar to perform a cheering motion as necessary. This embodiment differs from the first embodiment in that a cheering signal is not transmitted to the server 1. Note that the cheering instruction unit 52 may transmit a cheering signal to the server 1, as in the first embodiment.

When the cheering instruction unit 52 inputs a cheering operation, the reaction unit 53 replaces the motion of the broadcaster avatar delivered from the server 1 with a reaction motion at a predetermined timing. The predetermined timing is determined by the reaction unit 53. One example of the predetermined timing is the timing when the cheering motion of the broadcaster avatar ends. To determine the reaction timing, a reaction signal may be received from the server 1 or the broadcaster terminal 3.

The reaction data storage unit 54 stores reaction motion data, similar to the reaction data storage unit 14 of the server 1 in the first embodiment.

Next, the processing of the viewer terminal 5 will be explained with reference to the flowchart in FIG. 9.

When the cheering instruction unit 52 inputs a cheering operation, it causes the viewer avatar to cheer on the broadcaster avatar and notifies the reaction unit 53 that the broadcaster has been cheered on (step S31). The broadcaster may also be notified that the viewer has cheered on.

The reaction unit 53 determines whether it is time to react (step S32).

When the reaction timing arrives, the reaction unit 53 replaces the motion of the distributor avatar distributed from the server 1 with the reaction motion stored in the reaction data storage unit 14 (step S33).

Next, we will explain the reactions of viewers to cheering by referring to the sequence diagram in Figure 10.

The viewer terminal 5A receives a cheering operation from viewer A (step S41).

The distributor terminal 3 transmits the normal motion to the server 1 (step S42), and the server 1 distributes the normal motion to the viewer terminals 5A to 5C (steps S43 to S45).

When the reaction timing arrives, the viewer terminal 5A replaces the normal motion delivered from the server 1 with the reaction motion (step S46).

In this way, the viewer terminal 5A can detect cheering and replace the motion of the distributor avatar with a reaction motion on the viewer terminal 5A itself, allowing the viewer to see the distributor avatar's reaction.

[Third embodiment]
In the third embodiment, the distributor terminal performs a process of replacing the normal motion of the distributor avatar with a reaction motion. In the third embodiment, the distributor terminal 3 performs live distribution directly to the viewer terminal 5 without going through the server 1. Of course, live distribution may also be performed through the server 1. Note that, in the second embodiment, live distribution may also be performed without going through the server 1.

With reference to FIG. 11, a configuration example of a distributor terminal 3 and a viewer terminal 5 used in the live streaming service of the third embodiment will be described. In FIG. 11, the number of viewer terminals 5 is one, but in reality, the distributor terminal 3 performs live streaming to multiple viewer terminals 5.

The broadcaster terminal 3 includes a VR function unit 31, a reaction instruction unit 32, a reaction unit 33, a cheer detection unit 34, and a reaction data storage unit 35.

The VR function unit 31 has the functions necessary for live streaming in a VR space, similar to the VR function unit 31 of the broadcaster terminal 3 in the first embodiment.

The reaction instruction unit 32 accepts reaction operations from the distributor and notifies the reaction unit 33.

The reaction unit 33 replaces the normal motion sent to the viewer terminal 5 of the cheering viewer with a reaction motion in response to an instruction from the reaction instruction unit 32.

The cheering detection unit 34 detects viewers who cheered on a distributor and notifies the reaction unit 33 of the distributor who cheered on the viewer.

The reaction data storage unit 35 stores reaction motion data, similar to the reaction data storage unit 14 of the server 1 in the first embodiment.

Next, the processing of the distributor terminal 3 will be explained with reference to the flowchart in FIG. 12.

When the cheer detection unit 34 detects cheers from a viewer, it notifies the reaction unit 33 (step S51), and the reaction unit 33 waits for instructions from the reaction instruction unit 32 (step S52).

When the reaction instruction unit 32 accepts a reaction operation and notifies the reaction unit 33, the reaction unit 33 transmits the reaction motion stored in the reaction data storage unit 35 to the viewer terminals 5 of the viewers who cheered (step S53), and transmits the normal motion to the viewer terminals 5 of the viewers who did not cheer (step S54).

Next, we will explain the reactions of viewers to cheering by referring to the sequence diagram in Figure 13.

When viewer A operates viewer terminal 5A to cheer on the broadcaster, viewer terminal 5A transmits a cheering signal to broadcaster terminal 3 (step S61).

When the distributor confirms that viewer A has supported him/her, he/she inputs a reaction operation into the distributor terminal 3 (step S62).

The distributor terminal 3 transmits a reaction motion to the viewer terminal 5A that transmitted the cheering signal (step S63), and transmits normal motion to the other viewer terminals 5B and 5C (steps S64 and S65).

The above are the various embodiments for real-time distribution, but each embodiment can also be applied to distribution of programs in a VR space that has been distributed in the past or that has been prepared for distribution (also called "time-shift distribution"). In time-shift distribution, data for rendering the VR space, including motion data of the distributor's avatar, is stored in a server or other configuration. In response to a request from a viewer terminal, the server distributes data required for playing the program in the VR space (e.g., the voice of the distributor and motion data of the distributor's avatar, etc.) to the viewer terminal. The viewer terminal renders the VR space based on the received data and plays the program. In time-shift distribution, operations such as pausing, rewinding, and fast-forwarding may be possible.

Although data of a program to be time-shifted is stored in advance, by applying each embodiment, it is possible to make the broadcaster avatar react to the cheers of the viewers. When reaction motion data is stored in the viewer terminal as in the second embodiment, the viewer terminal, upon detecting a cheering operation by the viewer, immediately or after a predetermined time has elapsed, reads out the reaction motion data of the broadcaster avatar stored in advance in the viewer terminal, and replaces the motion data of the broadcaster avatar distributed from the server with the reaction motion data. This allows the viewer to see an effect in which the broadcaster responds to the cheers of the viewers. A section in which the motion of the broadcaster avatar is prohibited from being replaced may be set. For example, the motion replacement is prohibited while the broadcaster avatar is singing, and the broadcaster avatar is made to perform a reaction motion during the interlude. In time-shifted distribution, if the viewer is not participating in the program, the reaction motion of the broadcaster avatar may be directed toward the virtual camera when the viewer terminal renders the VR space.

When time-shift distribution is applied to the first and third embodiments, similarly to the above, the server that stores the reaction motion data or the distributor terminal detects the cheering from the viewer and replaces the motion of the distributor avatar with a reaction motion and transmits it only to that viewer terminal.

As described above, according to this embodiment, cheering by a viewer avatar for a broadcaster avatar is detected, and when the broadcaster avatar reacts to the cheering, it performs a reaction motion for the cheering viewer avatar and a normal motion, which is not a reaction motion, for the viewer avatars who are not cheering. As a result, a reaction motion is performed individually for the cheering viewer avatar, so that viewers can get a reaction from the broadcaster more reliably.

In each embodiment, the motion data, appearance data, bone data, etc. of the VR space including the replaced data are sent from the server 1 or the distributor terminal 3 to the viewer terminal 5, and rendering is performed at the viewer terminal 5 that receives the data. However, in both the case of real-time distribution and time-shift distribution, the server 1 or the distributor terminal 3 may render the VR space. For example, in the first embodiment in which the server 1 replaces the motion, the distributor terminal 3 or the viewer terminal 5 may transmit information such as the position and shooting direction of the virtual camera to the server 1, and the server 1 may render the VR space based on the information of the virtual camera to generate an image, and transmit the generated image to the viewer terminal 5. In either configuration, the viewer terminal 5 that detects the cheering can view the rendered image after replacing the motion with a reaction motion. Similarly, in the third embodiment, the distributor terminal 3 may render the VR space and transmit the image.

1...Server 11...Distribution unit 12...Reaction unit 13...Cheering detection unit 14...Reaction data storage unit 3...Distributor terminal 31...VR function unit 32...Reaction instruction unit 33...Reaction unit 34...Cheering detection unit 35...Reaction data storage unit 5...Viewer terminal 51...VR function unit 52...Cheering instruction unit 53...Reaction unit 54...Reaction data storage unit 100...HMD 101...Controller

Claims (7)

A distributor terminal used for communication between a distributor and a viewer using a distributor avatar and a viewer avatar that exist in the same virtual space,
The distributor terminal and the viewer terminal communicate with each other via a server,
a cheering detection unit that detects that a cheering signal indicating that cheering has been performed for the distributor avatar from the first viewer terminal ;
a reaction unit that, when the broadcaster avatar reacts to a cheer, transmits different reaction data between the first viewer terminal that transmitted the cheer signal and a second viewer terminal that did not transmit the cheer signal . The distributor terminal according to claim 1, characterized in that the reaction unit differentiates the reaction data to be sent to the first viewer terminal from the reaction data to be sent to the second viewer terminal at a timing instructed by the distributor controlling the distributor avatar. The distributor terminal according to claim 1 or 2, characterized in that the reaction unit uses reaction data held by the distributor terminal as either the reaction data to be transmitted to the first viewer terminal or the reaction data to be transmitted to the second viewer terminal . 4. The distributor terminal according to claim 1, wherein the reaction unit changes reaction data to be transmitted to the first viewer terminal in accordance with an amount of cheering signals received from the first viewer terminal . A distribution server used for communication between a distributor and a viewer using a distributor avatar and a viewer avatar that exist in the same virtual space,
a distribution unit that receives motion of the distributor avatar from a distributor terminal and distributes the motion, and receives motion of the viewer avatar from a viewer terminal and distributes the motion;
a cheering detection unit that receives a cheering signal from the viewer terminal to the distributor avatar;
a reaction unit that, when transmitting reaction data to the cheering of the distributor avatar, differentiates reaction data to be distributed to a viewer terminal that has transmitted the cheering signal from reaction data to be distributed to a viewer terminal that has not transmitted the cheering signal ;
The reaction unit distributes the reaction data at a timing when a predetermined time has elapsed after receiving the support signal.
A distribution server comprising: A distribution method using a distributor terminal used for communication between a distributor and a viewer using a distributor avatar and a viewer avatar that exist in the same virtual space, comprising:
The distributor terminal and the viewer terminal communicate with each other via a server,
detecting a reception of a cheering signal indicating that a cheer has been given to the distributor avatar from the first viewer terminal ;
and when the distributor avatar reacts to the cheering, the reaction data transmitted to the first viewer terminal which transmitted the cheering signal is different from the reaction data transmitted to the second viewer terminal which did not transmit the cheering signal . A program for operating a computer as each part of a distributor terminal used for communication between a distributor and a viewer using a distributor avatar and a viewer avatar that exist in the same virtual space,
The distributor terminal and the viewer terminal communicate with each other via a server,
On the computer,
A process of detecting that a cheering signal indicating that a cheer has been given to the distributor avatar from a first viewer terminal is received ;
A process of causing the distributor avatar to transmit different reaction data between the first viewer terminal that transmitted the cheering signal and a second viewer terminal that did not transmit the cheering signal when the distributor avatar reacts to the cheering signal;
A program characterized by executing the above.

Images