JP7515280B2 - program - Google Patents

Description

The present invention relates to a program .

Patent Document 1 describes an information processing system that distributes data for displaying content such as live video and game screens on a terminal device to the terminal device. In this information processing system, a display screen is displayed on the terminal device, which is configured with a content display area in which content distributed from a content server is displayed, and a background area other than the content display area. In the background area, images including avatars that display decorations, tips, and messages in speech bubbles are displayed.

JP 2012-120098 A

However, in the system of Patent Document 1, objects such as decorations and tips are displayed in the background area, making it difficult to create a sense of unity between the delivered content and the objects. In addition, when decorations, tips, and other objects are displayed in the content display area, there is a risk that the objects will impede the visibility of the content.

The present invention has been devised in view of the above-mentioned circumstances, and its purpose is to provide a program that can create a sense of unity between objects and content while preventing the objects from reducing the visibility of the content.

According to one aspect of an embodiment disclosed herein, there is provided a program executed on a computer , which causes the computer to function as a display means for displaying an image of a virtual space including a character that moves in response to the movements of an actor, a selection means for selecting an object to be displayed in the virtual space from among a plurality of types of objects having at least different display sizes, and an identification means for identifying a position in the virtual space where the object selected by the selection means is to be displayed , the position corresponding to the display size of the selected object, and the display means is capable of displaying the selected object at the position in the virtual space identified by the identification means .

The present invention makes it possible to create a sense of unity between the object and the content while preventing the object from reducing the visibility of the content.

FIG. 1 is a diagram showing an example of an overview of a system according to an embodiment. FIG. 2 is a block diagram showing an example of a hardware configuration of a user terminal according to an embodiment. FIG. 2 is a block diagram showing an example of a hardware configuration of a server according to an embodiment. 1 is a block diagram showing an example of a hardware configuration of a game playing terminal according to an embodiment. FIG. 2 is a block diagram showing an example of a hardware configuration of a distribution terminal according to an embodiment. FIG. 2 is a block diagram showing an example of a functional configuration of a user terminal, a server, and an HMD set according to an embodiment. FIG. 2 is a block diagram showing an example of a functional configuration of a distribution terminal according to an embodiment. 11 is a flowchart illustrating a part of a process executed in a user terminal and a game playing terminal according to an embodiment. 1A to 1C are diagrams illustrating an example of a virtual space provided to a player and a field of view image viewed by the player according to an embodiment. A figure showing an example of a virtual space provided to a user of a user terminal and a field of view image viewed by the user according to one embodiment. FIG. 11 is a diagram showing another example of a field of view image visually recognized by a user of a user terminal. A figure showing yet another example of a field of view image visually recognized by a user of a user terminal. 1 is a flowchart illustrating a part of a process executed in a game playing terminal according to an embodiment. 10 is a flowchart illustrating a part of a process executed in a user terminal according to an embodiment. 11 is a flowchart illustrating a part of a process executed in a server according to an embodiment. FIG. 2 illustrates an example of a list of users who have joined a game, according to one embodiment. 11 is a flowchart illustrating a part of a process executed in a distribution terminal according to an embodiment. FIG. 13 shows a specific example of a screen displayed on a distribution terminal according to an embodiment. FIG. 13 is a diagram showing another specific example of a screen displayed on a distribution terminal according to an embodiment. FIG. 13 illustrates one example of voice input by a player according to one embodiment. 13A to 13C are diagrams showing yet another specific example of a screen displayed on a distribution terminal according to an embodiment and an overview of distribution of operation instruction data. FIG. 13 illustrates another example of voice input by a player according to one embodiment. 13A to 13C are diagrams showing yet another specific example of a screen displayed on a distribution terminal according to an embodiment and an overview of distribution of operation instruction data. A diagram showing an overview of the transmission of game progress information from a game playing terminal to a user terminal according to an embodiment. 10 is a flowchart illustrating a part of a process executed in a user terminal according to an embodiment. FIG. 13 is a diagram showing a specific example of video playback. FIG. 11 is a diagram showing another specific example of video playback. FIG. 28A is a flowchart for explaining the tip item reflection process executed by the user terminal side, and FIG. 28B is a flowchart for explaining the tip item reflection response process executed by the distribution side such as a distribution terminal. FIG. 1A is a diagram for explaining the display size and displayable area in which tip items belonging to each of the first to third categories can be reflected, and FIG. 1B is a diagram for explaining each of a plurality of areas defined as displayable areas in a virtual space. FIG. 13 is a diagram illustrating a display example of a tip effect that is executed when a tip item less than a predetermined amount is inserted. FIG. 13 is a diagram illustrating a display example of a tipping effect with an icon that is executed when a tip item of a predetermined amount or more is inserted. FIG. 13 is a diagram illustrating a display example of a tipping effect with an icon that is executed when a tip item of a predetermined amount or more is inserted.

The system disclosed herein is a system for providing a game to multiple users. The system will be described below with reference to the drawings. Note that the present invention is not limited to these examples, but is indicated by the claims, and it is intended that all modifications within the meaning and scope equivalent to the claims are included in the present invention. In the following description, the same elements in the description of the drawings will be given the same reference numerals, and redundant description will not be repeated.

<Overview of System 1 Operation>
FIG. 1 is a diagram showing an overview of a system 1 according to this embodiment. The system 1 includes a plurality of user terminals 100 (computers), a server 200, a game play terminal 300 (external device, second external device), and a distribution terminal 400 (external, first external device). Note that in FIG. 1, user terminals 100A to 100C, in other words, three user terminals 100, are shown as an example of the plurality of user terminals 100, but the number of user terminals 100 is not limited to the example shown. In this embodiment, when there is no need to distinguish between the user terminals 100A to C, they are referred to as "user terminals 100". The user terminals 100, the game play terminals 300, and the distribution terminals 400 are connected to the server 200 via a network 2. The network 2 is composed of the Internet and various mobile communication systems constructed by wireless base stations not shown. Examples of such mobile communication systems include so-called 3G and 4G mobile communication systems, LTE (Long Term Evolution), and wireless networks (such as Wi-Fi (registered trademark)) that can be connected to the Internet via a predetermined access point.

(Game Overview)
In this embodiment, a game that is mainly played by a user of a game play terminal 300 will be described as an example of a game provided by the system 1 (hereinafter, this game). Hereinafter, the user of the game play terminal 300 will be referred to as a "player". As an example, the player (actor) progresses through the game by operating a character that appears in the game. In addition, in this game, the user of the user terminal 100 plays a role of supporting the player's progress through the game. Details of this game will be described later. Note that the game provided by the system 1 may be a game in which multiple users participate, and is not limited to this example.

(Game Play Terminal 300)
The game play terminal 300 progresses the game in response to input operations by the player. Furthermore, the game play terminal 300 sequentially distributes information generated by the player's game play (hereinafter, game progress information) to the server 200 in real time.

(Server 200)
The server 200 transmits game progress information (second data) received in real time from the game playing terminal 300 to the user terminal 100. The server 200 also mediates the transmission and reception of various types of information between the user terminal 100, the game playing terminal 300, and the distribution terminal 400.

(Distribution terminal 400)
The distribution terminal 400 generates action instruction data (first data) in response to an input operation by a user of the distribution terminal 400, and distributes the action instruction data to the user terminal 100 via the server 200. The action instruction data is data for playing a video on the user terminal 100, and specifically, data for causing a character appearing in the video to perform a motion.

In this embodiment, as an example, the user of the distribution terminal 400 is a player of the game. Also, as an example, the video played on the user terminal 100 based on the action instruction data is a video of a character operated by the player in the game moving. "Movement" refers to moving at least a part of the character's body, and includes speaking. For this reason, the action instruction data in this embodiment includes, for example, voice data for making the character speak, and motion data for moving the character's body.

As an example, the action instruction data is transmitted to the user terminal 100 after the game ends. Details of the action instruction data and the video that is played based on the action instruction data will be described later.

(User terminal 100)
The user terminal 100 receives game progress information in real time, and generates and displays a game screen using the information. In other words, the user terminal 100 reproduces the game screen of the game being played by the player by real-time rendering. This allows the user of the user terminal 100 to view the same game screen as the player views while playing the game, at approximately the same timing as the player.

In addition, the user terminal 100 generates information to assist the player in progressing through the game in response to input operations by the user, and transmits the information to the game play terminal 300 via the server 200. Details of the information will be described later.

The user terminal 100 also receives the action instruction data from the distribution terminal 400, and generates and plays a video (image) using the action instruction data. In other words, the user terminal 100 renders and plays the action instruction data.

<Hardware configuration of System 1>
Fig. 2 is a diagram showing the hardware configuration of the user terminal 100. Fig. 3 is a diagram showing the hardware configuration of the server 200. Fig. 4 is a diagram showing the hardware configuration of the game playing terminal 300. Fig. 5 is a diagram showing the hardware configuration of the distribution terminal 400.

(User terminal 100)
In this embodiment, as an example, the user terminal 100 is realized as a smartphone, but the user terminal 100 is not limited to a smartphone. For example, the user terminal 100 may be realized as a feature phone, a tablet computer, a laptop computer (a so-called notebook computer), or a desktop computer. The user terminal 100 may also be a game device suitable for playing games.

As shown in FIG. 2, the user terminal 100 includes a processor 10, a memory 11, a storage 12, a communication interface (IF) 13, an input/output IF 14, a touch screen 15 (display unit), a camera 17, and a distance measurement sensor 18. These components of the user terminal 100 are electrically connected to each other via a communication bus. Note that the user terminal 100 may include an input/output IF 14 to which a display (display unit) configured separately from the user terminal 100 main body can be connected instead of or in addition to the touch screen 15.

As shown in FIG. 2, the user terminal 100 may be configured to be able to communicate with one or more controllers 1020. The controller 1020 establishes communication with the user terminal 100 according to a communication standard such as Bluetooth (registered trademark). The controller 1020 may have one or more buttons, etc., and transmits output values based on user input operations on the buttons, etc., to the user terminal 100. The controller 1020 may also have various sensors such as an acceleration sensor and an angular velocity sensor, and transmits output values of the various sensors to the user terminal 100.

In addition to or in addition to the user terminal 100 being equipped with the camera 17 and distance measurement sensor 18, the controller 1020 may have the camera 17 and distance measurement sensor 18.

For example, when starting a game, the user terminal 100 desirably has the user using the controller 1020 input user identification information, such as the user's name or login ID, via the controller 1020. This enables the user terminal 100 to link the controller 1020 with the user, and to identify which user the output value belongs to based on the source (controller 1020) of the received output value.

When the user terminal 100 communicates with multiple controllers 1020, each user can hold a controller 1020 and realize a multi-play on the single user terminal 100 without communicating with other devices such as the server 200 via the network 2. In addition, the user terminals 100 can realize a multi-play locally by connecting to each other via a wireless standard such as a wireless LAN (Local Area Network) standard (connecting to communicate without going through the server 200). When the above-mentioned multi-play is realized locally by a single user terminal 100, the user terminal 100 may further include at least some of the various functions of the server 200, which will be described later. In addition, when the above-mentioned multi-play is realized locally by multiple user terminals 100, the multiple user terminals 100 may have various functions of the server 200, which will be described later, distributed among them.

Even when the above-mentioned multiplay is realized locally, the user terminal 100 may communicate with the server 200. For example, information indicating the results of a game, such as the score or win/loss, may be associated with user identification information and transmitted to the server 200.

The controller 1020 may be configured to be detachable from the user terminal 100. In this case, a connection part for the controller 1020 may be provided on at least one of the faces of the housing of the user terminal 100. When the user terminal 100 and the controller 1020 are connected by a wire via the connection part, the user terminal 100 and the controller 1020 transmit and receive signals via the wire.

As shown in FIG. 2, the user terminal 100 may accept the attachment of a storage medium 1030 such as an external memory card via the input/output IF 14. This allows the user terminal 100 to read programs and data recorded on the storage medium 1030. The program recorded on the storage medium 1030 is, for example, a game program.

The user terminal 100 may store in the memory 11 of the user terminal 100 a game program acquired by communicating with an external device such as the server 200, or may store in the memory 11 a game program acquired by reading it from the storage medium 1030.

As described above, the user terminal 100 includes a communication IF 13, an input/output IF 14, a touch screen 15, a camera 17, and a distance sensor 18 as examples of mechanisms for inputting information to the user terminal 100. Each of the above-mentioned components as input mechanisms can be considered as an operation unit configured to accept input operations by the user.

For example, when the operation unit is composed of at least one of the camera 17 and the distance measurement sensor 18, the operation unit detects an object 1010 in the vicinity of the user terminal 100 and identifies an input operation from the detection result of the object. As an example, a user's hand or a marker of a predetermined shape is detected as the object 1010, and the input operation is identified based on the color, shape, movement, or type of the object 1010 obtained as the detection result. More specifically, when the user's hand is detected from an image captured by the camera 17, the user terminal 100 identifies and accepts a gesture (a series of movements of the user's hand) detected based on the captured image as the user's input operation. Note that the captured image may be a still image or a video.

Alternatively, if the operation unit is configured with a touch screen 15, the user terminal 100 identifies and accepts a user operation performed on the input unit 151 of the touch screen 15 as a user input operation. Alternatively, if the operation unit is configured with a communication IF 13, the user terminal 100 identifies and accepts a signal (e.g., an output value) transmitted from the controller 1020 as a user input operation. Alternatively, if the operation unit is configured with an input/output IF 14, the user terminal 100 identifies and accepts a signal output from an input device (not shown) connected to the input/output IF 14 and different from the controller 1020 as a user input operation.

(Server 200)
The server 200 may be, for example, a general-purpose computer such as a workstation or a personal computer. The server 200 includes a processor 20, a memory 21, a storage 22, a communication IF 23, and an input/output IF 24. These components of the server 200 are electrically connected to each other by a communication bus.

(Game Play Terminal 300)
The game play terminal 300 may be, for example, a general-purpose computer such as a personal computer. The game play terminal 300 comprises a processor 30, a memory 31, a storage 32, a communication IF 33, and an input/output IF 34. These components of the game play terminal 300 are electrically connected to each other by a communication bus.

As shown in FIG. 4, the game play terminal 300 according to this embodiment is, for example, included in an HMD (Head Mounted Display) set 1000. In other words, it can be expressed that the HMD set 1000 is included in the system 1, and it can also be expressed that the player plays the game using the HMD set 1000. Note that the device on which the player plays the game is not limited to the HMD set 1000. For example, the device may be any device that allows the player to virtually experience the game. The device may also be realized as a smartphone, a feature phone, a tablet computer, a laptop computer (a so-called notebook computer), or a desktop computer. The device may also be a game device suitable for game play.

The HMD set 1000 includes, in addition to the game play terminal 300, an HMD 500, an HMD sensor 510, a motion sensor 520, a display 530, and a controller 540. The HMD 500 includes a monitor 51, a gaze sensor 52, a first camera 53, a second camera 54, a microphone 55, and a speaker 56. The controller 540 may include the motion sensor 520.

The HMD 500 is worn on the player's head and can provide the player with a virtual space during operation. More specifically, the HMD 500 displays an image for the right eye and an image for the left eye on the monitor 51. When each eye of the player views the respective image, the player can recognize the image as a three-dimensional image based on the parallax between the two eyes. The HMD 500 can include both a so-called head-mounted display equipped with a monitor, and a head-mounted device to which a smartphone or other terminal equipped with a monitor can be attached.

The monitor 51 is realized, for example, as a non-transparent display device. In one aspect, the monitor 51 is disposed on the main body of the HMD 500 so as to be positioned in front of both eyes of the player. Therefore, when the player visually recognizes the three-dimensional image displayed on the monitor 51, the player can be immersed in the virtual space. In one aspect, the virtual space includes, for example, images of a background, objects that the player can control, and menus that the player can select. In one aspect, the monitor 51 can be realized as a liquid crystal monitor or an organic EL (Electro Luminescence) monitor provided on a so-called smartphone or other information display terminal.

In another aspect, the monitor 51 may be realized as a transmissive display device. In this case, the HMD 500 may be an open type such as glasses, rather than a closed type that covers the player's eyes as shown in FIG. 1. The transmissive monitor 51 may be temporarily configurable as a non-transmissive display device by adjusting its transmittance. The monitor 51 may include a configuration that simultaneously displays a portion of an image that constitutes the virtual space and the real space. For example, the monitor 51 may display an image of the real space captured by a camera mounted on the HMD 500, or the transmittance of a portion of the monitor may be set high to make the real space visible.

In one aspect, the monitor 51 may include a sub-monitor for displaying an image for the right eye and a sub-monitor for displaying an image for the left eye. In another aspect, the monitor 51 may be configured to display an image for the right eye and an image for the left eye as a single unit. In this case, the monitor 51 includes a high-speed shutter. The high-speed shutter operates to alternately display an image for the right eye and an image for the left eye so that the image is recognized by only one of the eyes.

In one aspect, the HMD 500 includes multiple light sources (not shown). Each light source is realized, for example, by an LED (Light Emitting Diode) that emits infrared rays. The HMD sensor 510 has a position tracking function for detecting the movement of the HMD 500. More specifically, the HMD sensor 510 reads multiple infrared rays emitted by the HMD 500 and detects the position and tilt of the HMD 500 in real space.

In another aspect, the HMD sensor 510 may be realized by a camera. In this case, the HMD sensor 510 can detect the position and inclination of the HMD 500 by performing image analysis processing using image information of the HMD 500 output from the camera.

In another aspect, the HMD 500 may include a sensor (not shown) as a position detector instead of or in addition to the HMD sensor 510. The HMD 500 may use the sensor to detect the position and inclination of the HMD 500 itself. For example, if the sensor is an angular velocity sensor, a geomagnetic sensor, or an acceleration sensor, the HMD 500 may use any of these sensors instead of the HMD sensor 510 to detect the position and inclination of the HMD 500 itself. As an example, if the sensor included in the HMD 500 is an angular velocity sensor, the angular velocity sensor detects the angular velocity of the HMD 500 around three axes in real space over time. The HMD 500 calculates the change in angle of the HMD 500 around the three axes over time based on each angular velocity, and further calculates the inclination of the HMD 500 based on the change in angle over time.

The gaze sensor 52 detects the direction in which the gaze of the player's right eye and left eye is directed. In other words, the gaze sensor 52 detects the player's gaze. Detection of the gaze direction is achieved, for example, by a known eye tracking function. The gaze sensor 52 is achieved by a sensor having the eye tracking function. In some situations, the gaze sensor 52 preferably includes a sensor for the right eye and a sensor for the left eye. The gaze sensor 52 may be, for example, a sensor that irradiates the player's right eye and left eye with infrared light and detects the rotation angle of each eyeball by receiving reflected light from the cornea and iris in response to the irradiated light. The gaze sensor 52 can detect the player's gaze based on each detected rotation angle.

The first camera 53 photographs the lower part of the player's face. More specifically, the first camera 53 photographs the player's nose, mouth, etc. The second camera 54 photographs the player's eyes, eyebrows, etc. The housing of the HMD 500 on the player's side is defined as the inside of the HMD 500, and the housing of the HMD 500 on the opposite side from the player is defined as the outside of the HMD 500. In one aspect, the first camera 53 may be disposed on the outside of the HMD 500, and the second camera 54 may be disposed on the inside of the HMD 500. The images generated by the first camera 53 and the second camera 54 are input to the game play terminal 300. In another aspect, the first camera 53 and the second camera 54 may be realized as a single camera, and the player's face may be photographed by this single camera.

The microphone 55 converts the player's speech into an audio signal (electrical signal) and outputs it to the game play terminal 300. The speaker 56 converts the audio signal into sound and outputs it to the player. In another aspect, the HMD 500 may include earphones instead of the speaker 56.

The controller 540 is connected to the game play terminal 300 by wire or wirelessly. The controller 540 accepts input of commands from the player to the game play terminal 300. In one aspect, the controller 540 is configured to be held by the player. In another aspect, the controller 540 is configured to be attached to a part of the player's body or clothing. In yet another aspect, the controller 540 may be configured to output at least one of vibration, sound, and light based on a signal transmitted from the game play terminal 300. In yet another aspect, the controller 540 accepts operations from the player to control the position and movement of an object placed in a virtual space.

In one aspect, the controller 540 includes multiple light sources. Each light source is realized, for example, by an LED that emits infrared light. The HMD sensor 510 has a position tracking function. In this case, the HMD sensor 510 reads multiple infrared rays emitted by the controller 540 and detects the position and tilt of the controller 540 in real space. In another aspect, the HMD sensor 510 may be realized by a camera. In this case, the HMD sensor 510 can detect the position and tilt of the controller 540 by performing image analysis processing using image information of the controller 540 output from the camera.

In one aspect, the motion sensor 520 is attached to the player's hand and detects the movement of the player's hand. For example, the motion sensor 520 detects the rotation speed, number of rotations, etc. of the hand. The detected signal is sent to the game play terminal 300. The motion sensor 520 is provided, for example, in the controller 540. In one aspect, the motion sensor 520 is provided, for example, in the controller 540 configured to be held by the player. In another aspect, for safety in the real space, the controller 540 is attached to something that is not easily blown away by being worn on the player's hand, such as a glove type. In yet another aspect, a sensor that is not worn by the player may detect the movement of the player's hand. For example, a signal from a camera that captures the player may be input to the game play terminal 300 as a signal representing the player's action. The motion sensor 520 and the game play terminal 300 are connected to each other wirelessly, as an example. In the case of wireless communication, the communication form is not particularly limited, and for example, Bluetooth or other known communication methods are used.

The display 530 displays an image similar to the image displayed on the monitor 51. This allows users other than the player wearing the HMD 500 to view the same image as the player. The image displayed on the display 530 does not need to be a three-dimensional image, and may be an image for the right eye or an image for the left eye. Examples of the display 530 include a liquid crystal display and an organic EL monitor.

Based on various information acquired from the various parts of the HMD 500, the controller 540, and the motion sensor 520, the game play terminal 300 moves the character operated by the player to progress through the game. Here, "movement" includes moving various parts of the body, changing posture, changing facial expression, moving, speaking, touching or moving objects placed in the virtual space, and using weapons, tools, etc. held by the character. That is, in this game, when the player moves various parts of his body, the character also moves various parts of his body in the same way as the player. Also, in this game, the character speaks the contents of what the player speaks. In other words, in this game, the character is an avatar object that acts as an alter ego of the player. As an example, at least a part of the character's movements may be executed by input to the controller 540 by the player.

In this embodiment, the motion sensors 520 are attached to the player's hands, both feet, waist, and head, as an example. The motion sensors 520 attached to the player's hands may be provided in the controller 540, as described above. The motion sensor 520 attached to the player's head may be provided in the HMD 500. The motion sensors 520 may further be attached to the user's elbows and knees. By increasing the number of motion sensors 520 attached to the player, the player's movements can be more accurately reflected in the character.
Also, instead of attaching the motion sensors 520 to various parts of the body, the player may wear a suit to which one or more motion sensors 520 are attached. In other words, the method of motion capture is not limited to the example using the motion sensor 520.

(Distribution terminal 400)
The distribution terminal 400 may be a mobile terminal such as a smartphone, a personal digital assistant (PDA), or a tablet computer, or may be a so-called stationary terminal such as a desktop personal computer.

As shown in FIG. 5, the distribution terminal 400 includes a processor 40, a memory 41, a storage 42, a communication IF 43, an input/output IF 44, and a touch screen 45. Note that instead of or in addition to the touch screen 45, the distribution terminal 400 may include an input/output IF 44 to which a display (display unit) configured separately from the distribution terminal 400 main body can be connected.

The controller 1021 may have a physical input mechanism such as one or more buttons, levers, sticks, wheels, etc. The controller 1021 transmits to the broadcasting terminal 400 an output value based on an input operation input to the input mechanism by an operator of the broadcasting terminal 400 (a player in this embodiment). The controller 1021 may also have various sensors such as an acceleration sensor and an angular velocity sensor, and may transmit the output values of the various sensors to the broadcasting terminal 400. The above-mentioned output values are received by the broadcasting terminal 400 via the communication IF 43.

The distribution terminal 400 may be equipped with a camera and a distance measurement sensor (both not shown). Instead of or in addition to the distribution terminal 400 being equipped with a camera and a distance measurement sensor, the controller 1021 may have the camera and distance measurement sensor.

As described above, the distribution terminal 400 includes a communication IF 43, an input/output IF 44, and a touch screen 45 as an example of a mechanism for inputting information to the distribution terminal 400. Each of the above-mentioned units as an input mechanism can be considered as an operation unit configured to accept input operations by the user.

When the operation unit is configured with a touch screen 45, the distribution terminal 400 identifies and accepts a user operation performed on the input unit 451 of the touch screen 45 as a user input operation. Alternatively, when the operation unit is configured with a communication IF 43, the distribution terminal 400 identifies and accepts a signal (e.g., an output value) transmitted from the controller 1021 as a user input operation. Alternatively, when the operation unit is configured with an input/output IF 44, the distribution terminal 400 identifies and accepts a signal output from an input device (not shown) connected to the input/output IF 44 as a user input operation.

<Hardware components of each device>
Processors 10, 20, 30, and 40 control the overall operation of user terminal 100, server 200, game play terminal 300, and distribution terminal 400, respectively. Processors 10, 20, 30, and 40 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit). Processors 10, 20, 30, and 40 read programs from storages 12, 22, 32, and 42, respectively, which will be described later. Then, processors 10, 20, 30, and 40 load the read programs into memories 11, 21, 31, and 41, which will be described later, respectively. Processors 10, 20, and 30 execute the loaded programs.

Memories 11, 21, 31, and 41 are main storage devices. Memories 11, 21, 31, and 41 are composed of storage devices such as ROM (Read Only Memory) and RAM (Random Access Memory). Memory 11 provides a working area for processor 10 by temporarily storing programs and various data read by processor 10 from storage 12 (described later). Memory 11 also temporarily stores various data generated while processor 10 is operating according to the program. Memory 21 provides a working area for processor 20 by temporarily storing various programs and data read by processor 20 from storage 22 (described later). Memory 21 also temporarily stores various data generated while processor 20 is operating according to the program. Memory 31 provides a working area for processor 30 by temporarily storing various programs and data read by processor 30 from storage 32 (described later). Memory 31 also temporarily stores various data generated while processor 30 is operating according to the program. The memory 41 provides a working area for the processor 40 by temporarily storing the programs and various data that the processor 40 reads from the storage 42 (described later). The memory 41 also temporarily stores various data that the processor 40 generates while it is operating according to the programs.

In this embodiment, the program executed by processors 10 and 30 may be a game program for the game. In this embodiment, the program executed by processor 40 may be a distribution program for realizing the distribution of action instruction data. Furthermore, processor 10 may further execute a viewing program for realizing the playback of video.

In this embodiment, the program executed by the processor 20 may be at least one of the game program, distribution program, and viewing program described above. The processor 20 executes at least one of the game program, distribution program, and viewing program in response to a request from at least one of the user terminal 100, the game play terminal 300, and the distribution terminal 400. Note that the distribution program and the viewing program may be executed in parallel.

In other words, the game program may be a program that realizes a game through cooperation between the user terminal 100, the server 200, and the game play terminal 300. The distribution program may be a program that realizes the distribution of action instruction data through cooperation between the server 200 and the distribution terminal 400. The viewing program may be a program that realizes the playback of a video through cooperation between the user terminal 100 and the server 200.

Storages 12, 22, 32, and 42 are auxiliary storage devices. Storages 12, 22, 32, and 42 are configured with storage devices such as flash memory or HDDs (Hard Disk Drives). Storages 12 and 32 store, for example, various data related to games. Storage 42 stores various data related to the distribution of action instruction data. Storage 12 stores various data related to the playback of videos. Storage 22 may store at least some of the various data related to games, the distribution of action instruction data, and the playback of videos.

The communication IFs 13, 23, 33, and 43 respectively control the transmission and reception of various data in the user terminal 100, the server 200, the game play terminal 300, and the distribution terminal 400. The communication IFs 13, 23, 33, and 43 control, for example, communication via a wireless LAN (Local Area Network), Internet communication via a wired LAN, a wireless LAN, or a mobile phone line network, and communication using short-range wireless communication, etc.

The input/output IFs 14, 24, 34, and 44 are interfaces through which the user terminal 100, the server 200, the game playing terminal 300, and the distribution terminal 400 accept data input and output data, respectively. The input/output IFs 14, 24, 34, and 44 may input and output data via a USB (Universal Serial Bus) or the like. The input/output IFs 14, 24, 34, and 44 may include physical buttons, a camera, a microphone, a speaker, a mouse, a keyboard, a display, a stick, a lever, and the like. The input/output IFs 14, 24, 34, and 44 may also include a connection section for transmitting and receiving data to and from peripheral devices.

The touch screen 15 is an electronic component that combines an input unit 151 and a display unit 152 (display). The touch screen 45 is an electronic component that combines an input unit 451 and a display unit 452. The input units 151 and 451 are, for example, touch-sensitive devices, and are configured, for example, by a touchpad. The display units 152 and 452 are, for example, configured by a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like.

The input units 151 and 451 have a function of detecting the position where a user operation (mainly a physical contact operation such as a touch operation, a slide operation, a swipe operation, or a tap operation) is input on the input surface, and transmitting information indicating the position as an input signal. The input units 151 and 451 may be provided with a touch sensing unit (not shown). The touch sensing unit may be of any type, such as a capacitive type or a resistive film type.

Although not shown, the user terminal 100 and the distribution terminal 400 may each include one or more sensors for identifying the holding posture of the user terminal 100 and the distribution terminal 400. The sensors may be, for example, an acceleration sensor or an angular velocity sensor.

If the user terminal 100 and the distribution terminal 400 are equipped with sensors, the processors 10 and 40 can respectively identify the holding orientation of the user terminal 100 and the distribution terminal 400 from the output of the sensor and perform processing according to the holding orientation. For example, when the user terminal 100 and the distribution terminal 400 are held vertically, the processors 10 and 40 can respectively set the display units 152 and 452 to display a vertically long image in a vertical screen display. On the other hand, when the user terminal 100 and the distribution terminal 400 are held horizontally, the processors 10 and 40 can respectively set the display units 152 and 452 to display a horizontally long image in a horizontal screen display. In this way, the processors 10 and 40 can respectively switch between a vertical screen display and a horizontal screen display depending on the holding orientation of the user terminal 100 and the distribution terminal 400.

<Functional Configuration of System 1>
Fig. 6 is a block diagram showing the functional configuration of the user terminal 100, the server 200, and the HMD set 1000 included in the system 1. Fig. 7 is a block diagram showing the functional configuration of the distribution terminal 400 shown in Fig. 6.

The user terminal 100 functions as an input device that accepts input operations from the user, and as an output device that outputs images and sounds from the game. The user terminal 100 functions as a control unit 110 and a storage unit 120 through cooperation between the processor 10, memory 11, storage 12, communication IF 13, input/output IF 14, touch screen 15, etc.

The server 200 has a function of mediating the transmission and reception of various information between the user terminal 100, the HMD set 1000, and the distribution terminal 400. The server 200 functions as a control unit 210 and a storage unit 220 through cooperation of the processor 20, memory 21, storage 22, communication IF 23, input/output IF 24, etc.

The HMD set 1000 (game playing terminal 300) functions as an input device that accepts input operations from the player, as an output device that outputs game images and sounds, and as a function that transmits game progress information in real time to the user terminal 100 via the server 200. The HMD set 1000 functions as a control unit 310 and a memory unit 320 through cooperation of the processor 30, memory 31, storage 32, communication IF 33, input/output IF 34, HMD 500, HMD sensor 510, motion sensor 520, controller 540, etc. of the game playing terminal 300.

The distribution terminal 400 has a function of generating action instruction data and transmitting the action instruction data to the user terminal 100 via the server 200. The distribution terminal 400 functions as a control unit 410 and a storage unit 420 through cooperation of the processor 40, memory 41, storage 42, communication IF 43, input/output IF 44, touch screen 45, etc.

(Data stored in the memory unit of each device)
The storage unit 120 stores a game program 131 (program), game information 132, and user information 133. The storage unit 220 stores a game program 231, game information 232, user information 233, and a user list 234. The storage unit 320 stores a game program 331, game information 332, and user information 333. The storage unit 420 stores a user list 421, a motion list 422, and a delivery program 423 (program, second program).

The game programs 131, 231, and 331 are game programs executed by the user terminal 100, the server 200, and the HMD set 1000, respectively. The game is realized by each device operating in cooperation based on the game programs 131, 231, and 331. The game programs 131 and 331 may be stored in the storage unit 220 and downloaded to the user terminal 100 and the HMD set 1000, respectively. In this embodiment, the user terminal 100 renders data received from the distribution terminal 400 based on the game program 131 and plays a video. In other words, the game program 131 is also a program for playing a video using video instruction data distributed from the distribution terminal 400. The program for playing the video may be different from the game program 131. In this case, the storage unit 120 stores the program for playing the video separately from the game program 131.

Game information 132, 232, 332 is data that the user terminal 100, server 200, and HMD set 1000 refer to when executing a game program, respectively. User information 133, 233, 333 is data related to the account of the user of the user terminal 100. Game information 232 is game information 132 of each user terminal 100 and game information 332 of the HMD set 1000. User information 233 is user information 133 of each user terminal 100 and user information of the player included in user information 333. User information 333 is user information 133 of each user terminal 100 and user information of the player.

User list 234 and user list 421 are lists of users who have participated in the game. User list 234 and user list 421 may include a list of users who have participated in the most recent game play by the player, as well as a list of users who have participated in each game play prior to that game play. Motion list 422 is a list of multiple motion data created in advance. Motion list 422 is, for example, a list in which motion data is associated with information identifying each motion (for example, a motion name). Distribution program 423 is a program for realizing the distribution of action instruction data for playing a video on user terminal 100 to user terminal 100.

(Functional configuration of server 200)
The control unit 210 executes a game program 231 stored in the storage unit 220 to comprehensively control the server 200. For example, the control unit 210 mediates the transmission and reception of various types of information between the user terminal 100, the HMD set 1000, and the distribution terminal 400.

The control unit 210 functions as a communication intermediation unit 211, a log generation unit 212, and a list generation unit 213 according to the description of the game program 231. The control unit 210 can also function as other functional blocks (not shown) to mediate the sending and receiving of various information related to game play and distribution of action instruction data, and to support the progress of the game.

The communication mediation unit 211 mediates the transmission and reception of various information between the user terminal 100, the HMD set 1000, and the distribution terminal 400. For example, the communication mediation unit 211 transmits game progress information received from the HMD set 1000 to the user terminal 100. The game progress information includes data indicating the movement of the character operated by the player, the parameters of the character, items and weapons held by the character, enemy characters, and other information. The server 200 transmits the game progress information to the user terminals 100 of all users participating in the game. In other words, the server 200 transmits common game progress information to the user terminals 100 of all users participating in the game. As a result, the game progresses in the user terminals 100 of all users participating in the game in the same way as in the HMD set 1000.

Also, for example, the communication intermediation unit 211 transmits information received from one of the user terminals 100 to assist the player in progressing through the game to the other user terminals 100 and the HMD set 1000. As an example, the information may be item information indicating an item that allows the player to progress through the game advantageously and is provided to the player (character). The item information includes information indicating the user who provided the item (user name, user ID, etc.). The communication intermediation unit 211 may also mediate the distribution of action instruction data from the distribution terminal 400 to the user terminal 100.

The log generating unit 212 generates a log of the game progress based on the game progress information received from the HMD set 1000. The list generating unit 213 generates a user list 234 after the game play ends. Although details will be described later, each user in the user list 234 is associated with a tag indicating the content of the support provided by that user to the player. The list generating unit 213 generates a tag based on the game progress log generated by the log generating unit 212 and associates it with the corresponding user. Note that the list generating unit 213 may associate the content of the support provided by each user to the player, which is input by the game operator or the like using a terminal device such as a personal computer, with the corresponding user as a tag. This makes the content of the support provided by each user more detailed. Note that when a user participates in a game, the user terminal 100 transmits information indicating the user to the server 200 based on the user's operation. For example, the user terminal 100 transmits the user ID input by the user to the server 200. In other words, the server 200 holds information indicating each user for all users participating in the game. The list generation unit 213 can use this information to generate the user list 234.

(Functional configuration of the HMD set 1000)
The control unit 310 executes the game program 331 stored in the storage unit 320 to comprehensively control the HMD set 1000. For example, the control unit 310 progresses the game in accordance with the game program 331 and the player's operation. Furthermore, while the game is progressing, the control unit 310 communicates with the server 200 as necessary to transmit and receive information. The control unit 310 may transmit and receive information directly to and from the user terminal 100 without going through the server 200.

The control unit 310 functions as an operation reception unit 311, a display control unit 312, a UI control unit 313, an animation generation unit 314, a game progression unit 315, a virtual space control unit 316, and a reaction processing unit 317 according to the description of the game program 331. Depending on the nature of the game being executed, the control unit 310 can also function as other functional blocks (not shown) for the purpose of controlling characters appearing in the game, etc.

The operation reception unit 311 detects and accepts input operations by the player. The operation reception unit 311 accepts signals input from the HMD 500, motion sensor 520, controller 540, etc., determines what type of input operation has been performed, and outputs the results to each element of the control unit 310.

The UI control unit 313 controls user interface (hereinafter, UI) images displayed on the monitor 51, display 530, etc. The UI images are tools that allow the player to input to the HMD set 1000 necessary for the progression of the game, or tools that allow the player to obtain information output from the HMD set 1000 while the game is in progress. Examples of UI images include, but are not limited to, icons, buttons, lists, menu screens, etc.

The animation generation unit 314 generates animations showing the motion of various objects based on the control modes of the various objects. For example, the animation generation unit 314 may generate animations that show an object (e.g., a player's avatar object) moving as if it were actually there, moving its mouth, changing its facial expression, etc.

The game progression unit 315 progresses the game based on the game program 331, input operations by the player, and the actions of the avatar object in response to the input operations. For example, the game progression unit 315 performs a predetermined game process when the avatar object performs a predetermined action. For example, the game progression unit 315 may receive information representing an operation of a user at the user terminal 100, and perform game processing based on the user's operation. The game progression unit 315 also generates game progress information according to the progress of the game and transmits it to the server 200. The game progress information is transmitted to the user terminal 100 via the server 200. As a result, the progress of the game at the HMD set 1000 is shared at the user terminal 100. In other words, the progress of the game at the HMD set 1000 and the progress of the game at the user terminal 100 are synchronized.

The virtual space control unit 316 performs various controls related to the virtual space provided to the player in accordance with the progress of the game. As an example, the virtual space control unit 316 generates various objects and places them in the virtual space. The virtual space control unit 316 also places a virtual camera in the virtual space. The virtual space control unit 316 also operates the various objects placed in the virtual space in accordance with the progress of the game. The virtual space control unit 316 also controls the position and tilt of the virtual camera placed in the virtual space in accordance with the progress of the game.

The display control unit 312 outputs to the monitor 51 and the display 530 a game screen that reflects the results of the processing performed by each of the above-mentioned elements. The display control unit 312 may display an image based on the field of view from a virtual camera placed in a virtual space as a game screen on the monitor 51 and the display 530. The display control unit 312 may also include an animation generated by the animation generation unit 314 in the game screen. The display control unit 312 may also draw the above-mentioned UI image controlled by the UI control unit 313 by superimposing it on the game screen.

The reaction processing unit 317 receives feedback from the user of the user terminal 100 regarding the reaction of the player to the gameplay of the player, and outputs this to the player. In this embodiment, for example, the user terminal 100 can create a comment (message) addressed to an avatar object based on an input operation by the user. The reaction processing unit 317 receives comment data of the comment and outputs it. The reaction processing unit 317 may display text data corresponding to the user's comment on the monitor 51 or the display 530, or may output audio data corresponding to the user's comment from a speaker (not shown). In the former case, the reaction processing unit 317 may draw an image corresponding to the text data (i.e., an image including the content of the comment) superimposed on the game screen.

(Functional configuration of user terminal 100)
The control unit 110 executes the game program 131 stored in the storage unit 120 to comprehensively control the user terminal 100. For example, the control unit 110 progresses the game in accordance with the game program 131 and the user's operation. Furthermore, while the game is progressing, the control unit 110 communicates with the server 200 as necessary to transmit and receive information. The control unit 110 may transmit and receive information directly to the HMD set 1000 without going through the server 200.

The control unit 110 functions as an operation reception unit 111, a display control unit 112, a UI control unit 113, an animation generation unit 114, a game progression unit 115, a virtual space control unit 116, and a video playback unit 117 according to the description of the game program 131. The control unit 110 can also function as other functional blocks (not shown) for the progression of the game according to the nature of the game being executed.

The operation reception unit 111 detects and receives user input operations on the input unit 151. The operation reception unit 111 determines what kind of input operation has been performed based on the actions performed by the user on the touch screen 15 and the console via the other input/output IF 14, and outputs the results to each element of the control unit 110.

For example, the operation reception unit 111 receives an input operation on the input unit 151, detects the coordinates of the input position of the input operation, and identifies the type of the input operation. The operation reception unit 111 identifies, for example, a touch operation, a slide operation, a swipe operation, a tap operation, etc. as the type of input operation. Furthermore, when a continuously detected input is interrupted, the operation reception unit 111 detects that the contact input from the touch screen 15 has been released.

The UI control unit 113 controls UI images to be displayed on the display unit 152 to construct a UI in response to at least one of the user's input operations and the received game progress information. The UI images are tools that allow the user to input necessary for the progress of the game to the user terminal 100, or tools that allow the user to obtain information from the user terminal 100 that is output during the progress of the game. Examples of UI images include, but are not limited to, icons, buttons, lists, and menu screens.

The animation generation unit 114 generates animations showing the motion of various objects based on the control aspects of the various objects.

The game progression unit 115 progresses the game based on the game program 131, the received game progress information, and input operations by the user. When the game progression unit 115 performs a predetermined game process based on an input operation by the user, it transmits information related to the game process to the HMD set 1000 via the server 200. This causes the predetermined game process to be shared in the HMD set 1000. In other words, the progress of the game in the HMD set 1000 and the progress of the game in the user terminal 100 are synchronized. The predetermined game process is, for example, a process of providing an item to an avatar object, and in this example, the information related to the game process is the item information described above.

The virtual space control unit 116 performs various controls related to the virtual space provided to the user in accordance with the progress of the game. As an example, the virtual space control unit 116 generates various objects and places them in the virtual space. The virtual space control unit 116 also places a virtual camera in the virtual space. The virtual space control unit 116 also operates the various objects placed in the virtual space in accordance with the progress of the game, specifically, the received game progress information. The virtual space control unit 316 also controls the position and tilt of the virtual camera placed in the virtual space in accordance with the progress of the game, specifically, the received game progress information.

The display control unit 112 outputs to the display unit 152 a game screen reflecting the results of the processing executed by each of the above-mentioned elements. The display control unit 112 may display, as a game screen, an image based on the field of view from a virtual camera placed in a virtual space provided to the user on the display unit 152. The display control unit 112 may also include an animation generated by the animation generation unit 114 in the game screen. The display control unit 112 may also draw the above-mentioned UI image controlled by the UI control unit 113 by superimposing it on the game screen. In any case, the game screen displayed on the display unit 152 is the same game screen as the game screen displayed on the other user terminals 100 and the HMD set 1000.

The video playback unit 117 analyzes (renders) the action instruction data received from the distribution terminal 400 and plays the video.

(Functional configuration of distribution terminal 400)
The control unit 410 performs overall control of the distribution terminal 400 by executing a program (not shown) stored in the storage unit 420. For example, the control unit 410 generates action instruction data according to the program and an operation by a user (a player in this embodiment) of the distribution terminal 400, and distributes the action instruction data to the user terminal 100. Furthermore, the control unit 410 communicates with the server 200 as necessary to transmit and receive information. The control unit 410 may transmit and receive information directly to and from the user terminal 100 without going through the server 200.

The control unit 410 functions as a communication control unit 411, a display control unit 412, an operation reception unit 413, a voice reception unit 414, a motion identification unit 415, and an action instruction data generation unit 416 according to the program description. The control unit 410 can also function as other functional blocks (not shown) for generating and distributing action instruction data.

The communication control unit 411 controls the transmission and reception of information to and from the server 200 or the user terminal 100 via the server 200. As an example, the communication control unit 411 receives a user list 421 from the server 200. Also, as an example, the communication control unit 411 transmits operation instruction data to the user terminal 100.

The display control unit 412 outputs to the display unit 452 various screens reflecting the results of the processing executed by each element. As an example, the display control unit 412 displays a screen including the received user list 234. As an example, the display control unit 412 also displays a screen including a motion list 422 for allowing the player to select motion data for moving an avatar object, which is included in the action instruction data to be distributed.

The operation reception unit 413 detects and accepts the input operation of the player to the input unit 151. The operation reception unit 111 determines what kind of input operation has been performed from the action performed by the player on the touch screen 45 and the console via other input/output IF 44, and outputs the result to each element of the control unit 410.

For example, the operation reception unit 413 receives an input operation on the input unit 451, detects the coordinates of the input position of the input operation, and identifies the type of the input operation. The operation reception unit 413 identifies, for example, a touch operation, a slide operation, a swipe operation, a tap operation, etc. as the type of input operation. Furthermore, when a continuously detected input is interrupted, the operation reception unit 413 detects that the contact input from the touch screen 45 has been released.

The audio receiving unit 414 receives audio generated around the distribution terminal 400 and generates audio data for the audio. As an example, the audio receiving unit 414 receives audio spoken by a player and generates audio data for the audio.

The motion identification unit 415 identifies the motion data selected by the player from the motion list 422 in response to the player's input operation.

The motion instruction data generating unit 416 generates motion instruction data. As an example, the motion instruction data generating unit 416 generates motion instruction data that includes the generated voice data and the identified motion data.

The functions of the HMD set 1000, the server 200, and the user terminal 100 shown in FIG. 6, and the functions of the distribution terminal 400 shown in FIG. 7 are merely examples. Each of the HMD set 1000, the server 200, the user terminal 100, and the distribution terminal 400 may have at least some of the functions of the other devices. Furthermore, a device other than the HMD set 1000, the server 200, the user terminal 100, and the distribution terminal 400 may be a component of the system 1, and the other device may execute some of the processing in the system 1. That is, the computer that executes the game program in this embodiment may be any of the HMD set 1000, the server 200, the user terminal 100, and the distribution terminal 400, as well as other devices, or may be realized by a combination of multiple devices.

<Virtual space control processing>
Fig. 8 is a flowchart showing an example of the flow of a control process for a virtual space provided to a player and a virtual space provided to a user of a user terminal 100. Fig. 9 is a diagram showing a virtual space 600A provided to a player and a field of view image visually recognized by the player according to an embodiment. Fig. 10 is a diagram showing a virtual space 600B provided to a user of a user terminal 100 and a field of view image visually recognized by the user according to an embodiment. Hereinafter, when there is no need to distinguish between the virtual spaces 600A and 600B, they will be referred to as "virtual space 600".

In step S1, the processor 30, as the virtual space control unit 316, defines the virtual space 600A shown in FIG. 9. The processor 30 defines the virtual space 600A using virtual space data (not shown). The virtual space data may be stored in the game play terminal 300, may be generated by the processor 30 based on the game program 331, or may be acquired by the processor 30 from an external device such as the server 200.

As an example, the virtual space 600 has a spherical structure that covers the entire 360-degree area around a point defined as the center. In order to avoid complicating the explanation, Figs. 9 and 10 show an example of the upper half of the celestial sphere of the virtual space 600.

In step S2, the processor 30, functioning as the virtual space control unit 316, places an avatar object 610 (character) in the virtual space 600A. The avatar object 610 is an avatar object associated with the player, and moves according to the player's input operation.

In step S3, the processor 30, as the virtual space control unit 316, places other objects in the virtual space 600A. In the example of FIG. 9, the processor 30 places objects 631 to 634. The other objects may include, for example, character objects (so-called non-player characters, NPCs) that act according to the game program 331, control objects such as virtual hands, and objects that imitate animals, plants, artificial objects, natural objects, etc. that are placed as the game progresses.

In step S4, the processor 30, as the virtual space control unit 316, places a virtual camera 620A in the virtual space 600A. As an example, the processor 30 places the virtual camera 620A at the head position of the avatar object 610.

In step S5, the processor 30 displays the field of view image 650 on the monitor 51 and the display 530. The processor 30 defines a field of view area 640A, which is the field of view from the virtual camera 620A in the virtual space 600A, according to the initial position and inclination of the virtual camera 620A. The processor 30 then defines a field of view image 650 corresponding to the field of view area 640A. The processor 30 outputs the field of view image 650 to the monitor 51 and the display 530, thereby causing the field of view image 650 to be displayed on the HMD 500 and the display 530.

In the example of FIG. 9, as shown in FIG. 9(A), a portion of object 634 is included in field of view 640A, so field of view image 650 includes a portion of object 634 as shown in FIG. 9(B).

In step S6, the processor 30 transmits the initial placement information to the user terminal 100 via the server 200. The initial placement information is information that indicates the initial placement positions of various objects in the virtual space 600A. In the example of FIG. 9, the initial placement information includes information on the initial placement positions of the avatar object 610 and objects 631 to 634. The initial placement information can also be expressed as one piece of game progress information.

In step S7, the processor 30, as the virtual space control unit 316, controls the virtual camera 620A in accordance with the movement of the HMD 500. Specifically, the processor 30 controls the direction and inclination of the virtual camera 620A in accordance with the movement of the HMD 500, i.e., the position of the player's head. As described below, when the player moves his/her head (changes the position of the head), the processor 30 moves the head of the avatar object 610 in accordance with this movement. The processor 30 controls the direction and inclination of the virtual camera 620A, for example, so that the direction of the line of sight of the avatar object 610 and the direction of the line of sight of the virtual camera 620A coincide with each other. In step S8, the processor 30 updates the field of view image 650 in accordance with the change in the direction and inclination of the virtual camera 620A.

In step S9, the processor 30, as the virtual space control unit 316, moves the avatar object 610 in response to the movement of the player. As an example, the processor 30 moves the avatar object 610 in the virtual space 600A in response to the player's movement in real space. The processor 30 also moves the head of the avatar object 610 in the virtual space 600A in response to the player's movement of the head in real space.

In step S10, the processor 30, as the virtual space control unit 316, moves the virtual camera 620A so that it follows the avatar object 610. In other words, the virtual camera 620A is always at the position of the head of the avatar object 610, even if the avatar object 610 moves.

The processor 30 updates the field of view image 650 in response to the movement of the virtual camera 620A. That is, the processor 30 updates the field of view area 640A in response to the pose of the player's head and the position of the virtual camera 620A in the virtual space 600A. As a result, the field of view image 650 is updated.

In step S11, the processor 30 transmits action instruction data for the avatar object 610 to the user terminal 100 via the server 200. The action instruction data here includes at least any of the following during the virtual experience (e.g., during game play): motion data capturing the player's actions, audio data of the player's voice, and operation data indicating the content of the input operation on the controller 540. When the player is playing a game, the action instruction data is transmitted to the user terminal 100 as, for example, game progress information.

The processing of steps S7 to S11 is continuously and repeatedly executed while the player is playing the game.

In step S21, the processor 10 of the user terminal 100 of the user 3 defines the virtual space 600B shown in FIG. 10 as the virtual space control unit 116. The processor 10 defines the virtual space 600B using virtual space data (not shown). The virtual space data may be stored in the user terminal 100, may be generated by the processor 10 based on the game program 131, or may be acquired by the processor 10 from an external device such as the server 200.

In step S22, the processor 10 receives the initial placement information. In step S23, the processor 10, as the virtual space control unit 116, places various objects in the virtual space 600B according to the initial placement information. In the example of FIG. 10, the various objects are the avatar object 610 and objects 631 to 634.

In step S24, the processor 10, as the virtual space control unit 116, places a virtual camera 620B in the virtual space 600B. As an example, the processor 10 places the virtual camera 620B at the position shown in FIG. 10(A).

In step S25, the processor 10 displays the field of view image 660 on the display unit 152. The processor 10 defines a field of view area 640B, which is the field of view from the virtual camera 620B in the virtual space 600B, according to the initial position and inclination of the virtual camera 620B. The processor 10 then defines a field of view image 660 corresponding to the field of view area 640B. The processor 10 outputs the field of view image 660 to the display unit 152, thereby causing the display unit 152 to display the field of view image 660.

In the example of Figure 10, as shown in Figure 10(A), avatar object 610 and object 631 are included in field of view area 640B, so field of view image 660 includes avatar object 610 and object 631 as shown in Figure 10(B).

In step S26, the processor 10 receives the action instruction data. In step S27, the processor 10, as the virtual space control unit 116, moves the avatar object 610 in the virtual space 600B in accordance with the action instruction data. In other words, the processor 10 plays a video of the avatar object 610 moving by real-time rendering.

In step S28, the processor 10, as the virtual space control unit 116, controls the virtual camera 620B in response to the user's operation received by the operation receiving unit 111. In step S29, the processor 10 updates the field of view image 660 in response to changes in the position of the virtual camera 620B in the virtual space 600B and the direction and inclination of the virtual camera 620B. In addition, in step S28, the processor 10 may automatically control the virtual camera 620B in response to the movement of the avatar object 610, for example, the movement of the avatar object 610 or the change in the direction. For example, the processor 10 may automatically move the virtual camera 620B or change the direction and inclination so that the avatar object 610 is always photographed from the front. In addition, as an example, the processor 10 may automatically move the virtual camera 620B or change the direction and inclination so that the avatar object 610 is always photographed from the rear in response to the movement of the avatar object 610.

In this way, in virtual space 600A, avatar object 610 moves in response to the player's movements. Action instruction data indicating this movement is transmitted to user terminal 100. In virtual space 600B, avatar object 610 moves in response to the received action instruction data. As a result, avatar object 610 performs similar movements in virtual space 600A and virtual space 600B. In other words, user 3 can use user terminal 100 to visually recognize the movement of avatar object 610 in response to the player's movement.

＜Game Overview＞
11 is a diagram showing another example of a field of view image displayed on the user terminal 100. Specifically, it is a diagram showing an example of a game screen of a game (this game) executed by the system 1 and being played by a player.

As an example, this game is a game in which an avatar object 610 that controls a weapon such as a gun or a knife and multiple enemy objects 671 that are NPCs appear in a virtual space 600, and the avatar object 610 fights the enemy objects 671. Various game parameters such as the stamina of the avatar object 610, the number of usable magazines, the number of bullets remaining in the gun, and the number of remaining enemy objects 671 are updated as the game progresses.

In this game, a plurality of stages are prepared, and the player can clear each stage by achieving a predetermined achievement condition associated with the stage. The predetermined achievement condition may include, for example, a condition that is achieved by defeating all enemy objects 671 that appear, defeating a boss object among the enemy objects 671 that appear, acquiring a predetermined item, reaching a predetermined position, and the like. The achievement condition is defined in the game program 131. In this game, the player clears a stage when an achievement condition is achieved in accordance with the content of the game, in other words, the victory of the avatar object 610 over the enemy object 671 (win or loss between the avatar object 610 and the enemy object 671) is determined. In contrast, for example, if the game executed by the system 1 is a racing game or the like, the ranking of the avatar object 610 is determined when the condition of reaching the goal is achieved.

In this game, in order to share a virtual space between the HMD set 1000 and the multiple user terminals 100, game progress information is live-streamed to the multiple user terminals 100 at predetermined time intervals. As a result, a field of view image of the field of view area defined by the virtual camera 620B corresponding to the user terminal 100 is displayed on the touch screen 15 of the user terminal 100 while watching the game. In addition, parameter images representing the vitality of the avatar object 610, the number of available magazines, the number of bullets remaining in the gun, the number of enemy objects 671 remaining, etc. are superimposed and displayed on the upper right and upper left sections of the field of view image. This field of view image can also be expressed as a game screen.

As described above, the game progress information includes motion data capturing the player's actions, audio data of the voice spoken by the player, and operation data indicating the content of input operations on the controller 540. These pieces of data are information for identifying the position, posture, orientation, etc. of the avatar object 610, information for identifying the position, posture, orientation, etc. of the enemy object 671, and information for identifying the positions of other objects (e.g., obstacle objects 672, 673). The processor 10 analyzes (renders) the game progress information to identify the position, posture, orientation, etc. of each object.

The game information 132 includes data on various objects such as the avatar object 610, enemy object 671, obstacle objects 672, 673, etc. The processor 10 uses this data and the analysis results of the game progress information to update the position, posture, orientation, etc. of each object. As a result, the game progresses and each object in the virtual space 600B moves in the same way as each object in the virtual space 600A. Specifically, in the virtual space 600B, each object including the avatar object 610 behaves based on the game progress information, regardless of whether or not a user operation is performed on the user terminal 100.

On the touch screen 15 of the user terminal 100, as an example, UI images 701 and 702 are displayed superimposed on the field of view image. The UI image 701 is a UI image that accepts an operation for displaying on the touch screen 15 a UI image 711 that accepts an item insertion operation from the user 3 to support the avatar object 610. The UI image 702 is a UI image that accepts an operation for displaying on the touch screen 15 a UI image (described later) that accepts an operation from the user 3 to input and send a comment for the avatar object 610 (in other words, the player 4). The operation accepted by the UI images 701 and 702 may be, for example, an operation of tapping the UI images 701 and 702.

When UI image 701 is tapped, UI image 711 is displayed superimposed on the field of view image. UI image 711 includes, for example, UI image 711A depicting a magazine icon, UI image 711B depicting a first aid box icon, UI image 711C depicting a traffic cone icon, and UI image 711D depicting a barricade icon. An item insertion operation corresponds to, for example, tapping on any of the UI images.

As an example, when UI image 711A is tapped, the number of remaining bullets in the gun used by avatar object 610 increases. When UI image 711B is tapped, the stamina of avatar object 610 is restored. When UI images 711C and 711D are tapped, obstacle objects 672, 673 that impede the movement of enemy object 671 are placed in the virtual space. One of obstacle objects 672, 673 may be more likely to impede the movement of enemy object 671 than the other.

The processor 10 transmits item input information indicating that an item input operation has been performed to the server 200. The item input information includes at least information for identifying the type of item specified by the item input operation. The item input information may also include other information related to the item, such as information indicating the position where the item is placed. The item input information is transmitted to other user terminals 100 and the HMD set 1000 via the server 200.

Figure 12 is a diagram showing another example of a field of view image displayed on the user terminal 100. Specifically, it is a diagram showing an example of a game screen of the present game, and is a diagram for explaining communication between a player and the user terminal 100 during game play.

In the example of FIG. 12(A), the user terminal 100 causes the avatar object 610 to make an utterance 691. Specifically, the user terminal 100 causes the avatar object 610 to make an utterance 691 in accordance with the voice data included in the game progress information. The content of the utterance 691 is "I'm out of bullets!" uttered by player 4. In other words, the content of the utterance 691 informs each user that the magazine is empty and there is only one bullet loaded in the gun, and therefore they are about to lose the means to attack the enemy object 671.

Note that in FIG. 12(A), speech bubbles are used to visually indicate the speech of the avatar object 610, but in reality, audio is output from the speaker of the user terminal 100. Note that along with the audio output, the speech bubble shown in FIG. 12(A) (i.e., a speech bubble containing the text of the audio content) may be displayed in the field of view image. This is also true for speech 692, which will be described later.

When a tap operation on UI image 702 is received, the user terminal 100 displays UI images 705 and 706 (message UIs) superimposed on the field of view image, as shown in FIG. 12(B). UI image 705 is a UI image that displays a comment for the avatar object 610 (in other words, the player). UI image 706 is a UI image that receives a comment sending operation from user 3 to send the input comment.

As an example, when the user terminal 100 receives a tap operation on the UI image 705, it displays a UI image (not shown, hereinafter simply referred to as "keyboard") resembling a keyboard on the touch screen 15. The user terminal 100 displays text in the UI image 705 according to the user's input operation on the keyboard. In the example of FIG. 12(B), the text "I'll send you a magazine" is displayed on the UI image 705.

After inputting text, for example, when the user terminal 100 receives a tap operation on the UI image 706, the user terminal 100 transmits comment information including information indicating the input content (text content) and information indicating the user to the server 200. The comment information is transmitted to other user terminals 100 and the HMD set 1000 via the server 200.

UI image 703A is a UI image showing the username of the user who sent the comment, and UI image 704A is a UI image showing the content of the comment sent by that user. In the example of FIG. 12(B), a user with the username "BBBBB" has sent comment information with the content "Danger!" using his/her own user terminal 100, causing UI image 703A and UI image 704A to be displayed. UI image 703A and UI image 704A are displayed on the touch screens 15 of all user terminals 100 participating in the game, and on the monitor 51 of the HMD 500. Note that UI images 703A and 704A may be a single UI image. In other words, a single UI image may include the username and the content of the comment.

In the example of FIG. 12(C), a user with the user name "AAAAA", who is a user of the user terminal 100 shown in FIG. 12, has entered and sent a comment as described above, causing UI images 703B and 704B to be displayed on the touch screen 15. UI image 703B includes the user name "AAAAA", and UI image 704B includes the comment "I'll send you a magazine!" that was entered in the example of FIG. 12(B).

The example in FIG. 12(C) is a field of view image 611 after user "AAAAA" further inputs a tap operation on UI image 701, causes UI image 711 to be displayed on touch screen 15, and inputs a tap operation on UI image 711A. That is, as a result of item insertion information indicating a magazine being transmitted from user terminal 100 of user "AAAAA" to other user terminals 100 and HMD set 1000, user terminal 100 and HMD set 1000 place effect object 674 (described below) in virtual space 600. As an example, user terminal 100 and HMD set 1000 execute an effect related to effect object 674 after the elapsed time indicated in the item insertion information has elapsed, and execute a process to activate the effect of the item object.

In the example of FIG. 12(D), the number of magazines increases from 0 to 1 by executing a process to activate the effect of the item object. As a result, the player utters "Thank you!" to the user "AAAAA", and the voice data of this utterance is transmitted to each user terminal 100. As a result, each user terminal 100 outputs the voice "Thank you!" as the utterance 692 of the avatar object 610.

As described above, in this game, communication between users and avatar object 610 is realized by outputting the speech of avatar object 610 based on the player's speech and by inputting comments by each user.

(Game Progression Processing in the Game Playing Terminal 300)
FIG. 13 is a flow chart showing an example of the flow of a game progression process executed on the game playing terminal 300.

In step S31, the processor 30, as the game progression unit 315, progresses the game based on the game program 331 and the player's movements. In step S32, the processor 30 generates game progress information and distributes it to the user terminals 100. Specifically, the processor 30 transmits the generated game progress information to each user terminal 100 via the server 200.

In step S33, when the processor 30 receives the item insertion information (YES in S33), in step S34, the processor 30 places the item object in the virtual space 600A based on the item insertion information. As an example, the processor 30 places an effect object 674 in the virtual space 600A before placing the item object (see FIG. 11(C)). The effect object 674 may be, for example, an object resembling a gift box. As an example, the processor 30 may execute an effect related to the effect object 674 after the elapsed time indicated in the item insertion information has elapsed. The effect may be, for example, an animation of the lid of the gift box opening. After the execution of the animation, the processor 30 executes a process to activate the effect of the item object. For example, in the example of FIG. 11(D), an obstacle object 673 is placed.

After the animation is executed, the processor 30 may place an item object corresponding to the tapped UI image in the virtual space 600A. For example, when a tap operation is performed on the UI image 711A, the processor 30 places a magazine object indicating a magazine in the virtual space 600A after the animation is executed. Also, when a tap operation is performed on the UI image 711B, the processor 30 places a first aid box object indicating a first aid box in the virtual space 600A after the animation is executed. For example, when the avatar object 610 moves to the position of the magazine object or the first aid box object, the processor 30 may execute a process to activate the effect of the magazine object or the first aid box object.

The processor 30 continues and repeats the processing of steps S31 to S34 until the game ends. When the game ends, for example, when the player inputs a predetermined input operation to end the game (YES in step S35), the processing shown in FIG. 13 ends.

(Game Progression Processing in User Terminal 100)
FIG. 14 is a flowchart showing an example of the flow of a game progression process executed on the user terminal 100.

In step S41, the processor 10 receives game progress information. In step S42, the processor 10, as the game progression unit 115, progresses the game based on the game progress information.

In step S43, when the processor 10 accepts an item insertion operation by the user 3 (YES in step S43), in step S44, the processor 10 consumes the virtual currency and places the effect object 674 in the virtual space 600B. Here, the virtual currency may be purchased (charged for the game) by the user 3 performing a predetermined operation on the processor 10 before or during participation in the game, or may be granted to the user 3 when a predetermined condition is met. The predetermined condition may be something that requires participation in the game, such as clearing a quest in the game, or something that does not require participation in the game, such as answering a questionnaire. The amount of virtual currency (amount of virtual currency owned) is, for example, stored in the user terminal 100 as game information 132.

In step S45, the processor 10 transmits item insertion information to the server 200. The item insertion information is transmitted to the game play terminal 300 via the server 200.

After placing the effect object 674, when a predetermined time has elapsed, the processor 10 places an item object in the virtual space 600A. In the example of FIG. 11, an obstacle object 673 is placed. That is, when the user 3 inputs a tap operation on the UI image 711C, a predetermined amount of virtual currency is consumed and the obstacle object 673 is placed.

The processor 10 continues and repeats the processing of steps S41 to S45 until the game ends. When the game ends, for example, when the player performs a predetermined input operation to end the game, or when the user 3 performs a predetermined input operation to leave the game (YES in step S46), the processing shown in FIG. 14 ends.

(Game Progression Processing in Server 200)
FIG. 15 is a flowchart showing an example of the flow of a game progression process executed by the server 200.

In step S51, the processor 20 receives game progress information from the game play terminal 300. In step S52, the processor 20, as the log generation unit 212, updates the game progress log (hereinafter, the play log). Note that the play log is generated by the processor 20, for example, when initial placement information is received from the game play terminal 300.

In step S53, the processor 20 transmits the received game progress information to each user terminal 100.

If item input information is received from any of the user terminals 100 in step S54 (YES in step S54), in step S55, the processor 20 updates the play log as the log generating unit 212. In step S56, the processor 20 transmits the received item input information to the game play terminal 300.

The processor 20 continues and repeats the processing of steps S51 to S56 until the game ends. When the game ends, for example when information indicating that the game has ended is received from the game play terminal 300 (YES in step S57), in step S58, the processor 20, as the list generation unit 213, generates a list of users who participated in the game (user list 234) from the play log. The processor 20 stores the generated user list 234 in the server 200.

Figure 16 is a diagram showing a specific example of a user list 234. The "User" column stores information (e.g., username) indicating each user who participated in the game. The "Tag" column stores information (tags) generated based on the assistance provided by each user to the player. In the example of Figure 16, among the tags stored in the "Tag" column, those without quotation marks are information automatically generated by the processor 20, and those with quotation marks are information manually entered by the game operator.

In the example of FIG. 16, the user "AAAAA" is associated with the information magazine, 10th floor, boss, and "defeat boss by gifting magazine." This indicates that, for example, in a boss battle on stage 10th floor, the user "AAAAA" inserted a magazine, and the avatar object 610 defeated the boss with bullets from the inserted magazine.

Furthermore, the user "BBBBB" is associated with the information: first aid kit, 3F, weak enemy, and "recovered just before game over." This indicates that, for example, in a battle with a weak enemy on stage 3F, the user "BBBBB" throws in a first aid kit, and as a result, the avatar object 610's vitality is recovered just before it reaches 0 (game over).

Furthermore, the user "CCCCC" is associated with the information "barricade, 5th floor, weak enemies, and "two zombies held back by barricade." This indicates that, for example, in a battle with weak enemies on stage 5F, the user "CCCCC" put in a barricade (obstacle object 672 in FIG. 11), and as a result, succeeded in holding back two weak enemies.

In the example of FIG. 16, one support provided is associated with the username of each user 3, but the username of a user 3 who provided support multiple times is associated with a tag for each of the multiple support instances. It is preferable that each tag is distinguished in the user list 234. This allows a player who refers to the user list 421 using the distribution terminal 400 after the game ends to accurately understand the content of each support.

<Distribution of operation instruction data>
(Distribution process in distribution terminal 400)
Fig. 17 is a flowchart showing an example of the flow of a distribution process executed by distribution terminal 400. Fig. 18 is a diagram showing a specific example of a screen displayed on distribution terminal 400. Fig. 19 is a diagram showing another specific example of a screen displayed on the distribution terminal.

In step S61, the processor 40, as the operation receiving unit 413, receives a first operation for displaying a list of users who have participated in the game (user list 234). The download screen 721 shown in FIG. 18 (A) is a screen for downloading the user list 234 from the server 200 and displaying it on the display unit 452. As an example, the download screen 721 is a screen that is displayed immediately after an operation to start an application that executes the distribution process shown in FIG. 17 is input to the distribution terminal 400.

Download screen 721 includes UI images 722 and 723, as an example. UI image 722 accepts an operation for downloading user list 234, i.e., the first operation described above. The first operation may be, for example, an operation of tapping UI image 722. UI image 723 accepts an operation for closing an application. The operation may be, for example, an operation of tapping UI image 723.

When a tap operation on the UI image 722 is received, in step S62, the processor 40, as the communication control unit 411, acquires (receives) the user list 234 from the server 200. In step S63, the processor 40, as the display control unit 412, causes the display unit 452 to display the user list 234. Specifically, the processor 40 causes the display unit 452 to display a user list screen generated based on the user list 234. As an example, the user list screen may be the user list screen 731 shown in FIG. 18 (B). The user list screen 731 consists of record images corresponding to each record in the user list 234. In the example of FIG. 18 (B), record images 732A to 732C are described as record images, but the number of record images is not limited to three. In the example of FIG. 18(B), if the number of records in the user list 234 is greater than three (i.e., the number of users participating in the game is greater than three), the player can display other record images on the display unit 452, for example, by inputting an operation to scroll the screen (e.g., a drag operation or a flick operation) on the touch screen 45.

As an example, record images 732A-732C include user names 733A-733C, tag information 734A-734C, and icons 735A-735C, respectively. Hereinafter, when there is no need to distinguish between record images 732A-732C, user names 733A-733C, tag information 734A-734C, and icons 735A-735C, they will be referred to as "record image 732," "user name 733," "tag information 734," and "icon 735," respectively.

User name 733 is information that indicates each user who participated in the game, and is stored in the "user" column in user list 234. Tag information 734 is information that indicates a tag that is associated with each piece of information that indicates each user who participated in the game in user list 234. For example, record image 732A includes "AAAAA" as user name 733A. Therefore, record image 732A includes, as tag information 734A, "Magazine, 10F, Boss, 'Defeat boss by gifting magazine'," which is associated with "AAAAA" in user list 234. Icon 735 is, for example, an image that the user has set in advance.

The processor 40 may store the received user list in the distribution terminal 400 (user list 421 in FIG. 7). The download screen 721 may include a UI image (not shown) for displaying the user list 421 on the display unit 452. In this example, when the UI image is tapped, the processor 40 does not download the user list 234, but reads the user list 421, generates a user list screen from the user list 421, and displays it on the display unit 452.

In step S64, the processor 40, as the operation receiving unit 413, receives a second operation for selecting one of the users included in the user list screen 731. As an example, the second operation may be an operation of tapping one of the record images 732 in the user list screen 731. In the example of FIG. 18(B), the player inputs a tap operation on the record image 732A. That is, the player selects the user "AAAAA" as the user to whom the action instruction data is to be distributed.

When a tap operation on the record image 732 is received, in step S65, the processor 40, as the display control unit 412, causes the display unit 452 to display the motion list 422. Specifically, the processor 40 causes the display unit 452 to display a motion list screen generated based on the motion list 422. As an example, the motion list screen may be the motion list screen 741 shown in FIG. 19. The motion list screen 741 is made up of record images corresponding to each record in the motion list 422. In the example of FIG. 19, record images 742A to 742C are shown as record images, but the number of record images is not limited to three. In the example of FIG. 19, if the number of records in the motion list 422 is more than four, the player can display other record images on the display unit 452 by inputting an operation to scroll the screen (e.g., a drag operation or a flick operation) to the touch screen 45.

As an example, record images 742A-742C include motion names 743A-743C, motion images 744A-744C, and UI images 745A-745C, respectively. Hereinafter, when there is no need to distinguish between record images 742A-742C, motion names 743A-743C, motion images 744A-744C, and UI images 745A-745C, they will be referred to as "record image 7432", "motion name 743", "motion image 744", and "UI image 745", respectively.

The motion name 743 is information for identifying a motion stored in the motion list 422. The motion image 744 is an image generated from the motion data associated with each motion name in the motion list 422. As an example, the processor 40 includes an image of the avatar object 610 taking an initial posture in each motion data in the record image 742 as the motion image 744. The motion image 744 may be a UI image that accepts a predetermined operation by the player (e.g., a tap operation on the motion image 744). When the processor 40 accepts the predetermined operation, it may play a motion video in which the avatar object 610 moves based on the motion data. When the motion video ends, the processor 40 may automatically redisplay the motion list screen 741.

In addition, instead of the motion image 744, the record image 742 may include a UI image including the text "motion playback," for example.

In step S66, the processor 40, as the operation receiving unit 413, receives a third operation for selecting a motion. As one example, the third operation may be a tap operation on the UI image 745. In other words, the UI image 745 receives an operation for selecting motion data corresponding to each record image 742. Having received the third operation, the processor 40, as the motion identifying unit 415, identifies the motion data selected by the player.

In step S67, the processor 40, acting as the display control unit 412 and the voice receiving unit 414, receives voice input from the player while playing back a motion video in which the avatar object 610 moves based on the selected motion data.

Figure 20 is a diagram showing a specific example of voice input by player 4. As shown in Figure 20, player 4 is inputting speech 820A while playing back motion video 810A. This speech 820A is addressed to user 3 (hereinafter, user 3A) whose user name is "AAAAA." That is, in the example of Figure 20, player 4 selects user 3A (first user) in step S64 and creates action instruction data addressed to user 3A. It should be noted that the user terminal 100 used by user 3A is user terminal 100A.

Since the speech voice 820A is addressed to the user 3A, it is based on the content of the support that the user 3A provided to the avatar object 610 (in other words, the player 4). Specifically, the user 3A inserted a magazine in the boss battle on stage 10F, and the avatar object 610 defeated the boss with the bullets from the inserted magazine. Therefore, the speech voice 820A is "Thank you for giving me the magazine during the boss battle! Your timing was perfect, too! Thanks to AAAAAA, I was able to clear it!" In this way, it is preferable that the speech voice includes the content of the support that the user 3 provided in the game and gratitude to the user 3.

In one situation, the player 4 creates a speech content addressed to the user 3 before starting voice input, i.e., before inputting the third operation to the distribution terminal 400. In another situation, the speech content addressed to the user 3 may be automatically generated by the processor 40. The processor 40 may also display a tag associated with the user 3 selected by the second operation superimposed on the motion video 810A.

The processor 40 converts the received voice into voice data. In step S68, the processor 40, as the motion instruction data generator 416, generates motion instruction data including the voice data and the motion data of the selected motion.

In step S69, the processor 40, as the communication control unit 411, distributes the generated action instruction data to the user terminal 100 (first computer) of the selected user 3 (user 3A in the example of FIG. 20). FIG. 21 is a diagram showing yet another specific example of a screen displayed on the distribution terminal 400. After executing step S68, the processor 40, as the display control unit 412, causes the display unit 452 to display a distribution screen. As an example, the distribution screen may be the distribution screen 751 shown in FIG. 21(A). The distribution screen 751 includes a UI image 752 and a motion image 753A. Furthermore, the distribution screen 751 may include information indicating the user to which the action instruction data is to be distributed, as shown in FIG. 21(A).

The UI image 752 accepts an operation for delivering the action instruction data to the selected user 3. The operation may be, for example, a tap operation on the UI image 752. The motion image 753A is a UI image that accepts an operation for playing a video based on the generated action instruction data, i.e., a video based on the action instruction data generated for the user 3A. The operation may be, for example, a tap operation on the motion image 753A. Note that the UI image that accepts an operation for playing the generated video is not limited to the motion image 753A. For example, it may be a UI image including the text "Play video". The processor 40 may automatically redisplay the delivery screen 751 when the video ends.

It is preferable that the distribution screen 751 further includes a UI image that accepts an operation to return to receiving voice input. The operation may be, for example, a tap operation on the UI image. By including the UI image on the distribution screen 751, the player 4 can retry voice input when the voice input fails, for example, when the player 4 makes a mistake in what he or she wants to say. The UI image may be a UI image that accepts an operation to return to selecting motion data.

When a tap operation on the UI image 752 is received, the processor 40 transmits the action instruction data to the server 200 together with information indicating the user 3A. The server 200 identifies the user terminal 100 to which the action instruction data is to be transmitted based on the information indicating the user 3A, and transmits the action instruction data to the identified user terminal 100 (i.e., the user terminal 100A).

When the transmission of the operation instruction data is completed, the processor 40 may, for example, cause the display unit 452 to display a distribution completion screen 761 shown in FIG. 21 (B). For example, the distribution completion screen 761 includes UI images 762 and 763. Furthermore, the distribution completion screen 761 may include text indicating that the transmission of the operation instruction data is completed, as shown in FIG. 21 (B).

The UI image 762 accepts an operation to start creating action instruction data addressed to another user 3. The operation may be, for example, an operation of tapping the UI image 762. When the processor 40 accepts the tap operation, it causes the display unit 452 to display the user list screen again. That is, when the tap operation is accepted, the distribution process returns to step S63. At this time, the processor 40 may generate a user list screen based on the user list 421 stored in the distribution terminal 400 and cause the display unit 452 to display the user list screen. The UI image 763 accepts an operation to end the application. The operation may be, for example, an operation of tapping the UI image 763. When the operation is accepted, the distribution process ends.

In the example described with reference to Figures 20 and 21, as shown in Figure 21 (C), the distribution terminal 400 transmits action instruction data for a video addressed to user 3A (user 3 whose user name is "AAAA") only to user terminal 100A.

Figure 22 is a diagram showing another specific example of voice input by player 4. As shown in Figure 22, player 4 inputs speech 820B while playing back motion video 810B. This speech 820B is addressed to user 3 (hereinafter, user 3B) whose user name is "BBBBB". That is, in the example of Figure 22, player 4 inputs a tap operation on record image 732B corresponding to user 3B in step S64, and creates action instruction data addressed to user 3B. It is assumed that the user terminal 100 used by user 3B is user terminal 100B.

Since the speech voice 820B is addressed to user 3B, it is based on the content of the support that user 3B provided to the avatar object 610 (in other words, player 4). Specifically, in a battle with a weak enemy on stage 3F, user "BBBBB" threw in a first aid kit, and as a result, the avatar object 610's strength recovered just before it reached 0 (the game would be over). Therefore, the speech voice 820B is "Thanks to the first aid kit that BBBBB gave me as a gift, I didn't get a game over on the 3rd floor. Thank you so much!".

Fig. 23 is a diagram showing yet another specific example of a screen displayed on the distribution terminal 400. The distribution screen 751 shown in Fig. 23 (A) includes a UI image 752 and a motion image 753B. When a tap operation is received, the motion image 753B plays a video based on the action instruction data generated for the user 3B.

When a tap operation on the UI image 752 is received, the processor 40 transmits the action instruction data to the server 200 together with information indicating the user 3B. The server 200 identifies the user terminal 100 to which the action instruction data is to be transmitted based on the information indicating the user 3B, and transmits the action instruction data to the identified user terminal 100 (i.e., the user terminal 100B).

In the example described with reference to Figures 22 and 23, as shown in Figure 23 (C), the distribution terminal 400 transmits the action instruction data for the video addressed to user 3B (user 3 whose user name is "BBBBB") only to user terminal 100B.

As described above, the content of the voice based on the voice data included in the action instruction data is based on the content of the support that the user 3 provided to the player 4 when participating in the most recent game. Since the content of the support differs for each user 3, the content of the voice will be different for each user 3. In other words, after the game ends, action instruction data including voices with different content is transmitted to the user terminals 100 of at least some of the users 3 who participated in the game.

In addition, the motion of the avatar object 610 in the example of FIG. 22 is different from the motion in the example of FIG. 20. That is, when generating the action instruction data for user 3B, the player 4 selects motion data that is different from the motion instruction data for user 3A. Specifically, in step S66, the player 4 inputs a tap operation on the UI image 745B, which selects the motion data corresponding to the record image 742B. In this way, the player 4 can make the motion data included in the action instruction data different for each user 3.

Then, the motion instruction data for each user 3, including the voice data with different content for each user 3 and the motion data selected for each user 3, is transmitted only to the user terminal 100 of each user 3. In other words, the motion instruction data that is unique for each user terminal 100 is transmitted to each of the user terminals 100 of the selected users 3.

Figure 24 is a diagram showing an overview of the transmission of game progress information from a game playing terminal 300 to a user terminal 100. While the action instruction data for video playback in the user terminal 100 is unique for each user terminal 100, as shown in Figure 24, the game progress information transmitted to the user terminals 100 of all users 3 participating in the game during game execution is common to each user terminal 100. In other words, the action instruction data included in the game progress information is also common to each user terminal 100. In this way, it can be said that the action instruction data for video playback and the action instruction data for progressing the game are different data in terms of similarities and differences between user terminals 100 and transmission destinations.

(Video Playback Processing in User Terminal 100)
FIG. 25 is a flowchart showing an example of the flow of a video playback process executed by the user terminal 100.

In step S71, the processor 10, functioning as the video playback unit 117, receives the action instruction data. In step S72, the processor 10, functioning as the video playback unit 117, notifies the user 3 of the reception of the action instruction data. As an example, the processor 10 notifies the user 3 of the reception of the action instruction data by at least one of displaying a notification image on the display unit 152, playing a notification sound from a speaker (not shown), and lighting or blinking a lighting unit (not shown) constituted by an LED (light-emitting diode) or the like.

In step S73, the processor 10, as the operation receiving unit 111, receives a first playback operation for playing the video. The first playback operation may be, for example, an operation of tapping on a notification image. In step S74, the processor 10, as the video playback unit 117, renders the action instruction data and plays the video. For example, the processor 10 may launch an application for playing the game and play the video, or may launch a video playback application separate from the application and play the video. Hereinafter, the video will be referred to as the "thank you video."

Figure 26 is a diagram showing a specific example of playback of a thank you video. Specifically, it is a diagram showing an example of playback of a thank you video on the user terminal 100 of user 3A. In the thank you video 910A played on the user terminal 100, the avatar object 610 speaks audio 920A while performing a certain motion. In other words, the processor 10 causes audio 920A to be output from a speaker (not shown) while playing back the thank you video 910A that includes the avatar object 610 performing a certain motion.

The motion in the thank you video 910A is based on the motion data selected by the player 4 when generating the action instruction data addressed to the user 3A, and the audio 920A is based on the audio data generated from the speech audio 820A input by the player 4 when generating the action instruction data. In other words, the audio 920A is audio that includes the content of the assistance provided by the user 3A in the game and gratitude for that assistance. In this way, by inputting the first playback operation, the user 3A can watch a thank you video in which the avatar object 610 speaks the content of the assistance he or she provided in the game and gratitude for that assistance.

As an example, the user terminal 100 may display at least one UI image on the touch screen 15 after the playback of the thank you video 910A has ended. The UI image may be, for example, a UI image that accepts an operation to play the thank you video 910A again, a UI image that accepts an operation to transition to another screen, or a UI image that accepts an operation to end the application.

Also, as an example, the user terminal 100 may display at least one UI image on the touch screen 15 while the thank you video 910A is being played. The UI image may be, for example, a plurality of UI images each of which accepts an operation to temporarily stop or end the thank you video 910A being played, or to change the scene being played.

Note that these UI images displayed during playback of the thank you video 910A and after playback of the thank you video 910A has stopped do not include UI images for replying to the avatar object 610. In other words, the thank you video 910A according to this embodiment does not include a means for replying to the avatar object 610.

Figure 27 is a diagram showing another specific example of playback of a thank you video. Specifically, it is a diagram showing an example of playback of a thank you video on the user terminal 100 of user 3B. In the thank you video 910B played on the user terminal 100, the avatar object 610 utters audio 920B while performing a certain motion. In other words, the processor 10 outputs audio 920B from a speaker (not shown) while playing back the thank you video 910B that includes the avatar object 610 performing a certain motion.

The motion in the thank you video 910B is based on the motion data selected by the player 4 in generating the action instruction data addressed to the user 3B, and the audio 920B is based on the audio data generated from the speech audio 820B input by the player 4 in generating the action instruction data. Therefore, the motion performed by the avatar object 610 in the example of FIG. 27 is different from the motion in the example of FIG. 26. Also, the audio 920B is audio including the content of the support provided by the user 3B in the game and gratitude for the support. Therefore, the content of the audio 920B in the example of FIG. 27 is different from the content of the audio 920A in the example of FIG. 26.

In this way, after the game ends, the thank you video received by at least some of the user terminals 100 of the users 3 who participated in the game is a video in which the speech content of the avatar object 610 differs for each user 3.

The processor 10 may display a UI image 930 including content encouraging participation in the next game, superimposed on the video 910. The UI image 930 may be distributed together with the action instruction data, or may be stored in the user terminal 100 as game information 132.

In a live-streaming game, a virtual space 600B, which is the same as the virtual space 600A defined by the game-playing terminal 300, is defined by a user terminal (viewing terminal, terminal device) 100. The game-playing terminal 300 distributes game progress information, including action instruction data that can identify the movements of the performers and the speech of the performers, to a plurality of user terminals 100 via a server 200. A plurality of types of object data for displaying objects, including an avatar object 610, are stored in advance in a memory 11 of the user terminal 100. When an operation is performed to start a live-streaming game, the user terminal 100 identifies the action instruction data from the game progress information distributed from the game-playing terminal 300 via the server 200, and identifies the type, position, posture, etc. of the object by analyzing (rendering) the action instruction data. In addition, the user terminal 100 places objects, including the avatar object 610, in the virtual space 600B using the object data stored in the memory 11 based on the analysis results.

The monitor 51 of the HMD 500 displays a field of view image 650 of the field of view area 640A that corresponds to the position, orientation, and tilt of the virtual camera 620A placed in the virtual space 600A. The touch screen 15 of the user terminal 100 displays a field of view image 660 of the field of view area 640B that corresponds to the position, orientation, and tilt of the virtual camera 620B placed in the virtual space 600B. A speaker (not shown) equipped in the user terminal 100 outputs the voice of the performer based on the action instruction data identified from the game progress information.

＜Game Overview＞
The game that can be provided by the system 1 according to the present embodiment includes, for example, a live distribution part performed during a live distribution period allocated before and after the live game period, and a live game part performed during the live game period. The live distribution part is a part in which an interaction such as a dialogue can be performed between the user and the avatar object 610. On the other hand, the live game part is a part in which a predetermined game can be performed between the user and the performer. The game that can be provided by the system 1 according to the present embodiment is composed of a plurality of types of parts including a live distribution part and a live game part, and progresses by shifting parts according to a predetermined order on the distribution side and the progress of the performer. During the live distribution, in both the live distribution part and the live game part, the user (viewer) can insert items, input comments, etc. at an appropriate timing while checking the status of the avatar object 610. For example, in response to a touch operation from the user, an object of an item designated by the user (hereinafter also referred to as a tip item) can be arranged or displayed (hereinafter, arrangement or display is also collectively referred to as reflection) in the virtual space in which the avatar object 610 is arranged. This allows the user's presence and contributions to be highlighted to performers and other users, increasing the interest of the game.

The live-streamed content includes multiple types of content with different characters displayed as avatar objects 610. The characters displayed as avatar objects 610 are associated with decorative objects such as clothing and accessories according to the type of character. Users can interact with the avatar objects 610 during the live stream.

Examples of interactions include item insertion and comment input. Item insertion means, for example, that a user selects a tipping item from among a plurality of tipping items (yells), and the object of the selected tipping item is reflected and displayed in the virtual space 600A of the game play terminal 300 and the virtual space 600B of each of the plurality of user terminals 100. In other words, during live distribution, the object of the tipping item selected by the user can be reflected in the virtual space in which the avatar object 610 is placed. In a live distribution game, a user inserts a tipping item to liven up the virtual space, appeal to the performers and other users about the user's presence and contribution, and improve parameters associated with the user, such as the user's intimacy with the character appearing as the avatar object 610. In addition, the control content in the live distribution game differs depending on the intimacy (parameter value) with the character appearing in the live distribution game. In the live streaming game of this embodiment, for example, depending on the user's intimacy with a character, the display mode of the comment UI for comments (e.g., UI images 705 and 706, the comment field in which the submitted comment is displayed), the display mode of the UI image for tipping items (e.g., UI image 711, the types of tipping items that can be inserted, the user icon that may be displayed when inserting, as described below, etc.), and the episodes (stories) of the character that can be viewed change.

In addition, there are multiple types of tipping items, and the display size is determined for each type. The multiple tipping items include, for example, relatively small tipping items whose height is equal to or less than a predetermined height lower than the avatar object 610, and relatively large tipping items whose height is higher than the avatar object 610. For this reason, if the position in the virtual space of the tipping item selected by the user is randomly determined, there is a risk of reducing the visibility of either the avatar object 610 or the object of the tipping item. For example, if a relatively large tipping item is placed in front of the avatar object 610 (in the direction in which the avatar object 610 is facing), the visibility of the avatar object 610 may be hindered by the relatively large tipping item. Therefore, in this embodiment, the reflection position (display position) at which the tipping item is reflected in the virtual space is specified based on the display size of the object of the tipping item. A specific description will be given below.

(How to determine the location of tip items)
The process for reflecting a tipping item in a virtual space will be described with reference to Figs. 28 and 29. Fig. 28 is a diagram for explaining a process for reflecting a tipping item in a virtual space based on a user's designation. Fig. 28(A) shows a tipping item reflection process executed by the user terminal 100, and Fig. 28(B) shows a tipping item reflection process executed by a computer on the distribution side such as the distribution terminal 400 (which may be the game play terminal 300). It is assumed that game progress information is distributed from the distribution side such as the distribution terminal 400, and a virtual space 600B including an avatar object 610 is defined in a plurality of user terminals 100 and live distribution is viewed.

First, in step S81 in FIG. 28(A), it is determined whether or not an operation for inserting a tip item has been performed on the touch screen 15 by the user while watching a live broadcast (a touch operation on a UI image, hereinafter also referred to as an insert operation). If it is determined that an insert operation has not been performed, the process proceeds to step S86, which will be described later. From S86 onwards, as will be described later, a process is performed based on game progress information to place the tip item in the virtual space 600B when the tip item is inserted, so that the tip item can be displayed on the touch screen 15. On the other hand, if it is determined that an insert operation has been performed, a process is performed to determine the reflection position at which the tip item designated by the user is reflected in the virtual space by the processes in steps S82 to S85.

In step S82, it is determined which category the display size of the tipping item specified by the user belongs to. In this embodiment, an example is shown in which the "height" of the tipping item object is used as one indicator of the display size. As shown in the top column of the second column from the left in FIG. 29(A), the multiple types of tipping items are classified into one of the first to third categories based on the value of the display size (height) of the tipping item when the display size (height) of the avatar object 610 is set to 1. In other words, the multiple types of tipping items are classified based on the ratio between the height of the avatar object in the virtual space and the height of each of the multiple types of tipping items.

For example, when the height of the avatar object 610 is "1", a tipping item whose height is less than "0.5" is determined to belong to the first category. When the height of the avatar object 610 is "1", a tipping item whose height is "0.5" or more and less than "0.8" is determined to belong to the second category. When the height of the avatar object 610 is "1", a tipping item whose height is "0.8" or more is determined to belong to the third category. In other words, a tipping item belonging to the first category corresponds to a tipping item that is less than half the height of the avatar object 610 when reflected in the virtual space. A tipping item belonging to the second category corresponds to a tipping item that is half or more the height of the avatar object 610 but less than the shoulder of the avatar object 610 when reflected in the virtual space. A tipping item belonging to the third category corresponds to a tipping item that is equal to or higher than the shoulder of the avatar object 610 when reflected in the virtual space. Each of the multiple types of tip items is determined in advance to belong to a certain category, and information for identifying the category is stored in association with the memory 11 or the like. In step S82, the category is identified from the display size (type) of the tip item.

In step S83, one of a plurality of displayable areas that are predefined according to the classification to which the tip item specified by the user belongs is identified by lottery (random lottery). The displayable area is an area defined in the virtual space (virtual space 600A, virtual space 600B) and is an area defined for each classification as an area in which the specified tip item can be reflected.

In this embodiment, as shown in FIG. 29(A), eight areas a1 to a8 are predefined as displayable areas that can reflect tipping items belonging to the first category, four areas b1 to b4 are predefined as displayable areas that can reflect tipping items belonging to the second category, and three areas c1 to c3 are predefined as displayable areas that can reflect tipping items belonging to the third category. In step S83, one of the multiple displayable areas according to the category to which the tipping item specified by the user belongs is selected by lottery.

Fig. 29(B) is a diagram for explaining a1 to a8, b1 to b4, and c1 to c3 that are determined as displayable areas in the virtual space (virtual space 600A, virtual space 600B). Fig. 29(B) shows the virtual space (virtual space 600A, virtual space 600B) when looking down from above (looking down from the ceiling to the surface equivalent to the floor). The display imitating a human face indicates the position (referred to as initial position m) where the avatar object 610 is placed when live distribution from the distribution side starts (when each part starts, etc.), and the display imitating a camera indicates the position (initial position n) where the virtual camera 620B that specifies the viewing area 640B of the user terminal 100 is placed when live distribution from the distribution side starts (when each part starts, etc.). The initial position of the virtual camera 620B is a position and orientation that allows the avatar object 610 to be viewed from the front.

The range o enclosed by the thick dotted line slanting left and the thick dotted line slanting right that extend from the initial position m to the outer edge of the virtual space indicates the range in which the facial expression (e.g., joy, anger, sadness, happiness, etc.) of the avatar object 610 can be read when the avatar object 610 is facing the initial position direction of the virtual camera 620B at the initial position. In other words, when the user terminal 100 is set up so that the virtual camera 620B is placed within the range o and facing the direction of the avatar object 610, an image in which the facial expression of the avatar object 610 can be read can be displayed on the user terminal 100. For this reason, the range o can be said to be the range in which the virtual cameras 620B of the multiple user terminals 100 are likely to be placed.

Displayable areas a1 to a8 are eight circular areas indicated by thin dotted lines in FIG. 29(B). Displayable areas b1 to b4 are four circular areas indicated by dashed lines in FIG. 29(B). Displayable areas c1 to c3 are three circular areas indicated by dashed lines in FIG. 29(B).

The displayable areas a1 to a8 are areas (first space) that overlap each other so as to surround the initial position m of the avatar object 610. Among the displayable areas a1 to a8, a1 to a3 include an area that is closer to the initial position (predetermined position) of the avatar object 610 (the side to which the avatar object 610 faces, the initial position side of the virtual camera 620B). The displayable areas a1 to a3 also include an area that overlaps with the range o in which the virtual camera 620B is likely to be placed. The area that is closer to the initial position (predetermined position) of the avatar object 610 and overlaps with the range o is also called a specific space. The displayable area corresponding to the first classification includes the displayable areas a1 to a3, and as a result, also includes the specific space. For this reason, if a tipping item is reflected in the displayable areas a1 to a3, there is a risk that the tipping item will impede the visibility of the avatar object 610. However, the first category of tipping items corresponding to the displayable areas a1 to a8 are tipping items that are less than half the height of the avatar object 610. Therefore, even if a tipping item belonging to the first category is placed in a specific space, more than half of the avatar object 610 (for example, at least the upper body) will be visible, preventing problems such as not being able to see the clothing or facial expression of the avatar object 610.

In contrast, the second category of tipping items corresponding to the displayable areas b1 to b4 are tipping items that are more than half the height of the avatar object 610, and the third category of tipping items corresponding to the displayable areas c1 to c3 are tipping items that are more than the shoulders of the avatar object 610. Therefore, if a tipping item belonging to the second or third category is placed in a specific space, at least half of the avatar object 610 may become invisible, and there is a risk that the clothes and facial expression of the avatar object 610 cannot be confirmed due to the tipping item, and the distributed content cannot be enjoyed. However, as shown in FIG. 29(B), the displayable areas b1 to b4 and c1 to c3 are all determined not to include the specific space. Therefore, it is possible to prevent the visibility of the avatar object 610 from being hindered by the tipping items belonging to the second or third category being reflected in the virtual space.

Furthermore, the displayable areas b1 to b4 include areas outside the displayable areas a1 to a8, and are areas that are defined as overlapping ranges so as to surround the left, right, and rear of the avatar object 610, but are areas (second space) that do not include a predetermined range that is within a first distance (approximately the radius of each of the displayable areas a1 to a8) from the initial position m of the avatar object 610, but are areas that surround the avatar object 610 away from the predetermined range. The displayable areas c1 to c3 include areas even further outside the displayable areas b1 to b4, and are areas that are defined as overlapping ranges so as to surround the left, right, and rear of the avatar object 610, but are areas that do not include a predetermined range that is within a second distance from the initial position m of the avatar object 610 that is longer than the first distance, but are areas that surround the avatar object 610 away from the predetermined range. This prevents tipping items belonging to the second and third categories from being reflected in close proximity to the avatar object 610, which would reduce the visibility of the avatar object 610, while also preventing the discomfort that would be caused by a relatively large tipping item being displayed in close proximity to the avatar object 610.

In step S84, the reflection position (display position) where the tip item is reflected is determined by lottery (random lottery) from within the displayable area determined by lottery (random lottery) in step S83. A plurality of positions that can be reflection positions are determined within one displayable area, and in step S84, the reflection position is determined from the positions determined within the determined displayable area. In this way, first, one of the multiple overlapping displayable areas is determined, and then the reflection position (coordinates in the virtual space, etc.) is determined from within that displayable area, so that the randomness of the reflection position of the tip item can be improved, and the tip item can be reflected in the virtual space with randomness. In step S85, the user information 133 of the user who has inserted the tip, the type of tip item, and the insertion information for identifying the determined reflection position (coordinates in the virtual space, etc.) are transmitted to the distribution side such as the distribution terminal 400 via the server 200.

On the distribution side, such as the distribution terminal 400, a tip item reflection response process is executed. In step S91, a message display process is performed to display a message indicating that a tip item has been inserted in the comment section of each user terminal 100 that is currently viewing, based on the insertion information from the user terminal 100. In the message display process, game progress information is delivered that can identify the "user name" who inserted the tip item, the "user icon" that includes a profile image registered in advance by the user, and the fact that a tip item has been inserted.

In step S92, it is determined whether the specified tip item is equal to or greater than a predetermined amount (for example, virtual currency: 500). When it is determined in step S92 that the specified amount is equal to or greater than the predetermined amount, in step S93, a reflection position in the virtual space 600A is identified from the tip information, and a process is performed to reflect (execute) the tip effect with icon at the reflection position. The tip effect with icon is an effect in which, at the reflection position in the virtual space 600A, the user icon of the user who has inserted the tip is first displayed, and then the tip item inserted from the user icon appears in the virtual space. The tip effect with icon is determined for each type of tip item equal to or greater than a predetermined amount, and the execution time is also determined for each type. In step S93, the tip effect with icon progresses in the virtual space 600A, and game progress information for displaying the user icon and the tip item in the virtual space 600B in accordance with the effect is delivered to the user terminal 100 being viewed. The tip effect with icon is reflected so that the user icon and the tip item appear in an orientation that makes them easier to see from the direction near the initial position of the virtual camera 620B. That is, as described in the reference orientation column of FIG. 29(A), the bottom side of the user icon and the front side of the tip item are reflected so as to face the direction of the initial position of the virtual camera 620B. This reduces the burden of performing orientation determination processing, for example, while allowing the inserted tip item to be reflected in a manner that makes it easy to view from near the initial position of the virtual camera 620B (or within range o). Note that display examples of the tip presentation with the icon will be described later with reference to FIG. 31 and FIG. 32.

On the other hand, when it is not determined in step S92 that the amount is equal to or greater than the predetermined amount, in step S94, a reflection position in the virtual space 600A is identified from the insertion information, and a process is performed to reflect (execute) the insertion effect at the reflection position. The insertion effect is an effect in which the inserted tip item appears in the virtual space at the reflection position in the virtual space 600A. In the insertion effect, the tip item appears without displaying the user icon of the inserted user in the virtual space. The insertion effect is determined for each type of tip item less than a predetermined amount, and the execution time is also determined for each type. In step S94, the insertion effect progresses in the virtual space 600A, and game progress information for displaying the tip item in the virtual space 600B in accordance with the effect is delivered to the user terminal 100 being viewed. The insertion effect is reflected so that the tip item appears in a direction that makes it easy to see from the initial position direction of the virtual camera 620B. That is, as described in the reference direction column of FIG. 29(A), the front side of the appeared tip item is reflected so as to face the initial position direction of the virtual camera 620B. An example of the display of the tip effect will be described later with reference to FIG. 30. The game progress information includes not only information for reflecting the tip effect with icon or the object corresponding to the tip effect in the virtual space 600B, but also information for making the avatar object 610 move and speak in response to the movement and speech of the performer in the virtual space 600B, as described above, and information for identifying background objects arranged in the virtual space 600B. Of the processes in FIG. 29(B), only the transmission and reception of information with the user terminal 100 may be performed by the distribution terminal 400, and the other processes may be performed by the game play terminal 300, or all of the processes in FIG. 29(B) may be performed by the distribution terminal 400 and necessary information may be transmitted to the game play terminal 300, or all of the processes in FIG. 29(B) may be performed by the game play terminal 300.

In step S86 of the tip item reflection process on the user terminal side, a message display process is performed based on the received game progress information. In the message display process, the "user name" of the user who inserted the tip item, a "user icon" including a profile image previously registered by that user, and a message such as "I'm sending you my support" to indicate that the tip item has been inserted are displayed in the comment section of the user terminal 100.

In step S87, a throw-in effect display process is performed based on the received game progress information. In the throw-in effect display process, a reflection position is identified based on the game progress information, and an image of the throw-in effect with an icon or an image related to the throw-in effect is generated at the reflection position in the virtual space 600B by real-time rendering, and a field of view image 660 from the virtual camera 620B is displayed on the user terminal 100. This makes it possible to display the throw-in effect with an icon or an image related to the throw-in effect together with the avatar object 610, etc., as an image in the virtual space 600B.

The message display process in step S86 and the tip effect display process in step S87 are performed based on the game progress information. Therefore, the message display process in step S86 and the tip effect display process in step S87 are performed not only on the user terminal 100 owned by the user who inserted the tip item, but also on the HMD 500 viewed by the performer and all user terminals 100 viewing the content being distributed.

(Example of display when tipping items are reflected)
Next, with reference to Fig. 30 to Fig. 32, a display example will be described from when a user inserts a tipping item to when the tipping item is reflected in the virtual space and ends. Fig. 30 shows a display example when a tipping item of less than a predetermined amount is inserted and a tipping performance is executed. Fig. 31 and Fig. 32 show a display example when a tipping item of more than a predetermined amount is inserted and a tipping performance with an icon is executed.

30 to 32, the virtual space 600A (and virtual space 600B) is assumed to have, for example, objects reminiscent of a hospital reception desk arranged as background objects, and a scene in which the avatar object 610 is visiting for a medical examination. Therefore, as shown in FIG. 30(A), the field of view image 660 displayed on the touch screen 15 of the user terminal 100 is assumed to have a reception object 615 corresponding to a reception desk, a chair object 616 corresponding to a waiting chair in front of it (near side), and an avatar object 610 standing up in the lower right area. Note that the point m of the avatar object 610 indicates the initial position of the avatar object 610 at the start of live distribution, and the direction from the initial position to the center of the bottom side of the touch screen 15 corresponds to the initial position of the virtual camera 620B. Also, in FIG. 30 to 32, the performer moves slightly left from the initial position after the start of live distribution, and the avatar object 610 is also displayed at a position slightly moved left from the initial position m. Note that Figures 30 to 32 show display examples in which the virtual camera 620B has not been moved from its initial position, but as described above, the position and orientation of the virtual camera 620B can be moved in response to operations on the touch screen 15 to change the field of view image 660 within the virtual space 600B.

Although not shown, a floor object (e.g., decorated with a predetermined pattern) equivalent to a floor is also placed in virtual space 600A (and virtual space 600B), and an avatar object 610, a reception object 615, and a chair object 616 are placed on the floor object. In addition, on the touch screen 15 of the user terminal 100, a comment field 700 showing comments sent to the avatar object 610 is displayed in the lower left of the screen superimposed on a field of view image 660 in virtual space 600B, and a UI image 701 for selecting a tip item and a UI image 702 for inputting a comment to the avatar object 610 are displayed in the lower right of the screen.

Figure 30 (B) shows an example of a display when UI image 701 is touched to display UI image 711 (window) including UI images 711A-711F depicting tipping items that can be inserted, superimposed on field of view image 660. UI image 711 shows the tipping items that can be inserted and the amount of virtual currency consumed when the tipping items are inserted. For example, UI image 711D shows a bomb tipping item, indicating that "300" virtual currency will be consumed when the tipping item is inserted. UI image 711B shows a jack-in-the-box tipping item, indicating that "500" virtual currency will be consumed when the tipping item is inserted. UI image 711F shows a firework tipping item, indicating that "700" virtual currency will be consumed when the tipping item is inserted. By performing a touch operation (deposit operation) on any of UI images 711A to 711F, the user can consume virtual currency and deposit the tip item that is the target of the touch operation into the virtual space.

FIG. 30(C) shows an example of the display when UI image 711D is touched to insert a "bomb." A "bomb" is a tipping item that belongs to the first category. In addition, since a "bomb" is a tipping item that consumes "300" virtual currency when inserted, it is a tipping item for which a throw-in performance is performed in step S94 of FIG. 28.

When the "bomb" insertion operation is performed, steps S82 to S85 in FIG. 28 are performed on the user terminal 100 owned by user name "555". As a result, a reflection position is identified from within the displayable areas a1 to a8 corresponding to the first category, and insertion information is transmitted. As the reflection position, displayable area a2 is identified, and a position within a2 on the front side of initial position m is identified. Meanwhile, on the distribution side, in steps S91 and S94 in FIG. 28, a message display process and a reflection position in virtual space 600A are identified based on the insertion information from the user terminal 100, and a process for reflecting the "bomb" insertion effect at the reflection position is performed, and game progress information is distributed to the user terminals 100 currently viewing the game, including the user terminal 100 owned by user name "555".

As a result, on the user terminal 100 during viewing, first, by processing step S86 in FIG. 28, the user name "555", the user icon of "555", a message such as "Send cheers", and a comment 700a including a gift box are displayed in the comment section as shown in FIG. 30(C). This makes it possible to appeal to the performer and other users that a tip item will be inserted from the user with the username "555".

In addition, in the user terminal 100 during viewing, the reflection position in the virtual space 600B is specified by the process of step S87 in FIG. 28, and the "bomb" object 617 is displayed so as to gradually appear in the virtual space 600B from the floor object at that position. FIG. 30(C) is an example of a display when the appearance of the "bomb" object 617 is completed. This allows the "bomb" tip item inserted by the user to be reflected in the same virtual space in which the avatar object 610 is placed, and a sense of unity with the content can be created. Even in this way, even if the tip item belonging to the first category is placed in a specific space in front of the initial position m and within the range o, it is possible to prevent the visibility of the avatar object 610 from being hindered as much as possible. Even if the avatar object 610 moves to the initial position m or slightly forward of the initial position m, more than half of the avatar object 610 will not become invisible, and the same effect will be achieved. The performer is also viewing the image in virtual space 600A through HMD 500 and is aware that "bomb" object 617 has been inserted. For this reason, FIG. 30(C) shows the performer reacting to the insertion of "bomb" object 617 by bringing his right hand to his mouth, as if he is surprised, and speaking.

30(D) shows the state when time has passed since the "bomb" object 617 was reflected in the virtual space 600B. The "bomb" object 617 is displayed in the same manner as in FIG. 30(C). In contrast, the comment 700a in the comment section 700 is displayed in a reduced size as shown in comment 700b. The avatar object 610 is shown reacting by bringing the left hand to the opposite right side. FIG. 30(E) shows an example of the display after the "bomb" insertion performance has ended. The "bomb" object 617 becomes invisible after a time period that is predetermined for the "bomb" has passed. This prevents the inserted tip item from being placed in the virtual space indefinitely, which would reduce the attention given to the avatar object 610 or overly scatter the user's gaze.

Figure 31 shows an example of the display when a "jackpot" tip item of a predetermined amount or more is inserted by the user with the name "555" and a throw-in effect is executed. The "jackpot" is a tip item belonging to the third category. In addition, the "jackpot" is a tip item that consumes "500" virtual currency when inserted, and is therefore a tip item for which a throw-in effect with an icon is executed in step S93 of Figure 28.

When the UI image 711B of the UI image 711 shown in FIG. 31(A) is touched to insert the "jack-in-a-box", steps S82 to S85 in FIG. 28 are performed on the user terminal 100 owned by the user name "555". As a result, a reflection position is identified from within the displayable areas c1 to c3 corresponding to the third category, and insertion information is transmitted. As the reflection position, the displayable area c2 is identified, and a position within c2 that is behind the initial position m (for example, a position within the range where c2, b2, and b3 overlap) is identified. On the other hand, on the distribution side, in steps S91 and S93 in FIG. 28, a message display process and a reflection position in the virtual space 600A are identified based on the insertion information from the user terminal 100, and a process for reflecting the insertion performance with the "jack-in-a-box" icon at the reflection position is performed, and game progress information is distributed to the user terminals 100 currently viewing the game, including the user terminal 100 owned by the user name "555".

As a result, on the user terminal 100 during viewing, first, by processing step S86 in FIG. 28, the user name "555", the user icon of "555", a message such as "Send cheers", and a comment 700a including a gift box are displayed in the comment section as shown in FIG. 31 (B). This makes it possible to appeal to the performer and other users that a tip item will be inserted by a user with the username "555".

In addition, on the user terminal 100 during viewing, the reflection position in virtual space 600B is identified by the process of step S87 in FIG. 28, and the user icon 618 of "555" is displayed on the floor object at that position. This makes it easier for the performer and other users to understand that the user of the user icon 618 has deposited a tip item of a predetermined amount or more. Next, when a certain amount of time (such as one second) has elapsed since the user icon 618 was displayed, the process of step S87 in FIG. 28 causes a "jack-in-the-box" box-shaped object 619 to be displayed so as to gradually emerge from the user icon 618 into the virtual space 600B.

Figure 31 (C) is an example of the display when the appearance of the "jack-in-a-box" box-shaped object 619 is complete. In Figure 31 (D), the display of the "jack-in-a-box" box-shaped object 619 continues, but the user icon 618 is hidden and the comment 700a is displayed in a reduced size. Meanwhile, in response to the introduction of the "jack-in-a-box" box-shaped object 619, the avatar object 610 is shown in Figures 31 (C) and 31 (D) reacting as if interested in what will come out and speaking.

When a certain time (e.g., 2 seconds) has elapsed since the box-shaped object 619 of the "jack-in-the-box" was dropped, as shown in FIG. 31(E) and FIG. 31(F), the lid of the box-shaped object 619 of the "jack-in-the-box" opens, and the doll object 620 pops out from the box-shaped object 619 together with a flurry of cherry blossoms, and then the doll object 620 swings from side to side. This allows the tipping item of the "jack-in-the-box" dropped by the user to be reflected in the same virtual space in which the avatar object 610 is placed, and a sense of unity with the content can be created. Even in this way, the tipping item belonging to the third category is not placed in a specific space in front of the initial position m and within the range o as shown in FIG. 29(B), so that the visibility of the avatar object 610 can be prevented as much as possible from being hindered. Even if the avatar object 610 had moved to the initial position m, the tipping item is placed at a distance longer than the first distance from the avatar object 610, so that the visibility of the tipping item can be prevented as much as possible from being hindered. Meanwhile, in response to the doll object 620 popping out, Figures 31(E) and 31(F) show the avatar object 610 reacting as if it were quite surprised and speaking.

In the "jack-in-a-box" icon-attached throw-in effect, as shown in FIG. 31(D), the user icon quickly becomes invisible while leaving the box-shaped object 619 displayed, while objects such as the box-shaped object 619 and doll object 620 remain displayed until a predetermined time (e.g., one minute) has elapsed. Note that in the icon-attached throw-in effect for a tipping item that reflects a predetermined object (e.g., a castle, tower, tree, etc.) such as the "jack-in-a-box," the user icon quickly becomes invisible while leaving the predetermined object displayed, similar to the "jack-in-a-box" icon-attached throw-in effect. If the user icon is displayed for a long period of time, there is a risk that it may not be possible to draw it if a large number of throw-in events are made, but this embodiment can prevent such a problem.

Figure 31 (G) shows an example of the display after the "jack-in-the-box" icon-illustrated throw-in performance has ended. The "jack-in-the-box" object becomes invisible after a predefined time corresponding to the "jack-in-the-box" has elapsed. This prevents the dropped tip item from remaining in the virtual space indefinitely, which could reduce attention to the avatar object 610 or cause the user's gaze to be too distracted.

Figure 32 shows an example of the display when a tipping effect is executed by inserting a tipping item "fireworks" of a predetermined amount or more by the user name "666". "Fireworks" is a tipping item belonging to the third category. Furthermore, since "Fireworks" is a tipping item that consumes "700" virtual currency when inserted, it is a tipping item for which an icon-illustrated tipping effect is executed in step S93 of Figure 28.

When the UI image 711F of the UI image 711 shown in FIG. 32(A) is touched to insert the "fireworks" icon, steps S82 to S85 in FIG. 28 are performed on the user terminal 100 owned by the user name "666". As a result, a reflection position is identified from within the displayable areas c1 to c3 corresponding to the third category, and insertion information is transmitted. As the reflection position, the displayable area c2 is identified, and a position within c2 that is behind the initial position m (for example, a position within the range where c2, b2, and b3 overlap) is identified. On the other hand, on the distribution side, in steps S91 and S93 in FIG. 28, a message display process and a reflection position within the virtual space 600A are identified based on the insertion information from the user terminal 100, and a process for reflecting the insertion performance with the "fireworks" icon at the reflection position is performed, and game progress information is distributed to the user terminals 100 currently viewing the game, including the user terminal 100 owned by the user name "666".

As a result, on the user terminal 100 during viewing, first, by processing step S86 in FIG. 28, the user name "666", the user icon of "666", a message such as "Send cheers", and a comment 700c including a gift box are displayed in the comment section as shown in FIG. 32 (B). This makes it possible to appeal to the performer and other users that a tip item will be inserted from the user with the username "666".

In addition, on the user terminal 100 during viewing, the reflection position in the virtual space 600B is identified by the process of step S87 in FIG. 28, and the user icon 621 of "666" is displayed on the floor object at that position. This makes it easier for the performer and other users to understand that the user of the user icon 621 has deposited a tip item of a predetermined amount or more. Next, when a certain amount of time (such as one second) has elapsed since the user icon 621 was displayed, the process of step S87 in FIG. 28 causes the "fireworks" launching object 622 to be displayed so as to gradually appear in the virtual space 600B from the user icon 621 (see FIG. 32(C) and FIG. 32(D)).

32(E) is a display example when an object 623 such as "Awesome" appears when the "fireworks" are launched and opened (exploded), and FIG. 32(F) is a display example when the "Awesome" transitions to an object 623a in which the "Awesome" fades over time. This allows the "fireworks" tipping item inserted by the user to be reflected in the same virtual space in which the avatar object 610 is placed, creating a sense of unity with the content. Even in this way, the tipping item belonging to the third category is not placed in a specific space in front of the initial position m and within the range o as shown in FIG. 29(B), so that the visibility of the avatar object 610 is prevented as much as possible from being hindered. Even if the avatar object 610 has moved to the initial position m, the tipping item is placed at a distance longer than the first distance from the avatar object 610, so that the visibility of the tipping item is prevented as much as possible from being hindered. Meanwhile, in response to the fireworks being launched and bursting, in Figures 31(E) and 31(F) the avatar object 610 is shown reacting as if extremely surprised or impressed, and speaking.

In FIG. 32(G), after the objects relating to "fireworks" are hidden, the user icon 621a is displayed in a reduced size, and in FIG. 32(H), the user icon is hidden and the comment is also displayed in a reduced size, as in comment 700d. In this way, the throw-in performance with the "fireworks" icon ends when the predetermined time corresponding to "fireworks" has elapsed. This prevents the thrown-in tip item from remaining in the virtual space indefinitely, which could reduce attention to the avatar object 610 or cause the user's gaze to be too scattered.

In a deposit performance accompanied by a prescribed effect, such as the "fireworks" icon deposit performance, the user icon remains displayed until objects such as "Awesome" 623 are hidden, as shown in Figures 32(E) and 32(F), and after the objects related to "fireworks" are hidden and the prescribed effect ends, the user icon is hidden as shown in Figure 32(G) etc. The display time of the user icon is set to be longer during the "fireworks" icon deposit performance, which consumes "700" virtual currency, than during the "jack-in-the-box" icon deposit performance, which consumes "500" virtual currency.

According to this embodiment, as shown in FIG. 28(A), on the user terminal 100 side, a tipping item is selected in response to an input operation by the user, and the reflection position is identified based on a classification according to the display size of the tipping item, so that the tipping item can be displayed at the reflection position in the virtual space 600B in which the avatar object 610 is located. This makes it possible to minimize the reduction in visibility of the avatar object 610, which constitutes part of the content, caused by the tipping item, compared to when the reflection position is determined regardless of the display size of the tipping item. In addition, because the tipping item is reflected in the virtual space, a sense of unity can be created between the tipping item and the avatar object 610.

As shown in Fig. 29(A) and (B), the positions that can be specified as the reflection positions according to the display size are as follows: for tipping items belonging to the first category that are less than half the height (predetermined size) of the avatar object 610, any of the displayable areas a1 to a8 that are in front of the initial position m and include a specific space within the range o; for tipping items belonging to the second and third categories that are more than half the height (predetermined size) of the avatar object 610, any of the displayable areas b1 to b4 or c1 to c3 that do not include a specific space are specified. The initial position m is the position where the avatar object 610 is placed at the start of the live broadcast, and the specific space is the space (range o) on the side where the avatar object 610 is facing at the start of the live broadcast, where the clothes and facial expression of the avatar object 610 can be easily confirmed. As a result, even if a tipping item belonging to the first category is placed in the specific space, at least half of the avatar object 610 (e.g., at least the upper body) will be visible, preventing problems such as not being able to see the clothing or facial expression of the avatar object 610. Furthermore, tipping items belonging to the second or third category will not be placed in the specific space, preventing the visibility of the avatar object 610 from being hindered by tipping items belonging to the second or third category.

29B, the displayable areas b1 to b4 include areas outside the displayable areas a1 to a8 and are areas that are defined as overlapping ranges so as to surround the left, right, and rear of the avatar object 610, but are areas (second space) that do not include a predetermined range that is within a first distance (approximately the radius of each of the displayable areas a1 to a8) from the initial position m of the avatar object 610 and surround the avatar object 610 from the predetermined range. The displayable areas c1 to c3 include areas even further outside the displayable areas b1 to b4 and are areas that are defined as overlapping ranges so as to surround the left, right, and rear of the avatar object 610, but are areas that do not include a predetermined range that is within a second distance from the initial position m of the avatar object 610 that is longer than the first distance and surround the avatar object 610 from the predetermined range. This prevents tipping items belonging to the second or third category from being reflected in close proximity to the avatar object 610 (e.g., directly to the side or directly behind), which would reduce the visibility of the avatar object 610, while also preventing the discomfort that would be caused by a relatively large tipping item being displayed in close proximity to the avatar object 610.

The displayable areas are defined as displayable areas a1 to a8 capable of reflecting tipping items whose display size belongs to the first category, displayable areas b1 to b4 capable of reflecting tipping items whose display size belongs to the second category, and displayable areas c1 to b3 capable of reflecting tipping items whose display size belongs to the third category. As shown in steps S83 and S84 of FIG. 28, the position that can be identified as the reflection position according to the display size is first identified as one of the displayable areas corresponding to the category according to the display size of the selected tipping item, and then identified from within that displayable area. This can improve the randomness of the reflection position of the tipping item, and can reflect the tipping item in the virtual space with randomness.

As shown in FIG. 29(A), the orientation of the tip item displayed in each of the displayable areas a1 to a8, b1 to b4, and c1 to c3 is set to the direction of the initial position n of the virtual camera 620B, and the tip item is reflected in that direction. This reduces the burden of performing processes such as determining the orientation of the tip item, and allows the inserted tip item to be reflected in a manner that is easy to view from near the initial position (or range o) of the virtual camera 620B.

Also, as shown in step S93 of FIG. 28(B) and in FIGS. 31 and 32, when a tipping item of a predetermined amount or more is reflected in the virtual space, a user icon related to the user who deposited the tipping item is displayed at the specified reflection position, and then the tipping item is reflected. This makes it easier for performers and other users to understand that a tipping item of a predetermined amount or more has been deposited by the user indicated by user icon 618. Meanwhile, the user icon is set to a fixed display size regardless of the display size of the deposited tipping item. This prevents the user icon from being displayed in various sizes, which can make the icon visually complicated and prevent the icon from effectively being displayed.

The tipping item is displayed so that it gradually appears from the user icon into the virtual space. Therefore, when a tipping item is inserted, the user's icon is drawn to the user's attention first, and then the tipping item appears, which makes it possible to prevent performers and other users from noticing and overlooking the insertion of a tipping item.

When a tipping item of less than a predetermined amount is reflected in the virtual space, the tipping item gradually appears at a specified reflection position as shown in FIG. 30, and the user icon is not displayed. This allows a user who has inserted a tipping item of a predetermined amount or more to feel a sense of superiority. Furthermore, as shown in FIG. 31(D) and FIG. 32(H), the user icon is hidden when a time period predetermined according to the type of tipping item has elapsed. In the above embodiment, the display time of the user icon is set longer for a tipping item that consumes a larger amount of virtual currency (the "fireworks" shown in FIG. 32) than for a tipping item that consumes a smaller amount of virtual currency (the "jack-in-the-box" shown in FIG. 31). This increases the motivation to insert a tipping item that consumes a larger amount of virtual currency.

In addition, regardless of the amount of virtual currency consumed, the tipping item is reflected (displayed) in a manner that it appears to grow from the bottom, which corresponds to the floor, in the virtual space toward the top, which corresponds to the ceiling, improving its appearance. Note that, regardless of the amount of virtual currency consumed, when a tipping item is inserted, the aforementioned icon-illustrated insertion performance may be executed. This can increase the satisfaction of the user who inserted the tipping item, and can increase the appeal to the performer and other users.

<Modification>
Modifications of the above-described embodiment are listed below.

(1) In the above embodiment, as shown in FIG. 28(A), the reflection position of the inserted tipping item is determined on the user terminal 100 side, and based on the determination result, the distribution side such as the distribution terminal 400 reflects the tipping item at the reflection position in the virtual space 600A (executes the tipping performance) and distributes corresponding game progress information, thereby similarly reflecting the tipping item at the reflection position in the virtual space 600B of the user terminal 100 being viewed. As a result, a process for determining the reflection position is performed in each user terminal 100 that has inserted the tipping item, thereby reducing the processing burden on the distribution side such as the distribution terminal 400. However, the reflection position of the inserted tipping item is not limited to being determined on the user terminal 100 side, and may be determined on the distribution side such as the distribution terminal 400. For example, information including the type of tipping item inserted from the user terminal 100 side based on the user's tipping operation is output, and the distribution side such as the distribution terminal 400 determines the reflection position based on the classification according to the display size of the tipping item based on the information, reflects the tipping item at the reflection position (executes the tipping performance) and distributes corresponding game progress information, thereby reflecting the tipping item at the reflection position in the virtual space 600B in the user terminal 100 being viewed. In addition, the information output from the user terminal 100 side is not limited to information that can identify the type of tipping item, but may be information that can identify the classification according to the display size of the tipping item, for example. This also makes it possible to prevent the visibility of the avatar object 610 that constitutes part of the content from being reduced by the tipping item as much as possible, compared to determining the reflection position regardless of the display size of the tipping item. In addition, since the tipping item is reflected in the virtual space, a sense of unity can be created between the tipping item and the avatar object 610.

(2) In the above embodiment, half the height of the avatar object 610 was given as an example of a predetermined size that is smaller than the character. However, the predetermined size is not limited to this as long as it is less than the height of the avatar object, and may be, for example, 3/4 the height of the avatar object 610 or a height equivalent to the shoulders. In addition, although the "height" of the object was given as an example of an index related to the display size, the display size is not limited to "height" and may instead or in addition be the "width" of the object, or a parameter equivalent to the "display area" or "volume" of the object.

(3) In the above embodiment, an example was described in which the tipping items belonging to the first category, which have a relatively small display size, are randomly determined from among the displayable areas a1 to a8 shown in FIG. 29. However, if the tipping items belonging to the first category are reflected, for example, in the displayable area a6 corresponding to the rear side of the initial position of the avatar object 610, there is a risk that the visibility of the tipping items will be hindered by the avatar object 610. For this reason, for example, the displayable areas a1 to a3 that are closer to the initial position (predetermined position) of the avatar object 610 may be set as the displayable areas corresponding to the tipping items belonging to the first category, and the reflection position may be determined from among the displayable areas a1 to a3. In addition, the positions of the tipping items belonging to the first category may be determined to be positions in the displayable areas a1 to a3 that are closer to the initial position (predetermined position) of the avatar object 610 at a higher rate than positions in the displayable areas a4 to a8. This makes it possible to prevent as much as possible the visibility of a tip item belonging to the first category from being obstructed by the avatar object 610, even if the tip item is reflected.

In addition, an example has been described in which tipping items belonging to the second or third category, which have a relatively large display size, are located in front of the initial position (predetermined position) of the avatar object 610 as shown in FIG. 29, and are not reflected in the specific space, which is an area that overlaps with range o. However, tipping items belonging to the second or third category, which have a relatively large display size, may be reflected in a position within the specific space at an extremely low rate, for example. This provides a sense of surprise to the user when tipping items belonging to the second or third category are reflected in the specific space at a low rate, thereby increasing the interest of the content.

(4) In the above embodiment, an example was described in which the multiple displayable areas are determined based on the initial position of the avatar object 610 at the start of the live broadcast, as shown in FIG. 29(B). However, the multiple displayable areas may be determined based on the current position of the avatar object 610 when determining the reflection position of the inserted tip item. Also, an example was described in which the specific space is the space toward which the avatar object 610 is facing when initially positioned at the start of the live broadcast. However, the specific space may be the space toward which the avatar object 610 is actually facing when determining the reflection position of the inserted tip item.

Also, as shown in FIG. 29(A), an example has been described in which the direction (reference direction) in which the inserted tipping item is reflected is the direction of the initial position of the virtual camera 620B. However, the direction (reference direction) in which the inserted tipping item is reflected may be the direction facing the current position of the avatar object 610 when the reflection position of the inserted tipping item is determined, or may be the same direction as the direction in which the avatar object 610 is actually facing (the direction of line of sight). In addition, the direction (reference direction) in which the inserted tipping item is reflected is not limited to being determined to be the same position regardless of which displayable area it is reflected in, but may be determined to be different for each displayable area. For example, for a certain displayable area, the direction facing the current position of the avatar object 610 when the reflection position of the inserted tipping item is determined may be determined as the reference direction, while for other displayable areas, the direction of the initial position of the virtual camera 620B may be determined as the reference direction. This allows the orientation of the inserted tipping item to be contingent and random.

(5) In the above embodiment, the reflection position of the inserted tipping item is determined by lottery, so it may be determined to be a position that overlaps (collides with) an already reflected tipping item. In this case, the new tipping item may be reflected in a manner that it grows from a lower position corresponding to the floor toward an upper position corresponding to the ceiling, and the previously reflected tipping item may be pushed aside and fall over, and then become invisible. This allows the new tipping item to be displayed with priority over the previous tipping item, and the satisfaction of the user who inserted the tipping item can be improved. Note that if the reflection positions overlap in a device that causes a tipping item to appear from an upper position to a lower position, the new tipping item may collide with the previous tipping item, causing the new tipping item to fall over and not be displayed with a good appearance. However, the above-mentioned configuration can eliminate such a problem, and the new tipping item can be displayed with a good appearance, thereby improving the satisfaction of the user who inserted the tipping item.

(6) In the above embodiment, the image of the virtual space 600B is displayed on the user terminal 100 using a real-time rendering method. However, the image of the virtual space 600B may also be displayed on the user terminal 100 using a video distribution method that distributes video data composed of multiple frame images.

<Additional Notes>
The matters described in the above embodiments will be supplemented below.

(Note 1):
According to one aspect of an embodiment of the present disclosure, a game program is executed on a terminal device (user terminal 100) having a processor, a memory, and a display unit, which causes the processor to execute the steps of displaying an image of a virtual space (virtual space 600B) including a character (avatar object 610) that moves in response to the movements of an actor, selecting an object to be displayed in the virtual space from among a plurality of types of objects having at least different display sizes (FIG. 30(B), step S81 of FIG. 28), and specifying a position in the virtual space at which the object selected in the selecting step is to be displayed, the position corresponding to the display size of the selected object (steps S82-S84 of FIG. 28), and the displaying step can display an image in which the selected object is displayed at the position in the virtual space specified in the specifying step (step S87 of FIG. 28).

(Note 2):
According to one aspect of an embodiment of the present disclosure, a game program executed on a computer (distribution terminal 400, game playing terminal 300) having a processor and a memory, causes the processor to execute a step of making an image of a virtual space including a character (avatar object 610) that moves in response to the movements of an actor displayable on a terminal device (user terminal 100) (see FIG. 8, etc.), and a step of specifying a position in the virtual space to display an object selected by a user as an object to be displayed in the virtual space from among a plurality of types of objects having at least different display sizes, at a position corresponding to the display size of the selected object (see steps S82 to S84 of FIG. 28), and the step of making the object displayable can display an image in which the selected object is displayed at the position in the virtual space specified by the specifying step (steps S93 and S94 of FIG. 28).

(Appendix 3):
In (Supplementary Note 1) or (Supplementary Note 2), a position that can be identified as a position according to the display size of the selected object is any position in a first space (displayable areas a1 to a8) including a specific space in front of the specific position in the virtual space when the display size of the selected object is smaller than the character and less than a predetermined size (for example, half the height of avatar object 610), and any position in a second space (displayable areas b1 to b4, c1 to c3) that does not include the specific space when the display size of the selected object is equal to or larger than the predetermined size (see FIG. 29(A) 9).

(Note 4):
In (Supplementary Note 3), the predetermined position is a position (initial position m) where the character is initially placed in the virtual space, and the specific space is a space (range o) to the side where the character is facing when the character is initially placed.

(Note 5):
In (Supplementary Note 4), the first space is a space that includes the specific space and surrounds the character when the character is initially placed, and the second space is a space that does not include the specific space and does not include a space within a predetermined range surrounding the character's initial placement, and surrounds the character away from the character's initial placement within the predetermined range (see FIG. 29 (B)).

(Note 6):
In any of (Supplementary Note 1) to (Supplementary Note 5), the virtual space is defined with a plurality of first areas (displayable areas a1 to a8) in which objects whose display sizes belong to a first range (first classification) can be displayed, and a plurality of second areas (displayable areas b1 to b4, c1 to c3) in which objects whose display sizes belong to a second range (second classification, third classification) can be displayed. The position specified as the position according to the display size of the selected object is a position specified within the first area after one of the plurality of first areas is specified when the display size of the selected object belongs to the first range, and is a position specified within the second area after one of the plurality of second areas is specified when the display size of the selected object belongs to the second range (see steps S83 and S84 in FIG. 28).

(Appendix 7):
In (Supplementary Note 6), a direction (reference direction) for displaying an object is determined for each of the plurality of first regions and the plurality of second regions, and the displaying step can display an image in which the selected object is displayed in the virtual space in the direction determined for a specific region among the plurality of first regions and the plurality of second regions (steps S93 and S94 in FIG. 28).

(Appendix 8):
In any of (Supplementary Note 1) to (Supplementary Note 7), when the selected object is displayed in the virtual space, an icon image related to the user who selected the object is displayed at the position in the virtual space identified in the identifying step, and then the selected object is displayed (see step S93 in Figure 28, Figures 31 and 32).

(Appendix 9):
In (Supplementary Note 8), the icon image is an image that is set to a constant size regardless of the display size of the selected object.

(Appendix 10):
In (Supplementary Note 8) or (Supplementary Note 9), the selected object is displayed so as to emerge from the icon image into the virtual space (so as to appear to grow from a lower position toward an upper position).

(Appendix 11):
In any of (Supplementary Note 8) to (Supplementary Note 10), if the selected object is a first type of object (the amount of virtual currency consumed is less than 500), the object is displayed without displaying the icon image (Figure 30). On the other hand, if the selected object is a second type of object (the amount of virtual currency consumed is 500 or more, YES in step S92), the icon image is displayed and then the object is displayed (Figures 31 and 32).

(Appendix 12):
In any one of (Supplementary Note 8) to (Supplementary Note 11), after the icon image is displayed, the icon image is made non-displayable when a predetermined time has elapsed.

(Appendix 13):
According to one aspect of an embodiment of the present disclosure, there is provided a game method executed in a terminal device (user terminal 100) having a processor, a memory, and a display unit, the method including the steps of: displaying an image of a virtual space (virtual space 600B) including a character (avatar object 610) that moves in response to the movements of an actor, selecting an object to be displayed in the virtual space from among a plurality of types of objects having at least different display sizes (FIG. 30(B), step S81 in FIG. 28); and specifying a position in the virtual space at which the object selected in the selecting step is to be displayed, the position corresponding to the display size of the selected object (steps S82 to S84 in FIG. 28), and the displaying step can display an image in which the selected object is displayed at the position in the virtual space specified in the specifying step (step S87 in FIG. 28).

(Appendix 14):
According to one aspect of an embodiment of the present disclosure, a terminal device (user terminal 100) includes a memory unit that stores a game program, and a control unit that controls operation of the terminal device by executing the game program, the control unit includes a step of displaying an image of a virtual space (virtual space 600B) including a character (avatar object 610) that moves in response to movements of an actor, a step of selecting an object to be displayed in the virtual space from among a plurality of types of objects having at least different display sizes (FIG. 30(B), step S81 of FIG. 28), and a step of specifying a position in the virtual space at which the object selected in the selecting step is to be displayed, the position corresponding to the display size of the selected object (steps S82 to S84 of FIG. 28), and the displaying step can display an image in which the selected object is displayed at the position in the virtual space specified in the specifying step (step S87 of FIG. 28).

(Appendix 15):
According to one aspect of an embodiment of the present disclosure, there is provided a game method executed on a computer (distribution terminal 400, game playing terminal 300) having a processor and a memory, the game method including the steps of: making an image of a virtual space including a character (avatar object 610) that moves in response to the movements of an actor displayed on the terminal device (user terminal 100) (see FIG. 8, etc.); and specifying a position in the virtual space at which an object selected by a user as an object to be displayed in the virtual space from among a plurality of types of objects having at least different display sizes is to be displayed, the position corresponding to the display size of the selected object (see steps S82 to S84 in FIG. 28), wherein the making-display step can display an image in which the selected object is displayed at the position in the virtual space specified by the specifying step (steps S93 and S94 in FIG. 28).

(Appendix 16):
According to one aspect of an embodiment of the present disclosure, a computer (distribution terminal 400, game playing terminal 300) includes a memory unit that stores a game program, and a control unit that controls operation of the computer by executing the game program, and the control unit includes a step of making an image of a virtual space including a character (avatar object 610) that moves in response to movements of an actor displayable on a terminal device (user terminal 100) (see FIG. 8, etc.), and a step of specifying a position in the virtual space to display an object selected by a user from a plurality of types of objects having at least different display sizes as an object to be displayed in the virtual space, at a position corresponding to the display size of the selected object (see steps S82 to S84 of FIG. 28), and the step of making the object displayable can display an image in which the selected object is displayed at the position in the virtual space specified by the specifying step (steps S93 and S94 of FIG. 28).

[Software implementation example]
The control blocks (particularly control units 110, 210, 310, 410) of the user terminal 100, server 200, game playing terminal 300 (HMD set 1000), and distribution terminal 400 may be realized by logic circuits (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

In the latter case, the user terminal 100, the server 200, the game play terminal 300 (HMD set 1000), and the distribution terminal 400 are equipped with a computer that executes instructions of a program, which is software that realizes each function. This computer is equipped with, for example, one or more processors, and a computer-readable recording medium that stores the program. The object of the present invention is achieved by the processor reading the program from the recording medium and executing it in the computer. The processor can be, for example, a CPU (Central Processing Unit). The recording medium can be a "non-transient tangible medium", such as a ROM (Read Only Memory), as well as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, etc. The computer can also be equipped with a RAM (Random Access Memory) that expands the program. The program can also be supplied to the computer via any transmission medium (such as a communication network or a broadcast wave) that can transmit the program. Note that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. The technical scope of the present invention also includes embodiments obtained by appropriately combining the technical means disclosed in different embodiments.

1 System, 2 Network, 3, 3A, 3B User (first user), 4 Player (performer), 10, 20, 30, 40 Processor, 11, 21, 31, 41 Memory, 12, 22, 32, 42 Storage, 13, 23, 33, 43 Communication IF, 14, 24, 34, 44 Input/output IF, 15, 45 Touch screen, 17 Camera, 18 Distance measurement sensor, 51 Monitor, 52 Gaze sensor, 53 First camera, 54 Second camera, 55 Microphone, 56 Speaker, 100, 100A, 100B, 100C User terminal (computer, first computer, first information processing device), 110, 210, 310, 410 Control unit (first control unit, second control unit), 111, 311, 413 Operation reception unit, 112, 312, 412 Display control unit, 113, 313 UI control unit, 114, 314 Animation generation unit, 115, 315 Game progress unit, 116, 316 Virtual space control unit, 117 Video playback unit, 120, 220, 320, 420 Memory unit (first memory unit, second memory unit), 131, 231, 331 Game program (program, first program), 132, 232, 332 Game information, 133, 233, 333 User information, 151, 451 Input unit, 152, 452 Display unit (display), 200 Server, 211 Communication mediation unit, 212 Log generation unit, 213 List generation unit, 234, 421 User list, 300 Game play terminal (external device, second external device), 317 Reaction processing unit, 400 Distribution terminal (external, first external device, computer, second information processing device), 411 Communication control unit, 414 Voice reception unit, 415 Motion identification unit, 416 Operation instruction data generation unit, 422 Motion list, 423 Distribution program (program, second program), 540, 1020, 1021 Controller, 500 HMD, 510 HMD sensor, 520 Motion sensor, 530 Display, 600A, 600B Virtual space, 610 Avatar object (character), 620A, 620B Virtual camera, 631, 632, 633, 634 Object, 640A, 640B View area, 650, 660 View image, 671 Enemy object, 672, 673 Obstacle object, 674 Performance object, 691, 692 Speech, 701, 702, 703A, 70B, 704A, 704B, 705, 706, 711, 711A, 711B, 711C, 711D, 722, 723, 745, 745A, 745B, 745C, 752, 762, 763, 930, 2011, 2022, 2031, 2032, 2033, 2034, 2037, 2038, 2051, 2063, 2072, 2073, 2075 UI image (message UI, UI), 721 Download screen, 731 User list screen (list), 732, 732A, 732B, 732C, 742, 742A, 742B, 742C Record image, 733, 733A, 733B, 733C User name, 734, 734A, 734B, 734C Tag information, 735, 735A, 735B, 735C Icon, 741 Motion list screen (options), 743, 743A, 743B, 743C Motion name, 744, 744A, 744B, 744C, 753 Motion image, 751 Distribution screen, 761 Distribution completion screen, 810A, 810B Motion video, 820A, 820B Speech voice, 910A, 910B Video, 920A, 920B Voice, 1000 HMD set, 1010 Object, 1030 Storage medium

Claims (11)

A program executed on a computer,
The computer,
A display means for displaying an image of a virtual space including a character that moves in response to the movement of a performer;
a selection means for selecting an object to be displayed in the virtual space from among a plurality of types of objects having at least different display sizes;
a specifying means for specifying a position in the virtual space where the object selected by the selecting means is to be displayed, the position corresponding to a display size of the selected object;
the display means is capable of displaying the selected object at a position in the virtual space identified by the identification means,
When displaying the selected object in the virtual space, the program first displays an icon image associated with the user who selected the object at the position in the virtual space identified by the identification means, and then displays the selected object. A program executed on a computer,
The computer,
A display means for displaying on a terminal device an image of a virtual space including a character that moves in response to the movement of a performer;
a specifying means for specifying a position in the virtual space in which an object selected by a user from among a plurality of types of objects having at least different display sizes as an object to be displayed in the virtual space is to be displayed at a position corresponding to the display size of the selected object;
the display means is capable of displaying the selected object at a position in the virtual space identified by the identification means,
When displaying the selected object in the virtual space, the program first displays an icon image associated with the user who selected the object at the position in the virtual space identified by the identification means, and then displays the selected object. The program according to claim 1 or 2, wherein the position that can be identified as the position according to the display size of the selected object is any position in a first space including a specific space in front of the specific position in the virtual space when the display size of the selected object is smaller than the character and is less than a predetermined size, and any position in a second space not including the specific space when the display size of the selected object is equal to or larger than the predetermined size. the predetermined position is a position where the character is initially placed in the virtual space,
The program according to claim 3 , wherein the specific space is a space toward which the character is facing when the character is initially positioned. the first space is a space including the specific space and surrounding the character when the character is initially placed;
5. The program according to claim 4, wherein the second space does not include the specific space and does not include a space within a predetermined range surrounding the initial position of the character, but is a space that is away from the initial position of the character within the predetermined range and surrounds the character. a plurality of first regions in which objects whose display sizes belong to a first range can be displayed and a plurality of second regions in which objects whose display sizes belong to a second range can be displayed are defined in the virtual space;
A program according to any one of claims 1 to 5, wherein the position identified as the position corresponding to the display size of the selected object is a position identified from within the first area after one of the multiple first areas is identified when the display size of the selected object belongs to the first range, and is a position identified from within the second area after one of the multiple second areas is identified when the display size of the selected object belongs to the second range. an orientation in which an object is displayed in each of the first regions and the second regions is determined; and
7. The program according to claim 6, wherein the display means is capable of displaying the selected object in the virtual space in an orientation that is determined in a specific area among the plurality of first areas and the plurality of second areas. 8. The program according to claim 1 , wherein the icon image is an image that is set to a constant size regardless of the display size of the selected object. 9. The program according to claim 1, wherein the selected object is displayed so as to appear from the icon image into the virtual space. A program according to any one of claims 1 to 9, wherein, if the selected object is a first type of object, the object is displayed without displaying the icon image, whereas, if the selected object is a second type of object, the icon image is displayed and then the object is displayed. 11. The program according to claim 1, further comprising: making the icon image non-displayable when a predetermined time has elapsed after the icon image is displayed.

Images