JP7680850B2 - Computer system, content viewing system and program - Google Patents

Description

The present invention relates to a computer system for providing video to viewers.

A technology for live streaming of video content is known that accepts voting operations from viewers to express praise for the video provided or support for the broadcaster (contributor) (see Patent Document 1). Some technologies also have various posting functions that allow viewers to comment on the video provided or provide monetary rewards to the broadcaster. In particular, the provision of rewards from viewers to contributors is called "tipping," after the donations from spectators to street performers, or is also called "gifts." Hereinafter, these will be collectively referred to as "tipping." When a viewer posts a tip or comment using this posting function, the posted information is reflected on the viewing screen, for example by the appearance of a corresponding object.

JP 2018-182546 A

The posting function described above provides posters with income according to the amount of tips they send, which has the effect of encouraging further posts. For viewers, it also brings joy to self-expression by publicly expressing one's will and position to other viewers through praise, support, etc., and is expected to have the effect of livening up the atmosphere as tips attract more tips. However, as the atmosphere becomes more lively and the number of posts such as tips increases, the number of objects displayed on the viewing screen also increases, which can lead to problems such as a deterioration in visibility of the essential video.

The problem that this invention aims to solve is to provide a technology that can ensure the visibility of the provided video while accepting viewer actions such as tips made by viewers in response to the provision of the video.

The first invention for solving the above problem is a computer system for making objects corresponding to the viewer actions of each viewer appear in a virtual viewing space and providing the target viewer with an image of the viewing space seen from the viewpoint of the target viewer, the computer system comprising: a field of view securing area setting means (e.g., the field of view securing area setting unit 237 in FIG. 12) for setting a field of view securing area for the target viewer; and a condition satisfying object display control means (e.g., the condition satisfying object display control unit 239 in FIG. 12) for differentiating the display control of objects that satisfy an area condition indicating that the object passes through or is located within the field of view securing area (hereinafter referred to as a "condition satisfying object") from the display control of objects that do not satisfy the condition (hereinafter referred to as a "condition non-satisfying object").

According to the first invention, it is possible to provide a viewer with an image of a virtual viewing space in which a corresponding object appears when the viewer performs a viewer action. Then, a field of view securing area is set for each viewer in this viewing space, and objects passing through the field of view securing area and condition-satisfying objects located within the field of view securing area are provided to each viewer with different display control from other condition-unsatisfying objects. This makes it possible to secure the visibility of the provided image while accepting viewer actions such as tipping made by the viewer in response to the provision of the image.

The second invention is a computer system according to the first invention, further comprising an object movement control means (e.g., the object movement control unit 235 in FIG. 12) for making the object appear in the viewing space and move it along a given trajectory.

According to the second invention, when a viewer action is performed by the viewer, a corresponding object can be made to appear in the viewing space and move within the viewing space along a given trajectory.

The third invention is a computer system according to the second invention, in which the object movement control means moves the object along the trajectory until a predetermined stopping condition is satisfied, and the condition-satisfying object display control means detects the condition-satisfying object from among the objects being moved by the object movement control means.

According to the third invention, it is possible to detect a condition-satisfying object among objects under movement control, and then control the display of the object depending on whether it is a condition-satisfying object or a condition-non-satisfying object.

The fourth invention is a computer system according to the second or third invention, in which the condition satisfying object display control means has a trajectory change control means (e.g., the condition satisfying object display control unit 239 in FIG. 12) that changes the trajectory of the condition satisfying object in a direction away from the field of view securing area.

According to the fourth invention, it is possible to change the trajectory of the condition-satisfying object and move it in a direction away from the field of view.

The fifth invention is a computer system according to any one of the first to fourth inventions, in which the condition-satisfying object display control means has a form change control means (e.g., form change control unit 243 of FIG. 12) that controls the display form of the condition-satisfying object to be different from the display form of the condition-non-satisfying object.

According to the fifth invention, condition-satisfying objects and condition-unsatisfying objects can be displayed in different display forms.

The sixth invention is a computer system according to the fifth invention, in which the form change control means changes the display form of the condition-satisfying object to one that is more transparent and/or smaller in size than the condition-non-satisfying object.

According to the sixth invention, the condition-satisfying object can be displayed with a higher transparency than the condition-unsatisfying object, or with a smaller size than the condition-unsatisfying object.

The seventh invention is a computer system according to any one of the first to sixth inventions, in which a character whose movement is controlled based on a given movement control is placed in the viewing space, and the field of view securing area setting means sets the field of view securing area based on the character.

According to the seventh invention, the field of view can be set based on a character whose movement is controlled within the viewing space.

The eighth invention is a computer system according to the seventh invention, in which the field of view securing area setting means sets the field of view securing area based on the positional relationship between the viewpoint and the character.

According to the eighth invention, the field of view can be set based on the positional relationship between the character and the viewpoint in the viewing space.

The ninth invention is a computer system according to the seventh invention, in which the field of view securing area setting means sets the field of view securing area based on a setting specification for the field of view securing area associated with the character.

According to the ninth aspect of the invention, for example, it is possible to determine the setting specifications for the field of view securing area for each character, and to set the field of view securing area using the setting specifications for the character in the viewing space.

The tenth invention is the computer system of the eighth invention, further comprising a viewpoint control means (e.g., viewpoint control unit 245 in FIG. 12) that changes the direction of the viewpoint and/or the position of the viewpoint based on an operational input from the target viewer, and the field of view securing area setting means updates and sets the field of view securing area in response to the control of the viewpoint control means.

According to the tenth aspect of the invention, when the direction or position of the viewpoint is changed based on the viewer's operational input, the field of view securing area can be updated and set according to the changed direction or position of the viewpoint.

An eleventh invention is a computer system according to any one of the first to tenth inventions, in which the field of view securing area setting means variably sets the field of view securing area based on the input status of the viewer action.

According to the eleventh invention, the field of view can be variably set according to the input status of the viewer's actions.

A twelfth invention is a computer system according to any one of the first to eleventh inventions, in which the condition satisfying object display control means variably controls the display control of the condition satisfying object based on the input status of the viewer action.

According to the twelfth invention, by performing display control according to the input status of the viewer action, it is possible to variably control the display control of the condition satisfying object.

A thirteenth invention is a computer system according to any one of the first to twelfth inventions, in which the field of view securing area setting means sets the field of view securing area within the field of view range of the viewpoint.

According to the thirteenth invention, the field of view securing area for the target viewer can be set within the field of view of the viewpoint for the target viewer.

A fourteenth invention is a computer system according to any one of the first to twelfth inventions, in which the condition satisfying object display control means excludes the object related to the target viewer from the condition satisfying objects, and detects the condition satisfying object from among the objects related to viewers other than the target viewer.

According to the fourteenth invention, the object of a viewer action performed by the target viewer is not regarded as a condition-satisfying object, and a condition-satisfying object is detected from the object of a viewer action performed by a viewer other than the target viewer, and the display of the object can be controlled according to whether it is a condition-satisfying object or a condition-non-satisfying object.

The fifteenth invention is a computer system according to any one of the first to fourteenth inventions, in which the viewer action is an operational input input by consuming a given consumption value from a parameter value possessed by the viewer.

According to the fifteenth invention, a viewer can input a viewer action by consuming his/her own parameter value.

The sixteenth aspect of the invention is a content viewing system (e.g., the content viewing system 1000 in FIG. 1) that includes a server system (e.g., the server system 1100 in FIG. 1) that is a computer system of any one of the first to fifteenth aspects of the invention that is communicatively connected to the viewer terminal of each viewer, and the viewer terminal (e.g., the viewer terminal 1500 in FIG. 1).

According to the sixteenth aspect, it is possible to realize a content viewing system that has the same effect as any one of the first to fifteenth aspects.

The seventeenth invention is a program for causing a computer system to make objects corresponding to the viewer actions of each viewer appear in a virtual viewing space and provide the target viewer with an image of the viewing space seen from the viewpoint of the target viewer, the program causing the computer system to function as a field of view securing area setting means for setting a field of view securing area for the target viewer, and a condition satisfying object display control means for differentiating the display control of objects that satisfy an area condition indicating that the object passes through or is located within the field of view securing area (hereinafter referred to as a "condition satisfying object") from the display control of objects that do not satisfy the condition (hereinafter referred to as a "condition non-satisfying object") (e.g., video provision program 504 in FIG. 18).

According to the seventeenth aspect, a program that has the same effect as the first aspect can be realized.

FIG. 1 is a diagram showing an example of the overall configuration of a content viewing system. FIG. 2 is a diagram showing an example of the device configuration of a viewer terminal. FIG. 1 is a diagram showing an example of a viewing space. FIG. 4 is a diagram showing an example of a viewing screen. FIG. 13 is a diagram showing an example of an object selection screen. FIG. 13 is a diagram showing an example of an action operation for executing posting. 6A and 6B are diagrams for explaining a field of view securing area setting process. 13 is another diagram for explaining the field-of-view securing area setting process. FIG. FIG. 13 is a diagram showing an example of a coefficient K. 11A and 11B are diagrams for explaining changes in the display form of a condition satisfying object. 13 is another diagram for explaining the change in the display form of the condition satisfying object. FIG. FIG. 2 is a block diagram showing an example of the functional configuration of the server system. FIG. 4 is a diagram showing an example of a data configuration of user management data. FIG. 4 is a diagram showing an example of the data configuration of video provision management data. FIG. 2 is a block diagram showing an example of the functional configuration of a viewer terminal according to the first embodiment. 6 is a flowchart showing a process flow in the server system. 4 is a flowchart showing a process for generating an image in an audio-visual space. FIG. 11 is a block diagram showing an example of the functional configuration of a viewer terminal according to the second embodiment. 13A and 13B are diagrams showing an example of setting a visual field securing area 60 in Modification 2. 13A and 13B are diagrams showing other examples of setting the visual field securing area 60 in the second modified example. 13A to 13C are diagrams for explaining adjustment of margins in Modification 2. 13A to 13C are diagrams for explaining a condition satisfying object display control process in Modification 3. FIG. 4 is a diagram showing an example of setting specification data.

Below, a preferred embodiment of the present invention will be described with reference to the drawings. Note that the present invention is not limited to the embodiments described below, and the forms to which the present invention can be applied are not limited to the following embodiments. In addition, in the description of the drawings, the same parts are given the same reference numerals.

First Embodiment
Fig. 1 is a diagram showing an example of the configuration of a content viewing system 1000 according to the present embodiment. As shown in Fig. 1, the content viewing system 1000 includes a server system 1100, which is a computer system, and a viewer terminal 1500 owned by a user (hereinafter also referred to as a "viewer") 2 who is a viewer of the content, and these are connected to each other via a network N so as to be able to communicate data with each other.

Network N refers to a communication path that allows data communication. In other words, network N includes a LAN (Local Area Network) such as a dedicated line (dedicated cable) for direct connection or Ethernet (registered trademark), as well as a telephone communication network, a cable network, the Internet, and other communication networks, and the communication method can be either wired or wireless.

The server system 1100 has, for example, a keyboard 1106, a touch panel 1108, and storage 1140, and the main body device 1101 is equipped with a control board 1150.

The control board 1150 is equipped with various microprocessors such as a CPU (Central Processing Unit) 1151, a GPU (Graphics Processing Unit), and a DSP (Digital Signal Processor), various IC memories 1152 such as a VRAM, RAM, and ROM, and a communication device 1153. Note that a part or the whole of the control board 1150 may be realized by an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a SoC (System on a Chip).

The server system 1100 realizes, by the control board 1150 performing calculations based on a predetermined program and data, (1) a user management function for registering users (viewers) 2 and managing information on registered users 2, (2) a distribution service function for allowing an unspecified number of people to enjoy given content, such as multimedia content, simultaneously, (3) a viewer action function for viewers 2 of the content to post "tips" as a given viewer action, and (4) a settlement function for settling the amount of tips associated with the posted tips.

In this embodiment, an example of a live distribution service function that provides content to a viewer terminal 1500 according to a distribution schedule is shown. To realize this distribution service function, the server system 1100 manages a content publication site. However, the management of the publication site may be configured to use an external video publication management server 1200. The video publication management server 1200 is an external server for operating and managing an existing video publication site (a website that accepts video submissions from an unspecified number of contributors via the Internet, etc., and provides the submitted videos to an unspecified number of viewers in a streaming format). In this case, the server system 1100 communicates with the video publication management server 1200 to automatically post the content, thereby realizing its publication.

As a viewer action function, an example of accepting tips during live streaming of content is shown. Viewer 2 can post tips in exchange for consuming virtual currency equivalent points (hereinafter simply referred to as "currency points"), which are parameter values that the viewer possesses. In this embodiment, viewer 2 posts tips using a tip object 4, as exemplified in FIG. 5 etc. Various tip objects 4 with different tip amounts, which are the consumption value of currency points, are available, and by selecting the one to use for posting and performing the action operation described below, a tip of that tip amount can be posted.

The payment function is a function that allows viewer 2 to purchase currency points, and in response to a request from viewer terminal 1500, it cooperates with an external electronic payment server 1300 operated by an electronic payment service provider or the like to carry out the currency point purchase procedure (billing process). During the billing process, electronic payment server 1300 responds to an inquiry from server system 1100 and performs a process of settling the purchase amount of currency points with viewer 2's credit card, prepaid card, or the like. Then, server system 1100 grants viewer 2 currency points equivalent to the purchase amount settled by electronic payment server 1300.

In FIG. 1, server system 1100 is depicted as a single server device, but multiple server devices connected online may share one or more of the functions described above. Server system 1100 may also be configured to include multiple blade servers sharing each function, connected to each other via an internal bus for data communication. Multiple independent server devices installed in separate locations may be configured to communicate data via network N, functioning as server system 1100 as a whole.

In addition, in FIG. 1, two viewer terminals 1500 for viewer 2 are shown, but in actual operation, there may be three or more viewers 2, each of whom accesses the server system 1100 using a viewer terminal 1500.

The viewer terminal 1500 is a computer system used individually to view content, and is an electronic device that can access the server system 1100 via the network N. The viewer terminal 1500 in this embodiment is a device known as a smartphone, but may also be a portable game device, a tablet computer, a personal computer, etc.

2 is a front view showing an example of the configuration of a viewer terminal 1500 in this embodiment. As shown in FIG. 2, the viewer terminal 1500 includes a directional input key 1502, a home key 1504, a touch panel 1506 that functions as an image display device and a contact position input device, an internal battery 1509, a speaker 1510, a microphone 1512, a camera 1520, a control board 1550, and a memory card reader 1542 that can read and write data to a memory card 1540 that is a computer-readable storage medium. In addition, a power button, a volume control button, etc. that are not shown in the figure are also provided. An IC card reader or the like that can read and write data contactlessly to an IC card-type credit card or prepaid card that can be used to pay tip amounts, etc. may also be provided.

The control board 1550 is equipped with various microprocessors such as a CPU 1551, a GPU, and a DSP, various IC memories 1552 such as a VRAM, a RAM, and a ROM, a wireless communication module 1553 for wireless communication with a mobile phone base station or a wireless communication base station connected to the network N, and an interface circuit 1557. The interface circuit 1557 includes a circuit for receiving signals from the direction input key 1502 and the home key 1504, a driver circuit for the touch panel 1506, an output amplifier circuit for outputting an audio signal to the speaker 1510, an audio signal generation circuit for generating an audio signal collected by the microphone 1512, a circuit for inputting image data of an image captured by the camera 1520, and a signal input/output circuit for the memory card reader 1542. Each of these elements mounted on the control board 1550 is electrically connected via a bus circuit or the like, and is connected so that data can be read and written and signals can be transmitted and received. Note that a part or all of the control board 1550 may be configured with an ASIC, an FPGA, or a SoC.

In the control board 1550, the IC memory 1552 stores a viewing program for enabling viewing of content and posting of tips, as well as various setting data required to execute this viewing program. The viewing program, etc. are downloaded from the server system 1100 at an appropriate time, such as when a user registers. The viewing program, etc. may also be read from a storage medium such as a memory card 1540 obtained separately. The CPU 1551 etc. then executes the viewing program to perform calculation processing, and controls each part of the viewer terminal 1500 in response to operational inputs to the touch panel 1506, directional input keys 1502, and home key 1504, thereby enabling viewing of content on the viewer terminal 1500.

[About providing content and posting tips]
In this embodiment, the content is a video of a live event (concert) held by a performer character such as a character in a virtual viewing space, and is live-streamed to the viewer terminal 1500. The performer character may be, for example, a character operated by a distributor (also called a video poster) who wishes to stream the content, or an avatar character of the distributor. The avatar character may be operated by a method in which the distributor moves in a video of the distributor (the so-called Vtuber method).

Figure 3 is a diagram showing an example of a viewing space 10, which is a virtual three-dimensional space constructed for video distribution. For example, as shown in Figure 3, the viewing space 10 includes a stage 11 and a viewing area 13 in front of it. During the concert, performer characters 3 perform live performances on the stage 11, such as singing songs, dancing, and playing musical instruments. The server system 1100 provides (distributes) content by live-distributing the concert video in this viewing space 10 to a viewer terminal 1500.

The viewing area 13 is a viewing position where viewers who have registered to watch the live broadcast become spectators 6 and watch the concert, and spectator-view cameras 8 are placed in one-to-one correspondence with the viewers. For example, the viewing area 13 is configured with multiple spectator seats in a predetermined layout, with one viewer assigned to each seat, and a spectator-view camera 8 for that viewer placed at each seat. Then, by shooting footage of the viewing space 10 with each spectator-view camera 8, a concert video is generated for each viewer. In other words, during the live broadcast, footage of the viewing space 10 as seen from each spectator-view camera 8 is provided to each viewer terminal 1500 as a concert video for the corresponding viewer.

In addition, in this embodiment, when a viewer posts a tip during a live broadcast, the tip object 4 (e.g., tip objects 4a and 4b) used to post the tip appears in the viewing space 10. Specifically, the tip object 4 appears, for example, from the audience seat of the viewer who posted the tip, and moves in the viewing space 10 along a given trajectory (e.g., a parabola) as if it was thrown from the audience seat, as shown by the arrow in FIG. 3. Eventually, it lands on the floor of the viewing area 13 or the stage 11, and is placed at the landing position. To achieve this, the server system 1100 performs a movement control process for the tip object 4 for each posted tip. Specifically, the tip object 4 is stopped when it lands at the landing position, and the tip object 4 is moved until the stopping condition is satisfied.

Here, the trajectory of the tipping object 4 is preset as a trajectory function determined according to a given action operation. For example, the insertion direction and flight distance (movement distance) of the tipping object 4 are determined according to the viewer's action operation. In this embodiment, as described later with reference to FIG. 6, the insertion direction is determined by the direction of the flick operation accepted as the action operation, and the flight distance is determined by the flick speed or the length of the flick. Note that multiple tipping trajectory functions may be prepared, and one may be determined by accepting a selection operation by the viewer.

[About the viewing screen]
Fig. 4 is a diagram showing an example of a viewing screen of content (concert video) on the viewer terminal 1500. The viewing screen is displayed when a viewer accesses the public server on the viewer terminal 1500 and completes viewer registration for the live streaming to be viewed. As shown in Fig. 4, the viewing screen includes a content display unit 21 on which the concert video of the live streaming is displayed, a camera operation unit 30 for operating the direction and zoom of the spectator viewpoint camera 8 assigned to the viewer, a tipping icon 23 for submitting a tip, and a purchase icon 25 for purchasing currency points.

The camera operation unit 30 includes a direction operation unit 31 for changing the shooting direction, such as panning and tilting, and a view angle operation unit 33 for changing the shooting angle of view. Therefore, while watching the concert video displayed on the content display unit 21, the viewer can change the direction and zoom of the spectator viewpoint camera 8 as appropriate using the camera operation unit 30.

In this embodiment, the position of the spectator-view camera 8 is fixed to the spectator seats and cannot be moved; however, for example, in a case where the viewing area 13 is set up as standing room only, the position (shooting position) of the spectator-view camera 8 may be changed by the viewer, assuming movement within the viewing area 13. In that case, this can be realized by providing the camera operation unit 30 with a position operation unit for changing the shooting position.

When the tip icon 23 is touched on the viewing screen, an action operation for posting can be performed. Specifically, when the tip icon 23 is touched, first, the object selection screen W1 shown in FIG. 5 is popped up. The object selection screen W1 displays a list of multiple tip objects 4 (4a, 4b, 4c, ...) prepared in advance. The tip amount is added to each tip object 4 and presented to the viewer. More specifically, the object selection screen W1 displays only tip objects 4 that the viewer has and can use because the viewer has currency points equal to or greater than the tip amount among the tip objects 4 prepared in advance in a selectable state. When a tip object 4 with an insufficient balance is purchased by touching the purchase icon 25 on the viewing screen in FIG. 4, it becomes selectable. In this way, when sending a tip, the viewer who sends the tip is requested to settle the tip amount through the electronic settlement server 1300.

The object selection screen W1 may display all tip objects 4 in a selectable state, while when the decision button B1 is touched, the balance may be checked, and if insufficient, the user may be prompted to purchase currency points. Furthermore, the configuration is not limited to purchasing currency points for the tip amount before posting a tip, but may allow tip objects 4 to be obtained (purchased) in advance in exchange for consuming currency points, and tip objects 4 in hand may be used for posting. In that case, for example, when the purchase icon 25 is touched, a sales screen showing a list of tip objects 4 for sale is displayed, and these are sold for their respective tip amounts.

When a viewer wants to post a tip, they select the tip object 4 to be used for the post on the object selection screen W1 and then touch the confirm button B1. Then, in exchange for consuming currency points equivalent to the tip amount, the viewer can perform the action operation to execute the post.

FIG. 6 is a diagram showing an example of an action operation for posting. For example, when the tip amount is settled, the object selection screen W1 is closed. Then, as shown in FIG. 6, a tip object 4a used to post a tip (the type of tip object selected on the object selection screen W1 immediately before) is displayed in the center of the lower part of the content display unit 21. The action operation is a flick operation (shaded arrow in FIG. 6) performed with the tip object 4a as the starting point. That is, the direction of the flick operation is used to input the insertion direction of the tip object 4a, and the speed of the flick or the length of the flick is used to input the flight distance (travel distance). When the action operation is then performed, the tip object 4a appears in the viewing space 10 and flies to the landing position along a trajectory calculated and set based on the insertion direction and flight distance.

Therefore, during the live broadcast, registered viewers can submit tips at any time they like while watching the concert video displayed on the content display unit 21.

[detail]
As the number of viewers of the live broadcast increases and the atmosphere becomes more lively, the number of tips posted may also increase. Since the tip object 4 appears from the seating position in the viewing area 13 and flies and moves within the viewing space 10, it may cross the visual field of other viewers and interfere with the viewing of the concert video for those other viewers. In this embodiment, in order to prevent the deterioration of the visibility of the concert video caused by viewers posting tips, first, a video of the viewing space 10 is generated excluding the tip object 4. Then, all viewers watching the live broadcast are sequentially set as target viewers, and a visual field securing area setting process and a condition fulfilling object display control process are performed to composite the display of the tip object 4 into the video of the viewing space 10.

1. About the view securing area setting process In the view securing area setting process, the server system 1100 sets a view securing area for a target viewer. FIG. 7 and FIG. 8 are diagrams for explaining the view securing area setting process. FIG. 7 focuses on a viewer shown as a spectator 6a in the viewing space 10 in FIG. 3 as a target viewer, and shows a view range 40 (40a) of the spectator viewpoint camera 8a and a view securing area 50 (50a) set for the target viewer. In a similar manner, FIG. 8 focuses on a target viewer shown as another spectator 6b in FIG. 3, and shows a view range 40 (40a) of the spectator viewpoint camera 8b and a view securing area 50 (50a) set for the target viewer. As shown in each of FIG. 7 and FIG. 8, in the view securing area setting process of this embodiment, a view securing area 50 for the corresponding target viewer (viewers of the spectators 6a and 6b) is set inside the view range 40. Specifically, the visual field securing area 50 is a spatial range narrower than the visual field range 40 and is set so as to be centered with the visual field range 40 .

The specific size of the field of view securing area 50 is determined, for example, using a predetermined coefficient K. FIG. 9 is a diagram showing an example of coefficient K. As shown in FIG. 9, coefficient K is determined within a range equal to or greater than 0 and smaller than an upper limit value T21 (T21<1). Coefficient K is determined, for example, so that it becomes larger (closer to T21) the greater the total number of tips posted up to the set time (total number of tips). This can be achieved by preparing a calculation formula for coefficient K shown in FIG. 9. Also, instead of the total number of tips, the number of tips currently being controlled to move along the trajectory may be used.

Then, the coefficient K determined based on the total number of tips as described above is used as the area ratio of the viewing area 50 on the projection surface 41 (the area ratio of the viewing area 50 to the display frame F of the concert video provided to the target viewer), and a viewing area 50 of the corresponding size is set for the target viewer.

Therefore, in Figures 7 and 8, a field of view securing area 50 is set inside the display frame F as shown by the dashed line, and while it is set small (narrow) while the total number of tips is low, it is possible to control it so that it gradually becomes larger (wider) as the total number of tips increases. Also, in the example of Figure 9, the coefficient K is set to "0" while the total number of tips does not exceed a predetermined threshold T23, so the field of view securing area 50 is not set during that time. This makes it possible to realize control to set the field of view securing area 50 when the total number of tips exceeds the threshold T23.

In addition, since the field of view securing area 50 is set so that its center is aligned with the field of view range 40 of the target viewer, if the target viewer changes the orientation of the spectator-viewpoint camera 8 using the camera operation unit 30, the field of view securing area 50 can be updated and set according to the changed orientation. In this embodiment, the position of the spectator-viewpoint camera 8 is fixed, but even if it is configured to be changeable, it is possible to update and set the field of view securing area 50 according to the changed position.

According to the view securing area setting process here, a view securing area 50 can be set within the view range 40 for each viewer. In addition, at that time, the size of the view securing area 50 can be changed according to the total number of tips posted up to the setting time. It is assumed that the atmosphere is not that lively while the total number of tips is small, and even if tipping objects 4 are in the field of view, they are few in number, so they are unlikely to interfere with viewing. In such a case, the view securing area 50 may not be set, or the view securing area 50 may be set small (narrow). On the other hand, when the atmosphere becomes lively and the total number of tips increases, the number of tipping objects 4 in the field of view may also increase, and the tipping objects 4 may frequently cross the field of view, interfering with viewing. In anticipation of such a case, the view securing area 50 can be set large (wide) as the total number of tips increases.

In addition, FIG. 9 shows an example in which the coefficient K is set to 0 while the total number of tips does not exceed the threshold T23. In contrast, the coefficient K may be determined to be a value equal to or greater than a predetermined value greater than 0 even while the total number of tips does not exceed the threshold T23. In that case, the field of view securing area 50 will always be set (even if no tips have been posted yet and the total number of tips is 0).

In addition, instead of preparing a formula for calculating coefficient K, the correspondence between the total number of tips and coefficient K may be defined in a table format, and coefficient K may be set to increase in stages as the total number of tips increases.

Here, the total number of tips is given as an example of the input status of a viewer action, and a case has been described in which the field of view securing area 50 is variably set based on the total number of tips. However, other items indicating the input status of a viewer action, such as the total amount of tips posted up to the time of setting, the timing of posting of the tips, and the type of tip object 4 used to post the tips, may be used instead of the total number of tips, or the coefficient K may be determined using the total number of tips or a combination of two or more of these items, and the field of view securing area 50 may be set to a size according to each value.

In addition, the field of view securing area 50 may be variably set based on the attributes of the target viewer. Examples of attributes include the total charge amount of the target viewer and the viewing time of the live broadcast (concert video) being viewed. In this case, for example, the upper limit value T21 (T21>1) of the coefficient K is adjusted so that it is larger the higher the total charge amount is and smaller the lower the total charge amount is. Similarly, the upper limit value T21 is increased as the viewing time is longer, and is adjusted so that it is smaller as the viewing time is shorter. In this way, the size of the field of view securing area 50 can be set taking into account the total charge amount and viewing time of the target viewer.

In addition, in the case of a configuration in which videos related to live streaming are obtained from video posters and provided, the relationship between the target viewer and the video poster (for example, whether or not they are friends with each other) can be included in the attributes. In this case, too, the upper limit value T21 of the coefficient K is adjusted to be larger when watching a friend's video compared to when watching a video by a video poster who is not a friend.

The field of view securing area 50 may also be set variably based on the number of viewers of the live broadcast. This is because as the number of viewers increases, the number of tips posted may also increase. For example, by determining the coefficient K according to the number of viewers, it is possible to set the field of view securing area 50 small (or not set) when the number of viewers is small, and to set the field of view securing area 50 larger as the number of viewers increases. Furthermore, by determining the coefficient K using the number of viewers in combination with the input status of the viewer actions described above and the attributes of the target viewers, the field of view securing area can also be set to a size according to each value.

In addition, the field of view securing area 50 may be set variably depending on the scene of the concert video at the time of setting. For example, in a lively scene, the field of view securing area 50 may be made small (or not set at all), and in a quiet scene, a wide area within the field of view 40 may be set as the field of view securing area 50. This makes it possible to change the display form of the tip object 4 in the viewing space 10 to one that is generally difficult to see by the subsequent condition fulfilling object display control process in a quiet scene, while not making such a change in a lively scene and prioritizing the display of a performance that shows that the atmosphere is lively.

2. Condition-Satisfying Object Display Control Processing In the condition-satisfying object display control processing, the server system 1100 performs different display control on the tipping object 4 that satisfies a predetermined in-area condition as a condition-satisfying object and the tipping object 4 that does not satisfy the in-area condition as a condition-non-satisfying object, thereby synthesizing the display of the tipping object 4 with the image of the viewing space 10 (more specifically, the image of the viewing space 10 excluding the tipping object 4). The in-area condition is set in advance as a condition indicating that the tipping object 4 passes through or is located within the visual field securing area 50 of the target viewer. By this processing, the image of the concert video to be displayed on the content display unit 21 on the viewing screen can be generated for each viewer.

In this embodiment, the server system 1100 first classifies the tip objects 4 in the viewing space 10 by classifying the tip objects 4 that are within the visual field securing area 50 of the target viewer as condition-satisfying objects because they satisfy the in-area condition, and classifying the remaining tip objects 4 as condition-unsatisfying objects.

The classification procedure first excludes from detection all tip objects 4 that relate to the target viewer, i.e., tip objects 4 posted by the target viewer. Then, condition-satisfying objects that satisfy the in-area conditions are detected from tip objects 4 posted by viewers other than the target viewer. Next, if any of the detected condition-satisfying objects has already completed its movement control process and is placed at its landing position, it is excluded from the condition-satisfying objects. In other words, of the detected condition-satisfying objects, those that are being controlled in movement by the movement control process are determined to be condition-satisfying objects.

After classifying the tip objects 4 in the viewing space 10 as described above, the server system 1100 controls the display of the tip objects 4 classified as condition non-satisfying objects based on the tip object definition data 510 (see FIG. 12). On the other hand, for the tip objects 4 classified as condition satisfying objects, the server system 1100 controls the display of the tip objects 4 in a display form different from the display form in the case where the tip objects 4 are the above-mentioned condition non-satisfying objects (i.e., the display form based on the tip object definition data 510).

10 and 11 are diagrams for explaining the control of changing the display form here. FIG. 10 shows a display frame F when the total number of tips is small and the visual field securing area 50 is small, and FIG. 11 shows a display frame F when the total number of tips is increased and the visual field securing area 50 is set large. In this embodiment, the display form of the tipping object 4 classified as a condition-satisfying object (i.e., moving within the viewing space 10) is changed to a display form with high transparency and small size. For example, the size is halved and the transparency is set to 50% to change the display form. Therefore, as shown in each of FIGS. 10 and 11, when the tipping object 4 moving within the viewing space 10 enters the visual field securing area 50, the tipping object 4 is reduced to half and changed to a semi-transparent display. However, the display form of the tipping object 4A of the tip posted by the target viewer is not changed even if it enters the visual field securing area 50, and is displayed in the same display form before entering the visual field securing area 50. Furthermore, the display form of the tip object 4B, for which the movement control process has already ended and which has already landed at the landing position, will not be changed.

According to the condition-satisfying object display control process here, the tip objects 4 in the viewing space 10 can be classified into condition-satisfying objects and condition-non-satisfying objects, and can be displayed in different display forms depending on which category they are classified into. More specifically, among the tip objects 4 in the field of view securing area 50, the size of the tip objects 4 that are under movement control and that relate to tips posted by viewers other than the target viewer can be reduced or their transparency can be increased. This makes it possible to display the tip objects 4 in the field of view securing area 50 in a display form that is difficult to see.

On the other hand, the display format of the tip object 4A of the tip posted by the target viewer does not change, so the tip object 4A posted by the user can be displayed in an easy-to-see manner.

Similarly, the display form of a tip object 4B that has already landed at a landing position is not changed even if the landing position is within the field of view securing area 50. This is because the display of a stationary tip object 4B is considered unlikely to obstruct the field of view. Also, by displaying them, it is possible to realize a dramatic display that shows that the atmosphere is getting lively.

Note that the change in the display form described here is merely an example, and is not limited to changing both the transparency and the size. For example, only the size may be changed to a smaller size, or only the transparency may be changed to a higher value (higher transparency). In addition, the transparency may be set to 100%, for example, to hide the corresponding tip object 4 (condition-satisfying object).

[Functional configuration]
1. Server System Fig. 12 is a block diagram showing an example of a functional configuration of a server system 1100. As shown in Fig. 12, the server system 1100 of this embodiment includes an operation input unit 100s, a server processing unit 200s, an image display unit 390s, a sound output unit 392s, a communication unit 394s, and a server storage unit 500s.

The operation input unit 100s is used to input various operations for system management, maintenance, etc., and can be realized by, for example, a keyboard, a mouse, a touch panel, etc. In FIG. 1, this corresponds to the keyboard 1106 and the touch panel 1108.

The server processing unit 200s can be realized by a processor, which is an arithmetic circuit such as a CPU, GPU, ASIC, FPGA, etc., or an electronic component such as an IC memory, and controls the input and output of data between each part of the device including the operation input unit 100s and the server storage unit 500s. It then performs various arithmetic processing based on predetermined programs and data, operation input signals from the operation input unit 100s, data received from the viewer terminal 1500, etc., and generally controls the operation of the server system 1100. In Figure 1, this corresponds to the control board 1150 and its CPU 1151.

The server processing unit 200s includes a user management unit 210, a billing processing unit 220, a video provision control unit 230, a timing unit 280s, an image generation unit 290s, a sound generation unit 292s, and a communication control unit 294s.

The user management unit 210 performs processes related to user registration and manages the data of each registered user linked to a user account. For example, it can execute processes such as assigning unique user accounts to registered users, registration information management processes that register and manage personal information for each user account, and usage history management processes that manage login and logout history, etc. Of course, it can also include management processes for other data linked to user accounts as appropriate.

The billing processing unit 220 performs billing processing in response to a user's purchase of currency points, and grants the user currency points equivalent to the purchase amount. It also performs monthly billing processing for paid members of the distribution service.

The video provision control unit 230 performs various processes to realize live streaming of concert videos in the viewing space. In this embodiment, a video of the viewing space is generated for each viewer, and control is performed to provide the video by live streaming to the viewer terminal 1500 of the corresponding viewer. In addition, the video provision control unit 230 also manages the live streaming schedule and viewers. This video provision control unit 230 includes a live progress control unit 231, a viewer-specific video generation unit 233, an object movement control unit 235, a field of view securing area setting unit 237, a condition satisfying object display control unit 239, and a viewpoint control unit 245.

The live progress control unit 231 controls the progress of the live stage in the viewing space by performing processes such as setting up a stage and viewing area to construct a viewing space, placing performer characters in the viewing space and controlling their movements, and placing and controlling each viewer's spectator viewpoint camera in the viewing space.

The viewer-specific video generation unit 233 generates, for each viewer of the live broadcast, a video of the viewing space as seen from the spectator viewpoint camera, excluding the tip object, for each frame.

The object movement control unit 235 is a functional unit that performs movement control processing, and detects the action operation of posting a tip made on the viewer terminal 1500, and controls the appearance and movement of a tip object in the viewing space. Specifically, it sets a landing position according to the throwing direction and flight distance input by the action operation, and sets a movement path for the tip object that moves along a given trajectory from the viewer's seat position to the landing position. Then, it controls the movement of the tip object according to the set movement path.

The field of view securing area setting unit 237 is a functional unit that performs field of view securing area setting processing for each viewer of the live broadcast who is sequentially designated as a target viewer, and sets a field of view securing area for that target viewer.

The condition satisfying object display control unit 239 is a functional unit that performs condition satisfying object display control processing for each viewer of the live broadcast who is sequentially designated as a target viewer, and by performing different display control for condition satisfying objects and condition non-satisfying objects, it synthesizes the display of the tip object into the video of the viewing space generated by the viewer-specific video generation unit 233. This condition satisfying object display control unit 239 has a condition satisfying object detection unit 241 and a form change control unit 243.

The condition-satisfying object detection unit 241 detects condition-satisfying objects that satisfy the in-area conditions from among the tip objects in the viewing space, and classifies the remaining objects as condition-unsatisfying objects.

The form change control unit 243 performs control to make the display form of the condition satisfying object different from the display form of the condition non-satisfying object. In this embodiment, the display form is changed by increasing the transparency of the tipping object detected as a condition satisfying object by the condition satisfying object detection unit 241 and reducing the size of the tipping object.

The viewpoint control unit 245 detects operations to change the shooting direction or shooting angle of view on the viewer terminal 1500, and controls the direction and zoom of the spectator viewpoint camera for that viewer.

The timing unit 280s uses the system clock to measure the current date and time, time limit, etc.

The image generation unit 290s generates images related to system management of the server system 1100, and outputs them to the image display unit 390s.

The sound generation unit 292s is realized by executing an IC or software that generates or decodes audio data, and generates or decodes audio data such as operation sounds and background music related to the system management and video distribution of the server system 1100. Audio signals related to system management are output to the sound output unit 392s.

The communication control unit 294s establishes a communication connection and processes data for data communication with an external device (e.g., the viewer terminal 1500) via the communication unit 394s, and realizes data exchange with the external device.

The image display unit 390s displays various screens for system management, etc., based on the image signal input from the image generation unit 290s. For example, this can be realized by an image display device such as a flat panel display, a projector, or a head-mounted display. In FIG. 1, this corresponds to the touch panel 1108.

The sound output unit 392s emits the audio signal input from the sound generation unit 292s. In FIG. 1, this corresponds to a speaker (not shown) provided in the main device 1101 or the touch panel 1108.

The communication unit 394s connects to the network N to realize communication. For example, this can be realized by a wireless communication device, a modem, a TA (terminal adapter), a jack for a wired communication cable, a control circuit, etc. In FIG. 1, this corresponds to the communication device 1153.

The server storage unit 500s stores in advance or temporarily stores each time processing is performed programs for operating the server system 1100 and implementing the various functions of the server system 1100, as well as data used during execution of these programs. For example, this can be implemented by IC memory such as RAM or ROM, a magnetic disk such as a hard disk, or an optical disk such as a CD-ROM or DVD. In FIG. 1, this corresponds to the IC memory 1152 and storage 1140.

The server storage unit 500s also stores a server program 501, a distribution viewing program 503, tipping object definition data 510, user management data 520, and video provision management data 530. In addition, other necessary data such as timers, counters, various tables, thresholds, and flags are also stored as appropriate.

The server program 501 is a program for causing the server processing unit 200s to function as the user management unit 210, the billing processing unit 220, and the video provision control unit 230. In addition, this may also include programs for causing the server processing unit 200s to function as the image generation unit 290s, the sound generation unit 292s, and the communication control unit 294s, as appropriate.

The viewing program for distribution 503 is the original of the viewing program 502 (see Figure 15) that is downloaded to the viewer terminal 1500.

The tipping object definition data 510 is data that defines a tipping object. In this embodiment, it is prepared for each tipping object that can be selected on the object selection screen W1 in FIG. 5, and stores the tipping amount and model data for displaying the tipping amount in association with the object ID of the corresponding tipping object.

User management data 520 is prepared for each user who has completed user registration, and stores various management data related to the user. Specifically, as shown in FIG. 13, one user management data 520 includes the user account (user ID) 521 of the corresponding user, payment medium ledger data 523, and viewing history 525 of content (concert videos) viewed by the user as a viewer.

The payment medium ledger data 523 stores information on the balance of electronic payment media (currency points in this embodiment) associated with the user, such as the purchase date and time of currency points and the history of the number of purchases (charged amount), and the consumption date and time and the number of currency points consumed. In addition, if the user is a paid member of a distribution service, the monthly charge amount, etc. are also stored.

Video provision management data 530 is generated for each live broadcast, and stores various data describing the latest state of the live broadcast. For example, as shown in FIG. 14, one video provision management data 530 includes a broadcast ID 531, a broadcast schedule 533, a live broadcast flag 535 indicating whether or not a live broadcast is in progress (ON: live broadcast in progress/OFF: provided), a viewer account list 537, tipping submission data 540, viewing space management data 550, and viewer-specific video data 570.

The tip posting data 540 is generated each time an action operation to post a tip is detected on the viewer terminal 1500. Each tip posting data 540 includes a posting ID 541 for identifying the tip, a posting viewer account 543 which is the user account of the viewer who posted the tip, an object ID 545 of the tip object used to post the tip, the tip amount 547, and a posting date and time 549.

The viewing space management data 550 includes performer character control data 551, audience viewpoint camera control data 553, tip object movement control data 555, and viewer-specific tip object display control data 560.

The performer character control data 551 stores data representing the state of the performer character 3 (see FIG. 3, etc.) in the viewing space, such as position coordinates, posture, movement speed, and motion control data.

Spectator viewpoint camera control data 553 is prepared for each viewer of the live broadcast (a viewer registered in viewer account list 537), and stores data such as the position, direction (attitude), and angle of view of the spectator viewpoint camera 8 (see Figure 3), which is the viewpoint for the corresponding viewer when rendering the concert video footage.

The tip object movement control data 555 stores data related to the movement control of the tip object used to post a tip on the viewer terminal 1500. For example, it includes the appearance position, landing position, movement path, position coordinate history, etc.

The viewer-specific tip object display control data 560 is prepared for each viewer of the live broadcast, and stores data for display control of tip objects applied to the corresponding viewer. For example, it stores field of view assurance area data 563 and an object classification table 565 in association with a viewer account 561, which is the user account of the corresponding viewer.

The field of view securing area data 563 sets the range of the field of view securing area for the viewer who has been set as the target viewer by the field of view securing area setting unit 237.

The object classification table 565 is a data table that sets the latest classification for each tip object in the viewing space as either a condition-satisfying object or a condition-non-satisfying object.

The viewer-specific video data 570 stores video data of a concert video generated for each viewer and provided to the corresponding viewer. Specifically, it stores video data of the viewing space excluding the tip object generated for each frame by the viewer-specific video generation unit 233, and in which the display of the tip object is synthesized by the condition fulfillment object display control unit 239.

15 is a block diagram showing an example of the functional configuration of a viewer terminal 1500. As shown in Fig. 15, the viewer terminal 1500 includes an operation input unit 100, a terminal processing unit 200, an image display unit 390, a sound output unit 392, a communication unit 394, and a terminal storage unit 500.

The operation input unit 100 allows the user to input various operations, and can be realized by, for example, a button switch, a joystick, a touch pad, a trackball, an acceleration sensor, an angular velocity sensor, a CCD module, etc. In FIG. 2, this corresponds to the direction input key 1502, the home key 1504, and the touch panel 1506.

The device processing unit 200 can be realized by a processor, which is an arithmetic circuit such as a CPU, GPU, ASIC, FPGA, or other electronic components such as an IC memory, and controls data input/output between each unit of the device including the operation input unit 100 and the device storage unit 500. It performs various arithmetic processing based on predetermined programs and data, operation input signals from the operation input unit 100, data received from the server system 1100, and the like, and generally controls the operation of the viewer terminal 1500. In FIG. 2, this corresponds to the control board 1550 and its CPU 1551. In this embodiment, the device processing unit 200 includes a viewer terminal arithmetic unit 270, a timer unit 280, an image generation unit 290, a sound generation unit 292, and a communication control unit 294.

The viewer terminal calculation unit 270 executes various calculation processes to cause the viewer terminal 1500 to function as a terminal for accessing the server system 1100 and viewing the content (concert video). For example, the viewer terminal calculation unit 270 includes an operation signal transmission control unit 271 and an image display control unit 273.

The operation signal transmission control unit 271 performs processing to transmit various data and request information to the server system 1100 in response to operation input to the operation input unit 100.

The image display control unit 273 performs control for displaying the viewing screen, etc., based on various data received from the server system 1100. In this embodiment, the image display control unit 273 performs control for accessing a public site to view and browse the concert video, that is, control for realizing the function of a so-called web browser. It is also possible to configure the image of various screens, such as the viewing screen, to be generated by the viewer terminal 1500, rather than by the server system 1100. In that case, the image display control unit 273 performs control for objects placed in a virtual three-dimensional space (viewing space) for generating 3DCG, for example, and the image generation unit 290 renders the 3DCG and executes various controls for generating the concert video in the viewing space.

The image generation unit 290 cooperates with the image display control unit 273 to generate various image data, and controls the generation and output of image signals for displaying those images on the image display unit 390 .
In FIG. 2, the GPU mounted on the control board 1550 corresponds to this.

The sound generation unit 292 is realized, for example, by a digital signal processor (DSP), a processor such as a voice synthesis IC, an audio codec for playing audio files, etc., and generates audio signals such as sound effects, operation sounds, background music, etc., and outputs them to the sound output unit 392.

The communication control unit 294 establishes a communication connection and processes data for data communication with an external device (e.g., the server system 1100) via the communication unit 394, and realizes data exchange with the external device.

The image display unit 390 displays various screens, such as a viewing screen, based on the image signal input from the image generation unit 290. For example, this can be realized by an image display device such as a flat panel display, a projector, or a head-mounted display. In FIG. 2, this corresponds to the touch panel 1506.

The sound output unit 392 is realized by a device that outputs (emits) sound based on the audio signal input from the sound generation unit 292. In FIG. 2, this corresponds to the speaker 1510.

The communication unit 394 connects to the network N to realize communication. For example, this can be realized by a wireless communication device, a modem, a TA, a jack for a wired communication cable, a control circuit, etc. In FIG. 2, this corresponds to the wireless communication module 1553.

In the terminal storage unit 500, programs for operating the viewer terminal 1500 and realizing the functions of the viewer terminal 1500, and data used during execution of these programs are stored in advance, or are temporarily stored each time processing is performed. For example, this can be realized by IC memory such as RAM or ROM, a magnetic disk such as a hard disk, or an optical disk such as a CD-ROM or DVD. In FIG. 2, this corresponds to the IC memory 1552 and the memory card 1540. A configuration using online storage is also possible.

The terminal storage unit 500 also stores a viewing program 502 for causing the terminal processing unit 200 to function as the viewer terminal calculation unit 270. The viewing program 502 may be a dedicated client program, or may be configured with a web browser program and a plug-in that realizes interactive image display. In this embodiment, it is a copy of the distribution viewing program 503 (see FIG. 12) provided by the server system 1100.

[Process flow]
Fig. 16 is a flowchart showing the process flow related to the live distribution of a concert video performed by the server system 1100, focusing on one live distribution. Fig. 17 is a flowchart showing the process flow related to the generation of an image of the viewing space started in step S5 of Fig. 16. The process described here is realized by the server processing unit 200s reading and executing the server program 501. Prior to this process, a distribution schedule 533 is set. Then, provision of the image of the viewing space to the viewer terminal 1500 is started according to the set distribution schedule 533.

That is, as shown in FIG. 16, the live progress control unit 231 first sets up a viewing space (virtual three-dimensional space) and places spectator viewpoint cameras for each viewer (step S1), and starts controlling the spectator viewpoint cameras and the movements of the performer characters (step S3). Then, it starts generating video of the viewing space and live streaming to the viewer terminal 1500 (step S5).

During the live broadcast, the action operation of posting a tip at each viewer terminal 1500 is monitored. If an action operation at any viewer terminal 1500 is detected (step S7: YES), the object movement control unit 235 places the tip object used to post the tip in the viewing space, makes it appear, and starts a movement control process to start controlling the movement of the tip object (step S9).

After that, the process returns to step S7 and repeats the above process until the live streaming ends (step S11: NO). Then, when the live streaming ends (step S11: YES), the process ends.

Next, in the process of generating an image of the viewing space that starts in step S5, the process of loop A is repeatedly executed for each frame (steps S501 to S517) as shown in FIG. 17. Then, in loop A, the process of loop B is executed first, sequentially treating all viewers as target viewers (steps S503 to S515).

In other words, in loop B, first, the viewer-specific image generating unit 233 generates an image of the viewing space excluding the tip object as seen from the spectator viewpoint camera of the target viewer (step S505). Next, the field of view securing area setting unit 237 performs a field of view securing area setting process to set a field of view securing area for the target viewer within the viewing space (step S507).

Then, the condition satisfying object display control unit 239 performs a condition satisfying object display control process. That is, first, the condition satisfying object detection unit 241 classifies all tipping objects in the viewing space into condition satisfying objects and condition non-satisfying objects by the classification procedure described above (step S509). Next, the form change control unit 243 combines the display of the tipping objects with the video of the viewing space generated in step S505 according to the classification in step S509. That is, the form change control unit 243 performs display control based on the tipping object definition data 510 for tipping objects classified as condition non-satisfying objects (step S511). In addition, the form change control unit 243 performs display control for tipping objects classified as condition satisfying objects after changing the size and transparency based on the display form based on the tipping object definition data 510 (step S513).

Then, once loop B has been processed for all viewers, loop A processing for one frame ends.

As described above, according to this embodiment, a viewing area can be set for each viewer, and the display form of the tip object can be made different inside and outside the viewing area. Specifically, the viewing area can be set to a size according to the total number of tips at the time of setting, with the center aligned with the viewing range of the spectator viewpoint camera for the corresponding viewer. The display form of the tip object in the viewing area can be changed so that it is difficult to see, for example by reducing the size or increasing the transparency. Therefore, it is possible to ensure the visibility of the concert video display (the image of the viewing space displayed on the content display unit 21) on the viewing screen for each viewer.

Second Embodiment
Next, a second embodiment to which the present invention is applied will be described. Hereinafter, differences from the first embodiment will be mainly described, and the same components as those in the first embodiment will be given the same reference numerals as those in the first embodiment, and duplicated descriptions will be omitted.

In the first embodiment, the video provision control unit 230 (see FIG. 12) is realized by the server system 1100, but in the second embodiment, the video provision control unit 230 is realized by a viewer terminal. In the second embodiment, the server system 1100 performs user registration and management of the user information, billing processing, and management of live streaming schedules and viewers. In addition, when the server system 1100 detects an action operation to post a tip on a viewer terminal, it generates tip posting data 540 (see FIG. 14) and transmits it to each viewer terminal 1500.

Fig. 18 is a block diagram showing an example of the functional configuration of the viewer terminal 1500A in this embodiment. As shown in Fig. 18, the viewer terminal 1500A includes an operation input unit 100, a device processing unit 200, an image display unit 390, a sound output unit 392, a communication unit 394, and a device storage unit 500.

The device processing unit 200 has a video provision control unit 230, and the image display control unit 273 of the viewer terminal calculation unit 270 is omitted. That is, the viewer terminal 1500A of this embodiment does not acquire video data of the viewing space 10 from the server system 1100, but rather the viewer terminal 1500A's own video provision control unit 230 generates the video of the viewing space.

The terminal storage unit 500 also stores a video provision program 504 for causing the terminal processing unit 200 to function as the viewer terminal calculation unit 270 and the video provision control unit 230, user management data 520 relating to the user of the viewer terminal 1500A, video provision management data 530 relating to the live broadcast that the user is viewing as a viewer, and tip object definition data 510. However, in the video provision management data 530, the spectator viewpoint camera control data 553 and viewer-specific tip object display control data 560 of the viewing space management data 550 are set only if they relate to the viewer of the viewer terminal 1500.

The processing flow of the viewer terminal 1500A in this embodiment is basically the same as the flowcharts shown in Figures 16 and 17, and each step can be interpreted as being executed by the video provision control unit 230 of the viewer terminal 1500A. However, in the processing related to video generation in Figure 17, loop B processing is performed for the viewer of that viewer terminal 1500. Also, in step S7 in Figure 16, an action operation in each viewer terminal 1500 is detected upon receiving tip posting data 540 from the server system 1100.

According to the second embodiment, it is possible to achieve the same effect as the first embodiment. In addition, since the viewer terminal 1500 generates an image of the viewing space as seen from the viewer's spectator viewpoint camera, the processing that is performed by the server system 1100 in the first embodiment can be shared among the viewer terminals 1500, thereby making it possible to reduce the processing load on the server system 1100.

The forms to which the present invention can be applied are not limited to the above-mentioned embodiments, and components can be added, omitted, or modified as appropriate.

[Modification 1]
For example, the server system 1100 and the viewer terminal 1500 may share roles and functions in an intermediate form between the first and second embodiments. In particular, the server system 1100 and the viewer terminal 1500 may share and execute the processing related to the video provision control unit 230. Specifically, among the functional units of the viewer terminal 1500A shown in FIG. 18, the live progress control unit 231 is left in the server system 1100, and the server system 1100 controls the movements of the performer characters, performs calculations to set the viewing space, and transmits various data of the viewing space to the viewer terminal 1500 (the viewer terminal 1500A in FIG. 18) as needed. The viewer terminal 1500 generates an image of the viewing space based on the various data of the viewing space.

[Modification 2]
7, 8, etc., a view securing area 50 for the viewer is set inside the view range 40 of the spectator viewpoint camera 8, with its center aligned with the view range 40. Alternatively, the view securing area may be set to a size that includes all or a major part of the performer character 3 within it, aligned with a predetermined reference position of the performer character 3. When there are multiple performer characters in the viewing space, a view securing area is set for each of those performer characters.

For example, if the field of view securing area is set to a size that includes the entire performer character 3, the center position of the performer character 3 can be used as the reference position. If the head of the performer character 3 is considered to be a key part and the field of view securing area is set to a size that includes the head of the performer character 3, the center position of the head can be used as the reference position. Furthermore, the size of the field of view securing area (whether a size that includes the entire performer character 3 or a size that includes a key part) and the reference position can be changed depending on the zoom ratio of the spectator viewpoint camera 8.

Figures 19 and 20 are diagrams showing examples of setting the field of view securing area 60 in this modified example, and each shows the field of view securing area 60 set in scenes where the posture of the performer character 3 is different. As shown in each of Figures 19 and 20, in this modified example, the size of the field of view securing area 60 is set with equal margins around the performer character 3, as indicated by the arrows, so that the entire performer character 3 is included inside.

According to this modified example, even if there is a tip object moving to a position overlapping with the performer character 3 as viewed from the spectator viewpoint camera 8 of the target viewer, the subsequent condition-satisfying object display control process can display the tip object in a display form that makes it difficult to see. Therefore, it is possible to ensure the visibility of the performer character 3 by preventing a situation in which the performer character 3 is obscured by a tip object moving in front while watching a live broadcast, and thus getting in the way. Furthermore, according to this embodiment, even if the performer character 3 moves and the distance to the spectator viewpoint camera 8 changes, the field of view securing area 60 can be set to follow the movement of the performer character 3.

[Modification 3]
In addition, in a configuration in which the field of view securing area 60 is set to follow the performer character 3, the width of the margin of the field of view securing area 60, as indicated by the arrow in Fig. 19, may be adjusted. For example, the margin may be adjusted to be wider on the side in the moving direction of the performer character 3. Fig. 21 is a diagram for explaining the adjustment of the margin in this modified example. As shown in Fig. 21, for example, in a scene in which the performer character 3 stands up from a sitting position as indicated by the dashed line in Fig. 21 and jumps, the field of view securing area 60A is set with wide margins in the front and upper directions, which are the moving directions of the performer character 3.

According to this modified example, a tip object approaching the moving direction of the performer character 3 can be displayed in a display form that is difficult to see by the condition satisfying object display control process. Therefore, the visibility of the performer character 3 can be more reliably ensured.

The width of the margin may be fixed at an initial setting, and the initial setting width may be set according to the type of performer character 3. For example, if there is a tendency in the movement pattern of the performer character, such as not moving much or jumping frequently, the initial setting of the margin width is set according to that tendency (according to the type of performer character). For example, if the performer character jumps a lot, the upper margin may be initially set to a width that is longer than the rest. If a range of motion is set for the performer character, the margin width may be initially set according to that range of motion, or may be adjusted as needed.

Also, the setting position of the field of view securing area 60 may be shifted in the direction in which the margin is expanded without changing the size of the field of view securing area 60. In this case, the margin on the opposite side to the direction in which the margin is expanded becomes smaller.

[Modification 4]
Furthermore, in a configuration in which different types of performer characters may appear in the viewing space, data relating to the setting location, reference position, shape, and initial margin settings of the viewing area may be determined in advance for each type of performer character.The viewing area may then be set according to the settings corresponding to the performer characters in the viewing space.When there are multiple performer characters in the viewing space, the viewing area is set for each performer character using settings according to their type.

For example, if a charming part is set for each type of performer character, the set part and the reference position may be set as the set part and its center position as the reference position. This makes it possible to variably set which part of the performer character is the key part described above in Variation 2 depending on the type of performer character.

In addition, regarding the initial margin settings, if the movement pattern tendencies and range of motion are set for each performer character, the width of the margins in each direction (up, down, left, right, front, back) can be set based on these settings.

[Modification 5]
Also, the setting specifications of the viewing field securing area may be determined in advance for each type of performer character. Then, the viewing field securing area may be set according to the setting specifications corresponding to the performer character in the viewing space. When there are multiple performer characters in the viewing space, the viewing field securing area is set for each performer character using the setting specifications according to the type of the performer character. FIG. 23 is a diagram showing an example of setting specification data determined for a certain performer character type. As shown in FIG. 23, the setting specification data is prepared for each character type, for example, and includes the setting of the principle of the setting specification and the setting of an exception when the corresponding condition is satisfied. Therefore, the viewing field securing area is set based on different setting specifications depending on whether or not the corresponding performer character in the viewing space is an exceptional case where the condition is satisfied. If there is a video poster, this setting specification data may be configured so that the video poster can set and edit the conditions and each item of the setting specification.

This modified example makes it possible to set the field of view area according to the settings of the performer character.

[Modification 6]
In the above-described embodiments and modifications, examples have been described in which the display control of the condition satisfying object and the condition non-satisfying object is made different by changing the display form of the two objects. In contrast, the display control of the condition satisfying object and the condition non-satisfying object may be made different by changing the trajectory (movement path) of the condition satisfying object from the initial setting based on the action operation.

Figure 22 is a diagram illustrating the condition fulfillment object display control process in this modified example. In Figure 22, the field of view securing area 60, which is set with the position of the performer character 3 as its center, is shown by a dashed line. In the condition fulfillment object display control process in this modified example, the condition fulfillment object display control unit 239 acts as a trajectory change control means and changes the trajectory of the tip object, which is the condition fulfillment object, in a direction away from the field of view securing area 60.

Specifically, for this purpose, the tipping object under movement control is targeted and a determination is made as to whether or not the moving path, which is the trajectory to the landing position, passes through the field of view securing area 60. The passing tipping object is then detected as a condition-satisfying object. In the example of FIG. 22, the tipping object 4 moving on the moving path L31 shown by the dashed line passes through the field of view securing area 60 before reaching the initially set landing position P31, and is therefore detected as a condition-satisfying object. If a condition-satisfying object is detected in this way, the landing position is moved to correct the moving path so that it does not pass through the field of view securing area 60. In the example of FIG. 22, the landing position P31 of the tipping object 4 is moved to the landing position P33, and the initial moving path L31 shown by the dashed line is corrected to the moving path L33 shown by the solid line.

In addition, the configuration is not limited to moving the landing position to correct the movement path, but for example, the movement path of the tip object 4 may be corrected so that it reflects at the boundary position with the field of view securing area 60 and falls outside the field of view securing area 60.

[Modification 7]
In addition, in the above embodiments, an example has been described in which the field of view securing area is variably set based on the input status of the viewer action, etc. (when the size is changed according to the total number of tips). In contrast, the size of the field of view securing area may be fixed, while the display control of the condition satisfying object may be variably controlled based on the input status of the viewer action, etc. Of course, the size of the field of view securing area may be changed in the same manner as in the above embodiments, and the display control of the condition satisfying object may also be variably controlled.

For example, if the size and transparency of a condition-satisfying object is changed as in the first embodiment, the extent of the change may be determined according to the total number of tips at the time of display control. The larger the total number of tips, the greater the control of the reduction rate and transparency.

[Modification 8]
Furthermore, when displaying content (image of viewing space) on the viewing screen (content display unit 21; see FIG. 4), the range of field of view securing areas 50 (see FIG. 7, etc.), 60 (see FIG. 19, etc.), 60A (see FIG. 21, etc.) may be presented to the viewer so that they can be seen. For example, a dashed line shown as the range of field of view securing area 50 in display frame F in FIG. 7, etc. may be added for presenting field of view securing area 50, and displayed on content display unit 21. In other words, the range of field of view securing area 50 on projection surface 41 may be illustrated on content display unit 21. Of course, the type of line is not limited to solid lines, dashed lines, etc.

The display surrounding the field of view securing area 50 is just one example, and the manner of display is not particularly limited as long as the display allows the inside and outside of the field of view securing area 50 to be distinguished. For example, the display may be performed by placing an object of the field of view securing area 50 within the viewing space. In that case, it is advisable to set the transparency of the object high so as not to reduce the visibility of the inside of the field of view securing area 50.

In addition, in the case of a configuration in which a field of view securing area is displayed, the size, shape, position, etc., of the area may be made changeable by the viewer. For example, this can be realized by accepting changes to each item by the viewer.

It is also possible to configure the present modified example so that the range of the field of view securing area can be switched between being displayed and not displayed. For example, a button for switching the display of the field of view securing area can be placed on the viewing screen, and a switching operation by the viewer can be accepted, allowing each viewer to switch between displaying and not displaying the range.

[Modification 9]
In addition, in the above-mentioned embodiments, an example is shown in which a concert video of a performer character in a viewing space is provided to a viewer terminal, but the content of the video is not particularly limited. Also, a configuration may be adopted in which a video contributor posts videos, and the posted videos are acquired from the video contributor's terminal and provided. For example, the present invention may be applied to live broadcasting of a musical performance or entertainment performed in a viewing space by a performer character played by the video contributor, or to providing a gameplay video in which a game character operated by the video contributor plays against an opponent character in the viewing space.

[Modification 10]
Also, the viewing screen may be configured to display the total number of tips posted up to that point in time for the content being viewed (total number of tips). Not limited to the total number of tips, the total number of tips received within a specified period of time in the most recent past may be displayed. The display may be in a form that displays the total number of posts in a visually recognizable manner, in addition to the value of the total number of posts. For example, this may be realized by displaying a gauge showing the value of the total number of posts, or a balloon that inflates as the value of the total number of posts increases.

For example, in a configuration in which a tipping object moving within the field of view is hidden by the condition fulfillment object display control process, the viewer may have difficulty grasping the status of tipping submissions because the viewer cannot see the corresponding tipping object. According to this modified example, even in such a case, the total number of tips submitted for the content being viewed can be confirmed, making it possible to grasp the actual status of tipping submissions.

[Modification 11]
Also, a function for setting a field of view securing area and performing different display control for condition-satisfying objects and condition-unsatisfying objects may be configured to be switched between "ON (performs the control)" and "OFF (does not perform the control)" as needed. For example, an ON/OFF switch button may be placed on the viewing screen to accept a switching operation by the viewer, so that the function can be switched ON/OFF for each viewer. Alternatively, a period (time period during live distribution) during which the function is turned OFF may be set in advance, and the function is turned OFF during that period. If there is a video poster, the video poster may be allowed to set the time period during which the function is turned OFF.

[Modification 12]
In addition, in the above embodiments, examples have been shown in which tipping is posted by selectively using tipping objects with different tipping amounts, but a configuration may be adopted in which a tipping amount can be input and any amount of tipping can be posted. Tips can include free tipping, in addition to tipping that requires consuming currency points. In addition, the configuration may be such that supportive comments, impression comments, etc. can be posted, and comments and tips can be posted in combination. It is possible to accept input of comments that include, in addition to text (letters, numbers, symbols), images such as illustrations and photographs (also called stamps, etc.) as appropriate.

1000...Content viewing system 1100...Server system 100s...Operation input unit 200s...Server processing unit 210...User management unit 220...Charge processing unit 230...Video provision control unit 231...Live progress control unit 233...Viewer-specific video generation unit 235...Object movement control unit 237...View securing area setting unit 239...Condition satisfying object display control unit 241...Condition satisfying object detection unit 243...Form change control unit 245...Viewpoint control unit 290s...Image generation unit 292s...Sound generation unit 294s...Communication control unit 390s...Image display unit 392s...Sound output unit 394s...Communication unit 500s...Server storage unit 501...Server program 503...Distribution viewing program 510...Tipping object definition data 520...User management data 523...Payment medium ledger data 530...Video provision management data 533: Distribution schedule 537: Viewer account list 540: Tipping posting data 550: Viewing space management data 551: Performer character control data 553: Spectator viewpoint camera control data 555: Tipping object movement control data 560: Tipping object display control data for each viewer 563: Viewing area data 565: Object classification table 570: Video data for each viewer 1500, 1500A: Viewer terminal 100: Operation input unit 200: Terminal processing unit 270: User terminal calculation unit 271: Operation signal transmission control unit 273: Image display control unit 290: Image generation unit 292: Sound generation unit 294: Communication control unit 390: Image display unit 392: Sound output unit 394: Communication unit 500: Terminal memory unit 502: Viewing program 504: Video provision program N: Network 1200: Video publication management server 1300: Electronic settlement server 2: User (viewer)
10: Viewing space 3: Performer character 4: Tip object 8: Spectator viewpoint camera 40: Viewing range 50, 60, 60A: Viewing area

Claims (19)

1. A computer system for making an object corresponding to a viewer action of each viewer appear in a virtual viewing space, and providing each viewer with an image of the viewing space seen from a viewpoint for the viewer, comprising :
a setting means for setting, in the viewing space , a viewpoint for each of the viewers and a viewing area based on the viewpoint ;
a condition satisfying object display control means for controlling the display of the video for each of the viewers by making the display control of an object satisfying an in-area condition indicating that the object passes through or is located within the view securing area of the viewer (hereinafter referred to as a "condition satisfying object") different from the display control of an object not satisfying the in-area condition (hereinafter referred to as a "condition non-satisfying object");
A computer system comprising: an object movement control means for making the object appear in the viewing space and move it along a given trajectory;
The computer system of claim 1 further comprising: A computer system for making an object corresponding to a viewer action of each viewer appear in a virtual viewing space, and providing a video of the viewing space seen from a viewpoint for a target viewer to the target viewer, comprising:
A setting means for setting a viewing area for the target viewer;
an object movement control means for causing the object to appear in the viewing space and to move along a given trajectory;
a condition satisfying object display control means for controlling display of an object that satisfies an in-area condition indicating that the object passes through or is located within the visual field securing area (hereinafter referred to as a "condition satisfying object") differently from display of an object that does not satisfy the in-area condition (hereinafter referred to as a "condition non-satisfying object");
Equipped with
the condition satisfying object display control means detects the condition satisfying object from among the objects whose movement by the object movement control means has not yet ended and which are still moving;
Computer system. the object movement control means moves the object along the trajectory until a predetermined end condition is satisfied ;
4. A computer system according to claim 2 or 3 . the condition satisfying object display control means includes trajectory change control means for changing the trajectory of the condition satisfying object in a direction away from the field of view securing area,
A computer system according to any one of claims 2 to 4 . the condition satisfying object display control means includes a form change control means for controlling a display form of the condition satisfying object to be different from a display form of the condition non-satisfying object;
The computer system according to any one of claims 1 to 5 . the form change control means changes the display form of the condition satisfying object to one having higher transparency and/or smaller size than the condition non-satisfying object;
7. The computer system of claim 6 . A character whose movement is controlled based on a given movement control is placed in the viewing space,
the setting means sets the visual field securing area based on the character.
A computer system according to any one of claims 1 to 7 . the setting means sets the field of view securing area based on a positional relationship between the viewpoint and the character.
9. The computer system of claim 8 . the setting means sets the visual field securing area based on a setting specification of the visual field securing area associated with the character;
9. The computer system of claim 8 . a viewpoint control means for changing the direction of the viewpoint and/or the position of the viewpoint based on an operational input from the viewer;
Further comprising:
The setting means updates and sets the visual field securing area in response to control by the viewpoint control means.
10. The computer system of claim 9 . the setting means variably sets the field of view securing area based on an input status of the viewer action.
A computer system according to any one of claims 1 to 11 . the condition satisfying object display control means variably controls display control of the condition satisfying object based on an input status of the viewer action.
A computer system according to any one of claims 1 to 12 . The setting means sets the visual field securing area within a visual field range of the viewpoint.
A computer system according to any one of claims 1 to 13 . the condition satisfying object display control means excludes the object corresponding to a viewer action of the viewer related to the view securing area from among the condition satisfying objects, and detects the condition satisfying object from among the objects corresponding to viewer actions of viewers other than the viewer related to the view securing area.
A computer system according to any one of claims 1 to 14. The viewer action is an operation input input by consuming a given consumption value from a parameter value owned by the viewer.
A computer system according to any one of claims 1 to 15 . A server system that is a computer system according to any one of claims 1 to 16 and is communicatively connected to a viewer terminal of each viewer;
The viewer terminal;
A content viewing system comprising: A program for causing a computer system to make an object corresponding to a viewer action of each viewer appear in a virtual viewing space, and to provide each of the viewers with an image of the viewing space seen from a viewpoint for the viewer, the program comprising:
a setting means for setting, in the viewing space , a viewpoint for each of the viewers and a field of view securing area based on the viewpoint ;
a condition satisfying object display control means for controlling the display of the video for each of the viewers by making the display control of an object satisfying an in-area condition indicating that the object passes through or is located within the view securing area of the viewer (hereinafter referred to as a "condition satisfying object") different from the display control of an object not satisfying the in-area condition (hereinafter referred to as a "condition non-satisfying object");
A program for causing the computer system to function as the A program for causing a computer system to make an object corresponding to a viewer action of each viewer appear in a virtual viewing space, and to provide a video of the viewing space seen from a viewpoint for a target viewer to the target viewer,
A setting means for setting a viewing area for the target viewer;
an object movement control means for making the object appear in the viewing space and move it along a given trajectory;
a condition satisfying object display control means for differentiating display control of an object satisfying an in-area condition indicating that the object passes through or is located within the visual field securing area (hereinafter referred to as a "condition satisfying object") from display control of an object not satisfying the in-area condition (hereinafter referred to as a "condition non-satisfying object");
and causing the computer system to function as
the condition satisfying object display control means detects the condition satisfying object from among the objects whose movement by the object movement control means has not yet ended and which are still moving;
program.

Images