JP7794411B2 - METHOD, DEVICE AND SYSTEM FOR DISPLAYING DANMAKE COMMENTS IN FREE VIEWPOINT VIDEO - Google Patents

Description

[ Technical Field]
This application relates to the field of Internet technology, and in particular to a method, an apparatus and a system for displaying bullet comments of a free viewpoint video.

Free viewpoint video (FVV) is a special-format video file in which a target object is jointly photographed by multiple surrounding cameras to generate a video stream that is encoded into a file such as a special-format video file. The generated free viewpoint video is then uploaded to a media server for storage and then distributed to a terminal through a content distribution network (CDN). When viewing the free viewpoint video by using a terminal, a user may freely switch viewpoints through a finger-dragging operation. Compared with traditional video, free viewpoint video allows users to freely select viewpoints, thereby overcoming the drawbacks of traditional video in which the viewpoint is determined by a director. Free viewpoint video provides users with an immersive and interactive playback experience.

During playback of a free viewpoint video, the user can enable the danmaku comment function. Danmaku comments refer to the scrolling, leaving, or even more action-effect-style display of a large number of user comments on the video image, which is currently a popular form of entertainment. It should be noted that the terminal not only displays the video image of the free viewpoint video from a certain viewpoint (e.g., viewpoint A), but also displays the danmaku comments from all viewpoints at the current playback point. For example, the terminal not only displays the danmaku comments from viewpoint A, but also displays the danmaku comments from viewpoint B, viewpoint C, etc. In this case, in some scenarios, the content of the danmaku comments from viewpoint B and/or viewpoint C may not be consistent with the content of the video image from viewpoint A, resulting in confusion and a disorienting and unsatisfactory video experience for the user.

The method, device, and system for displaying danmaku comments in free viewpoint video provided in this application improve the viewing experience by helping users focus on viewing the current video image and the content of the danmaku comments related to the current video image.

To achieve the above objectives, embodiments of this application provide the following technical solutions:

According to a first aspect, a method for displaying danmaku comments for a free viewpoint video, which is applied to a terminal, is provided. The method includes the following: The terminal displays a first interface for a first free viewpoint video at a first playback time point, the first interface including a video image of the first free viewpoint video at a first viewpoint and a first danmaku comment; the terminal receives a first operation, the first operation being an operation for switching the playback viewpoint of the first free viewpoint video from the first viewpoint to a second viewpoint; and in response to receiving the first operation, the terminal displays a second interface for the first free viewpoint video at the first playback time point, the second interface including a video image of the first free viewpoint video at a second viewpoint, and the second interface not including the first danmaku comment.

The offset angle between the first viewpoint and the third viewpoint corresponding to the first danmaku comment is within the first preset angle range. Therefore, the first danmaku comment is displayed in the first interface. The offset angle between the second viewpoint and the third viewpoint corresponding to the first danmaku comment is outside the first preset angle range. Therefore, the first danmaku comment is not displayed in the second interface.

It can be seen that when playing back the first free viewpoint video, the terminal can display danmaku comments within a first preset angle range (hereinafter referred to as the visible viewpoint range) and will not display danmaku comments outside the first preset angle range. Since the content of the danmaku comments within the first preset angle range is usually strongly related to the content of the video image currently being viewed by the user at the playback viewpoint, the user can focus on viewing the current video image and the content of the danmaku comments related to the current video image, improving the viewing experience.

In a possible implementation, the offset angle between the first viewpoint and the third viewpoint corresponding to the first danmaku comment is smaller than the offset angle between the second viewpoint and the third viewpoint corresponding to the first danmaku comment.

For example, the terminal may predefine the visible viewpoint range of the danmaku comment, or may receive the visible viewpoint range of the danmaku comment set by the user, as follows: The absolute value of the offset angle between the viewpoint of the danmaku comment and the playback viewpoint of the first free viewpoint video is less than the angle threshold E (e.g., 45 degrees). In this case, the terminal may determine that the visible viewpoint range of the danmaku comment is [playback viewpoint of the first free viewpoint video - angle threshold E, playback viewpoint of the first free viewpoint video + angle threshold E].

It can be seen that the offset angle between the first viewpoint and the third viewpoint corresponding to the first danmaku comment is within a first preset angle range, and the offset angle between the second viewpoint and the third viewpoint corresponding to the first danmaku comment is outside the first preset angle range. It can also be inferred that the offset angle between the first viewpoint and the third viewpoint corresponding to the first danmaku comment is smaller than the offset angle between the second viewpoint and the third viewpoint corresponding to the first danmaku comment.

In a possible implementation, before the terminal receives the first operation, the method further includes: the terminal receives a second operation to pause playback of the first free viewpoint video.

In one example, in the process of switching viewpoints, the terminal automatically pauses playback of the first free viewpoint video. Thus, before the user performs the first operation, the terminal plays video images of the first free viewpoint video at the first viewpoint. When the terminal detects that the user has performed the first operation at the first playback time point, it automatically pauses playback of the first free viewpoint video. After the first operation is performed and the playback viewpoint of the first free viewpoint video is switched from the first viewpoint to the second viewpoint, the first free viewpoint video begins to be played back from the first playback time point.

In another example, in the process of switching viewpoints, the terminal continuously plays back the first free viewpoint video. In this case, before performing the first operation, the user may first manually pause the playback of the first free viewpoint video. In this case, the first free viewpoint video remains at the first playback point. Then, after the user performs the first operation and the terminal switches the playback viewpoint of the first free viewpoint video from the first viewpoint to the second viewpoint, the video image of the first free viewpoint video still remains at the first playback point.

Alternatively, after the first operation is performed and the terminal switches the playback viewpoint of the first free viewpoint video from the first viewpoint to the second viewpoint, playback begins from a playback point after the first playback point. Alternatively, the user may manually drag the playback progress bar to adjust the playback point back to the first playback point.

In a possible implementation , the second interface includes a second danmaku comment, and an offset angle between the second viewpoint and a viewpoint corresponding to the second danmaku comment is within a first preset angle range.

In a possible implementation, the sending time of the first danmaku comment is relative to the first playback time, and the sending time of the second danmaku comment is relative to the first playback time.

It can be understood that the first danmaku comment is played back in the first interface and displayed on the video image at the first playback time. In this case, the first playback time is equal to the time of transmission of the first danmaku comment, or is after the time of transmission of the first danmaku comment and within the continuous display duration of the first danmaku comment. Similarly, the first playback time is equal to the time of transmission of the second danmaku comment, or is after the time of transmission of the second danmaku comment and within the continuous display duration of the second danmaku comment.

In a possible implementation, the first interface displayed by the terminal further includes a first control, the first control being associated with a first number of danmaku comments, the first number of danmaku comments being the number of danmaku comments obtained when the offset angle between the viewpoint of the danmaku comments and the playback viewpoint of the terminal is within a second preset angle range, and the second preset angle range does not overlap with the first preset angle range.

In one example, the first control displays the number of danmaku comments obtained when the offset angle between the viewpoint of the danmaku comment and the playback viewpoint of the device is within a second preset angle range.

In a possible implementation, the method further includes: the terminal receives a third operation to switch the viewpoint; the terminal displays a fourth interface during the third operation, and the number of barrage comments indicated by the first control changes in the fourth interface; after the third operation, the terminal displays a second interface of the first free viewpoint video at the second playback point.

In a possible implementation, the terminal receives a third operation for the first control. In response to receiving the third operation, the terminal displays a third interface, the third interface including the first barrage comment.

When a user is interested in the content of barrage comments outside the visible view range, it is understood that the content of barrage comments outside the visible view range may alternatively be expanded by using the corresponding controls to meet the user's viewing requirements in multiple scenarios.

In a possible implementation, after the terminal displays the second interface of the first free viewpoint video at the first playback time point in response to receiving the first operation, the method further includes: the terminal receives a fourth operation, the fourth operation being an operation for sending a third danmaku comment; and in response to receiving the fourth operation, the terminal sends data of the third danmaku comment to a danmaku comment server, where the data of the third danmaku comment includes a fourth viewpoint.

When the user submits a third danmaku comment in the second interface, it is understood that an association is established between the viewpoint of the third danmaku comment and the playback viewpoint of the first free viewpoint video in the second interface, i.e., the fourth viewpoint. Subsequently, when the third danmaku comment is displayed, it is determined whether or not to present the third danmaku comment based on the relationship between the playback viewpoint of the first free viewpoint video and the viewpoint of the third danmaku comment.

In a possible implementation, the first interface further includes a fourth danmaku comment, the fourth danmaku comment corresponding to the first viewpoint.

In a possible implementation, in the first interface, the first style of the first danmaku comment is different from the second style of the fourth danmaku comment.

It can be seen that the user may distinguish the danmaku comments in the first free viewpoint video at the playback viewpoint from the danmaku comments at other viewpoints based on the presentation style of the danmaku comments (e.g., first style or second style). In this way, the user will not be confused or suspicious when the video image at the playback viewpoint does not match the content of the danmaku comments at other viewpoints. This improves the viewing experience of the first free viewpoint video.

In a possible implementation, before the terminal displays a first interface of the first free viewpoint video at the first playback time point, the method includes: the terminal determines to display the first danmaku comment in a first style on the first interface based on a first offset angle between the first viewpoint and a third viewpoint corresponding to the first danmaku comment.

In a possible implementation, when the first offset angle is a positive value, the first style is a direction in which the text direction of the barrage comments is obtained by rotating the first direction clockwise by the first angle, and the first direction is a direction parallel to the first bezel of the terminal screen; alternatively, when the first offset angle is a negative value, the first style is a direction in which the text direction of the barrage comments is obtained by rotating the first direction counterclockwise by the first angle.

In a possible implementation, the method further includes: the terminal receives a fifth operation for switching the viewpoint; the terminal displays a fifth interface during the fifth operation, where the fifth interface and the first interface are the same; after the fifth operation, the terminal displays a sixth interface of the first free viewpoint video at the second playback point.

In other words, in the fifth operation process, the video image of the first free viewpoint video displayed by the terminal remains unchanged, and the presentation method of the barrage comments also remains unchanged.

In a possible implementation, the method further includes: the terminal receiving a sixth operation to switch the viewpoint.

The terminal displays a seventh interface during the sixth operation process, and the seventh interface includes a video image of the first free viewpoint video at the third playback point in time. After the sixth operation process, the terminal displays an eighth interface of the first free viewpoint video at the second playback point in time, and the third playback point in time is after the first playback point in time and before the second playback point in time.

In other words, in the sixth operation process, the terminal continuously plays the first free viewpoint video, and therefore the playback point of the first free viewpoint video changes.

In a possible implementation, the method further includes: the seventh interface further includes a fifth barrage comment, and the first interface does not include the fifth barrage comment.

In other words, in the sixth operation process, the terminal continuously plays back the first free viewpoint video, and the barrage comments displayed by the terminal also change accordingly as the playback time of the first free viewpoint video changes.

In a possible implementation, the method further includes: the seventh interface further includes a first danmaku comment, and the display style of the first danmaku comment in the seventh interface is different from the display style of the first danmaku comment in the first interface.

In other words, during the sixth operation process, the terminal adjusts the display style of each barrage comment in real time based on changes in the playback viewpoint.

In a possible implementation, the seventh interface further includes a sixth danmaku comment, and the sixth danmaku comment appears after the first playback point.

In other words, after the viewpoint of the first free viewpoint is switched, the device continuously plays back the first free viewpoint video, and the content of the barrage comments changes accordingly as the playback point of the first free viewpoint video changes.

According to a second aspect, there is provided a terminal including a processor, a memory, and a touchscreen. The memory and the touchscreen are coupled to the processor, and the memory is configured to store computer program code, the computer program code including computer instructions. When the processor reads the computer instructions from the memory, the terminal is enabled to execute a method according to the first aspect and any one of the possible implementations of the first aspect.

According to a third aspect, there is provided a computer-readable storage medium containing computer instructions. When the computer instructions are executed on a terminal, the terminal is enabled to perform a method according to the first aspect and any one of possible implementations of the first aspect.

According to a fourth aspect, there is provided a computer program product which, when run on a computer or a terminal , enables the computer or the terminal to perform the method according to the first aspect and any one of the possible implementations of the first aspect.

For technical effects that can be achieved by the terminal, computer-readable storage medium, and computer program product provided in the second to fourth aspects, please refer to the description of the technical effects of the first aspect and possible implementation methods of the first aspect. Details will not be described again here.

1 is a schematic diagram of the structure of a system for playing free viewpoint video according to an embodiment of this application; 1 is a schematic diagram of the structure of a terminal according to an embodiment of this application; FIG. 2 is a schematic diagram of the structure of a server according to an embodiment of the present application; 1 is a schematic flowchart of a method for transmitting barrage comments of a free viewpoint video according to an embodiment of the present application; 1 is a schematic diagram of several terminal interfaces according to an embodiment of the present application; 1 is a schematic diagram of several terminal interfaces according to an embodiment of the present application; 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 1 is a schematic flowchart of a method for displaying barrage comments according to an embodiment of the present application; 1 is a schematic diagram of a relationship between a viewpoint of a barrage comment and a playback viewpoint of a free viewpoint video according to an embodiment of the present application; 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 1 is a schematic flowchart of a method for displaying barrage comments in a process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; FIG. 10 is a schematic diagram of another relationship between the viewpoint of the barrage comments and the playback viewpoint of the free viewpoint video according to an embodiment of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application;

To facilitate understanding of the technical solutions provided in the embodiments of this application, we will first explain the technical terms used in the embodiments of this application.

In free viewpoint video, a target object is simultaneously captured by multiple cameras, and during playback, this is equivalent to the user selecting the capture angle of one of the cameras for viewing. Free viewpoint video may be generated in several ways:

Method 1: Multiple cameras arranged around a target object jointly capture the target object by using the target object as the center of the image, and specific video encoding is performed on the jointly captured video stream to generate a special format video file. When a user views the free viewpoint video, this is equivalent to the viewer selecting a viewpoint around the captured target object for viewing.

Method 2: Multiple cameras are arranged around the viewer, using the viewer as the center, to jointly capture a target object, and specific video encoding is performed on the jointly captured video stream to generate a special format video file, for example, to generate 3-DoF (degrees of freedom) video (e.g., 360-degree panoramic video). When a user views free-viewpoint video, this is equivalent to the viewer rotating their head to view the surrounding scenery without moving.

Method 3: The above two methods are combined. For example, a 6-DoF image is generated. When viewing a free viewpoint image, the user can move forward, backward, left, or right to view the image, or rotate their head to view the surrounding scenery.

A camera position is the number of the position at which each camera is located when multiple cameras around the target object or view capture the target object. Generally, one camera position corresponds to one shooting angle. At one shooting angle, images of the target object within a specific angle range may be captured.

The viewpoint is also called a viewing point , a playback viewpoint, etc., and is a shooting angle at which a user views a free viewpoint video. Specifically, an image of a target object seen at a viewpoint may be considered to be an image captured when a camera performs shooting at the position of the viewpoint (i.e., the camera position).

Video playback is the process of displaying a group of images in chronological order in a display window and playing the corresponding audio after a video file (including free viewpoint video) has been decoded, rendered, etc.

Danmaku comments are comments or annotations that appear on a video (including free viewpoint video) at a specific point or for a specific period of time according to specific rules, creating an interesting and unexpected video viewing experience.

Figure 1 is a schematic diagram of a system architecture according to an embodiment of this application. The system architecture includes a camera array 500, a content distribution network (CDN) 200, a media server 300, a CDN 400, a terminal 100, and a barrage comment server 600.

Camera array 500 includes multiple (e.g., 18-36) cameras or other devices with camera lenses surrounding a target object (photography object/ scene ) and/or viewer. These cameras correspond to camera position 0, camera position 1, ..., and camera position n, respectively. Camera array 500 separately captures images of the target object from multiple angles and uploads the video streams captured from the multiple angles to media server 300. In some examples, synchronization of the images captured by multiple cameras in the camera array or other devices with camera lenses may be achieved by using a metronome.

By using functional modules such as a load balancing module, a content distribution module, and a scheduling module on a central platform, the CDN 200 enables video streams collected by the camera array 500 from multiple angles to be uploaded to the media server 300 in an accelerated manner by using edge servers located at each location.

The media server 300 is configured to transcode the received video streams at multiple angles to generate free-viewpoint video. For example, processing processes performed by the media server 300 on the video streams at multiple angles include, but are not limited to, focusing, frame alignment synchronization encoding, rotation auxiliary stream generation, virtual viewpoint synthesis, etc. Focusing may be used to ensure a smooth transition of the image of a captured object in the process of switching viewpoints, in a manner similar to image cropping. Frame alignment synchronization encoding is used to align video frames captured by multiple cameras at the same time. In virtual viewpoint synthesis, a virtual video stream is inserted between two real video streams to reduce the number of cameras that need to be used during filming or to achieve a smooth transition of the image of a captured object in the process of switching viewpoints. The rotation auxiliary stream is used for displaying in a rotational manner, which helps to reduce the total stream usage.

It should be noted that the above processing process of the media server 300 is merely an example, and the specific technical solutions used in the process of generating free viewpoint videos based on the video streams uploaded by the camera array 500 are not limited to the embodiments of this application.

It should be further noted that the media server 300 herein may be at least one of an independent physical server, multiple independent physical servers, a cloud server providing cloud computing, a cloud computing platform, and a virtualization center.

It can be seen that the media server 300 stores and maintains a large amount of free-viewpoint video. When the media server 300 receives a request sent by the terminal 100 for free-viewpoint video content, the media server 300 quickly returns the corresponding free-viewpoint video content to the terminal 100 via the CDN 400.

It should be noted that the media server 300 here may be at least one of an independent physical server, multiple independent physical servers, a cloud server providing cloud computing, a cloud computing platform, and a virtualization center.

CDN 400 enables terminal users (e.g., terminal 100) to obtain nearby required content, such as free-viewpoint video, by using functional modules such as a load balancing module, a content distribution module, and a scheduling module on a central platform, and by using edge servers located at each location, thereby reducing network congestion and improving user access response speed and hit rate.

The terminal 100 may request free-viewpoint video from the media server 300 by using a free-viewpoint video client/application installed on the terminal 100, and then receive the free-viewpoint video file returned by the media server 300 through the CDN 400. Alternatively, the terminal 100 downloads the free-viewpoint video file from the corresponding server based on the free-viewpoint video playback address returned by the media server 300. Finally, the terminal performs processing such as decoding and rendering on the free-viewpoint video file, and then plays the free-viewpoint video. If the terminal 100 also enables the barrage comment function when playing back the free-viewpoint video, the terminal 100 must further request barrage comment data corresponding to the current playback time or period from the barrage comment server 600.

In this embodiment of the application, the danmaku comment data returned by the danmaku comment server 600 includes the content of the danmaku comment, the time when the danmaku comment appeared, the viewpoint of the danmaku comment, etc. The viewpoint of the danmaku comment is the viewpoint of the free viewpoint video being viewed by the user when the danmaku comment was sent. It can be understood that the content of the danmaku comment sent by the user is usually related to the video image at the current viewpoint . For example, the content of the danmaku comment sent by the user is usually a comment or explanation regarding the video image at the current viewpoint . In other words, in this embodiment of the application, the danmaku comment sent by the user is related to the current playback viewpoint.

In this embodiment of the application, the terminal 100 uses different display methods for multiple danmaku comments at different viewpoints based on the viewpoint of the danmaku comments to help the user distinguish between multiple danmaku comments at different viewpoints. In other words, the danmaku comments at different viewpoints are displayed differently. For example, in a video image at the user's current playback viewpoint 1, danmaku comments at viewpoint 1 and viewpoint 2 are currently displayed. In this case, the terminal 100 separately displays the danmaku comments at viewpoint 1 and the danmaku comments at viewpoint 2 in different display methods, thereby allowing the user to intuitively distinguish between the danmaku comments at viewpoint 1 and the danmaku comments at viewpoint 2. When the danmaku comments at viewpoint 2 do not match the video image at viewpoint 1, the user will not be confused or perplexed. This improves the video viewing experience.

Furthermore, when playing back the free viewpoint video, the terminal 100 may also receive new content of the danmaku comment entered by the user, establish an association between the new content of the danmaku comment and the viewpoint of the video currently being viewed by the user, and upload the association to the danmaku comment server 600 for storage.

The terminal 100 may further receive a viewpoint switching operation performed by the user (e.g., a user's finger sliding operation on the screen or a user's operation of pressing a button on a remote control), and may calculate a viewpoint switching direction (e.g., switching the viewpoint to the left or switching the viewpoint to the right) and a switching angle amplitude (e.g., switching by 10 degrees or 20 degrees) based on the viewpoint switching operation performed by the user, and switch the viewpoint of the current free viewpoint video. In some examples, the display of the terminal 100 is a touch screen. In this case, the terminal 100 may capture screen events, the screen events including a detected finger slide event, the finger slide direction and distance, and determine a viewpoint switching direction and a switching angle based on the screen event. In some other examples, the terminal 100 is configured as a remote control. In this case, the terminal 100 may receive a button event from a remote control, where the button event includes a button identifier, the number of times the button is pressed, or the duration of the button press, etc., and then determine a viewpoint switching direction and a switching angle based on the button event, for example, identify the button operated by the user based on the button identifier, for example, tapping the left button or the right button, to determine the switching direction, and identify the switching angle amplitude based on the number of times the button is pressed or the duration of the button press.

For example, the terminal 100 in this embodiment of the present application may be, for example, a mobile phone, a tablet computer, a personal computer (PC), a personal digital assistant (PDA), a netbook, a wearable device, an augmented reality (AR) device, a virtual reality (VR) device, an in-vehicle device, a smart screen, or an intelligent vehicle. The specific form of the terminal 100 is not particularly limited in this application.

The barrage comment server 600 is configured to receive barrage comment information transmitted by each terminal (including, for example, terminal 100), and the barrage comment information includes, but is not limited to, an identifier for the free viewpoint video, the time at which the barrage comment appears, the duration of the barrage comment, the content of the barrage comment, the viewpoint of the barrage comment, etc.

It should be noted that the barrage comment server 600 here may be at least one of an independent physical server, multiple independent physical servers, a cloud server providing cloud computing, a cloud computing platform, and a virtual center.

It can be understood that the system architecture shown in this embodiment of the present invention does not constitute a specific limitation on the system architecture to which the technical solutions are applicable. In some other embodiments of this application, the system architecture to which the technical solutions provided in the embodiments of this application are applicable may further include more or fewer devices than those shown in the drawings , or a combination of some of the devices, or a division of some of the devices .

FIG. 2 is an exemplary diagram of the structure of a terminal 100 according to an embodiment of the present application. The terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, a subscriber identification module (SIM) card interface 195, etc.

It may be understood that the structure shown in this embodiment of the present invention does not constitute a specific limitation on the terminal 100. In some other embodiments of this application, the terminal 100 may include more or fewer components than those shown in the drawings , or a combination of some of the components, or a division of some of the components, or a different arrangement of the components . The components shown in the drawings may be implemented in hardware, software, or a combination of software and hardware.

Processor 110 may include one or more processing units. For example, processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU), and/or the like. The different processing units may be separate components or may be integrated into one or more processors.

A video codec is configured to compress or decompress digital video. Terminal 100 may support one or more video codecs. Thus, terminal 100 may play or record video in multiple encoding formats, such as Moving Picture Experts Group (MPEG)-1, MPEG-2, MPEG-3, and MPEG-4. In this embodiment of the application, the video codec may decode free-viewpoint video downloaded by terminal 100 from media server 300 for playback on terminal 100.

The internal memory 121 may be configured to store computer-executable program code. The executable program code includes instructions. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store an operating system, applications required by at least one function (e.g., free-viewpoint video playback), etc. The data storage area may store data created when the terminal 100 is used (e.g., audio data and contacts), etc. Furthermore, the internal memory 121 may include high-speed random access memory, or may include non-volatile memory such as at least one magnetic disk storage device, flash memory, or universal flash storage (UFS). The processor 110 executes instructions stored in the internal memory 121 and/or instructions stored in memory located in the processor to perform various functional applications and data processing of the terminal 100.

In some embodiments of this application, internal memory 121 stores program code required by a free viewpoint video client/application. The free viewpoint video client/application may implement functions such as downloading free viewpoint video from media server 300 and playing the free viewpoint video, receiving the content of danmaku comments entered by the user, uploading the content of the danmaku comments entered by the user and other danmaku comment information (e.g., danmaku comment viewpoint information) to danmaku comment server 600, receiving danmaku comment data transmitted by danmaku comment server 600, and displaying danmaku comments from different viewpoints differently based on the viewpoint information carried in the danmaku comment data, etc.

The wireless communication functions of terminal 100 may be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, baseband processor, etc.

Mobile communication module 150 may provide wireless communication solutions, including 2G/3G/4G/5G/6G/next-generation wireless communication technologies, etc., applied to terminal 100. In some embodiments, at least some of the functional modules in mobile communication module 150 may be located in processor 110. In some embodiments, at least some of the functional modules of mobile communication module 150 may be located in the same device as at least some of the modules of processor 110.

The wireless communication module 160 may provide a solution for wireless communication applied to the terminal 100, including a wireless local area network (WLAN) (e.g., a wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc.

The terminal 100 may implement display functions by using a GPU, a display 194, an application processor, etc. The GPU is a microprocessor for image processing and is connected to the display 194 and the application processor. The GPU is configured to perform mathematical and geometric calculations and render images. The processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

The terminal 100 may realize the photographing function by using an ISP, camera 193, video codec, GPU, display 194, application processor, etc.

The external memory interface 120 may be used to connect an external memory card, such as a microSD card, to expand the storage of the terminal 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement data storage functions. For example, files such as music and videos may be stored on the external memory card.

The terminal 100 may implement audio functions such as music playback and recording by using an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, an application processor, etc.

The charging management module 140 is configured to receive a charging input from a charger. The charger may be a wireless charger or a wired charger. In some embodiments where wired charging is performed, the charging management module 140 may receive the charging input of the wired charger through the USB interface 130. In some embodiments where wireless charging is performed, the charging management module 140 may receive the wireless charging input through a wireless charging coil in the terminal 100. When charging the battery 142, the charging management module 140 may also provide power to the terminal by using the power management module 141.

Power management module 141 is configured to connect to battery 142, charging management module 140, and processor 110. Power management module 141 receives input from battery 142 and/or charging management module 140 and provides power to processor 110, internal memory 121, display 194, camera 193, wireless communication module 160, etc. Power management module 141 may be further configured to monitor parameters such as battery capacity, battery cycle count, and battery health status (fault or impedance). In some other embodiments, power management module 141 may alternatively be located in processor 110. In some other embodiments, power management module 141 and charging management module 140 may alternatively be located in the same component.

Figure 3 is a schematic diagram of the structure of the barrage comment server 600. The barrage comment server 600 includes one or more processors 610, one or more memories 620, and one or more communication interfaces 630.

The processor 610, memory 620, and communication interface 630 are connected via a bus. The processor 610 may include a general-purpose central processing unit (CPU) (e.g., CPU0 and CPU1), a microprocessor, an application-specific integrated circuit (ASIC), a graphics processing unit (GPU), a neural-network processing unit (NPU), an integrated circuit configured to control program execution of the solutions of this application, etc.

The memory 620 may be configured to store computer-executable program code. The executable program code includes instructions. The memory 620 may include a program storage area and a data storage area. The program storage area may store an operating system, application code, etc. In some examples, the data storage area stores danmaku comment data for each free viewpoint video uploaded by a user, and the danmaku comment data includes the viewpoint of each danmaku comment. Furthermore, the memory 620 may include high-speed random access memory and may further include non-volatile memory, such as at least one magnetic disk storage device, flash memory, or universal flash storage (UFS). The processor 610 executes instructions stored in the memory 620 to perform various functional applications and data processing of the danmaku comment server 600. In one example, the processor 610 may alternatively include multiple CPUs, and the processor 610 may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor, as used herein, may be one or more devices, circuits, or processing cores configured to process data (e.g., computer program instructions).

The communication interface 630 may be configured to communicate with other devices or communication networks, such as Ethernet or a wireless local area network (WLAN).

It should be noted that the media server 300, servers within CDN 200 or CDN 400, etc. shown in FIG. 1 refer to the structure of the barrage comment server 600 shown in FIG. 3. The media server 300 and servers within CDN 200 or CDN 400 may include more or fewer components than the barrage comment server 600, or some components may be combined, or some components may be separated, or a different component layout may exist. This is not a limitation of the embodiments of this application.

The technical solutions provided in the embodiments of this application are described in detail below with reference to the accompanying drawings and specific embodiments.

1. Solution for sending barrage comments

Figure 4 is a schematic flowchart of a method for sending barrage comments according to an embodiment of the present application. The method includes the following steps:

S400: Terminal 100 plays back the video image of the first free viewpoint video at viewpoint A.

For example, a client (e.g., a web page) or application (e.g., a video application on a mobile device) used to play free-viewpoint video is installed on terminal 100. The client/application may be referred to as a free-viewpoint video client/application. A user may launch the client/application and select to play the first free-viewpoint video in the display interface of the client/application. In some embodiments, terminal 100 plays video images of the first free-viewpoint video from viewpoint A by default. When detecting a user's operation selecting playback of the first free-viewpoint video, terminal 100 requests video file 1 of the first free-viewpoint video from viewpoint A from media server 300 via CDN 400. In response, media server 300 returns video file 1 to terminal 100 via CDN 400, and terminal 100 displays video images of the first free-viewpoint video from viewpoint A based on video file 1. In some other embodiments, terminal 100 first plays video images of the first free-viewpoint video from a viewpoint other than viewpoint A. When the terminal 100 detects a user operation indicating switching to viewpoint A, the terminal 100 requests video file 1 of the first free viewpoint video at viewpoint A from the media server 300 via the CDN 400. In response, the media server 300 returns video file 1 to the terminal 100 via the CDN 400, and the terminal 100 displays the video image of the first free viewpoint video at viewpoint A based on video file 1.

In some other examples, when the terminal 100 plays back the first free viewpoint video, the danmaku comment function is enabled. In this case, the terminal 100 further requests danmaku comment data at the corresponding playback point from the danmaku comment server 600 based on the playback progress. In this case, the video image played back by the terminal 100 further includes the corresponding danmaku comment.

For example, an example in which the terminal 100 is a mobile phone will be used for explanation. A video application installed on the mobile phone supports free-viewpoint video playback by integrating a software development kit (SDK) for playing free-viewpoint videos. In this case, a user can launch the video application on the mobile phone and select to play a first free-viewpoint video in the video application interface. For example, FIG. 5(1) shows a playback interface 601 displayed by the mobile phone and used to play a first free-viewpoint video. The playback interface 601 includes a video image 602 of the first free-viewpoint video at a specific viewpoint and danmaku comments (including, for example, danmaku comments 603) that appear on the video image 602 according to specific rules. The playback interface 601 may further include several function controls, such as a playback progress bar 604, a playback control 609, a next episode control 610, a danmaku enable/disable control 605, a danmaku comment input box 606, a danmaku comment send control 607, and a viewpoint control 608.

The playback progress bar 604 is used to display the playback progress of the current video and is used by the user to adjust the playback progress of the current video by dragging, etc. The playback control 609 is used to pause or continuously play the current video. The next episode control 610 is used to switch to the next episode or next video. The danmaku enable/disable control 605 is used to enable or disable the danmaku comment function. The danmaku comment input box 606 is used to accept the content of the danmaku comment entered by the user. The danmaku comment send control 607 is used to send the content of the danmaku comment entered by the user or the default content of the danmaku comment.

The viewpoint control 608 is used to display the viewpoint (e.g., 60 degrees) corresponding to the video image 602 currently being played by the mobile phone, and is used to call up a viewpoint ruler used to switch viewpoints. When it is detected that the user has operated the viewpoint control 608, the mobile phone displays the playback interface 611 shown in FIG. 5(2), and the viewpoint ruler 612 is displayed on the playback interface 611. The user may switch the viewpoint of the free viewpoint video currently being played by the mobile phone by sliding left or right on the viewpoint ruler 612 using their finger. For example, when it is detected that the user's finger has slid right on the ruler, the mobile phone displays the playback interface 613 shown in FIG. 6(1). The playback interface 613 is a video image of the first free viewpoint video at viewpoint A (e.g., 70 degrees).

It should be noted that the example in which the user invokes the viewpoint ruler 612 by manipulating the viewpoint control 608 is used here for illustrative purposes. In some other examples, the mobile phone may alternatively display the viewpoint ruler 612 directly on the free viewpoint video playback interface 601, invoke the viewpoint ruler 612 in other ways, etc. The interface for displaying the viewpoint ruler 612 is not limited to the embodiments of this application.

In some examples, after not receiving any user-performed operations on the viewpoint ruler for preset duration 1, the mobile phone may hide the viewpoint ruler, for example, as shown in the playback interface 614 in FIG. 6(2). In some other examples, when the mobile phone detects a user tapping the display in the playback interface 614, it displays the playback interface 615 shown in FIG. 6(3), and the above-mentioned functional controls, such as the playback progress bar 604, playback control 609, next episode control 610, danmaku enable/disable control 605, danmaku comment input box 606, danmaku comment submit control 607, and viewpoint control 608, are redisplayed in the playback interface 615.

S401: Terminal 100 receives an operation by a user to input the content of a first barrage comment and send the first barrage comment, and terminal 100 generates data for the first barrage comment, where the data for the first barrage comment includes the content of the first barrage comment, the time when the first barrage comment appeared, and the viewpoint of the first barrage comment.

For example, a user may input and send a danmaku comment through the video playback interface 601 shown in FIG. 5(1) or the video playback interface 615 shown in FIG. 6(3). Here, an example in which a user inputs a first danmaku comment into the video playback interface 615 shown in FIG. 6(3) and sends the first danmaku comment will be used for explanation purposes.

As shown in FIG. 6(3), when the mobile phone plays back the first free viewpoint video from viewpoint A, the mobile phone detects that the user has entered the content of the first danmaku comment into the danmaku comment input box 606 and operated the danmaku comment transmission control 607. The mobile phone generates data for the first danmaku comment. In an embodiment of the present application, the data for the first danmaku comment generated by the mobile phone includes the content of the first danmaku comment (e.g., "Amazing"), the time when the first danmaku comment appeared (e.g., 20:10), and the viewpoint of the first danmaku comment (viewpoint A, e.g., 70 degrees). Furthermore, the data for the first danmaku comment further includes one or more of an identifier for the first free viewpoint video, an identifier for the first danmaku comment, a duration ΔT of the first danmaku comment, a format of the first danmaku comment, a transmission position of the first danmaku comment ( the playback point at which the first danmaku comment is transmitted), etc. Table 1 below shows an example of a data format used for the data for the first danmaku comment.

The value of the movie identifier field is the identifier of the first free viewpoint video, for example, Movie 1. The value of the danmaku comment identifier field is the identifier of the first danmaku comment, for example, Danmaku Comment 1. The value of the danmaku comment content field is the content of the first danmaku comment entered by the user, for example, "Amazing."

The value of the appearance time field is the playback time at which the display begins to display the first danmaku comment, for example, playback time T1. In some embodiments, playback time T1 is equal to the transmission time of the first danmaku comment. For example, if a user of terminal 100 transmits a first danmaku comment at 20:10 in the free viewpoint video, the appearance time of the first danmaku comment, i.e., playback time T1, may be 20:10. In some other embodiments, it may be understood that the appearance time of the first danmaku comment is different from the transmission time of the first danmaku comment. For example, in some scenarios, the duration of a highlight in a video is long, or the user is accustomed to transmitting a danmaku comment at the start of a highlight. In this case, the transmission time of the first danmaku comment is determined as playback time T1, reminding other users to pay attention to the highlights in the video. In some other embodiments, playback time T1 is equal to the transmission time of the first danmaku comment minus preset time offset value 1. Preset time offset value 1 may be predefined or set by the user. This is not limited to the embodiments of this application. The setting and calculation methods of other preset time offset values in this specification (e.g., preset time offset value 2) are similar to those of preset time offset value 1 and may be the same as or different from those of preset time offset value 1. Details will not be described below. For example, if the sending time of the first danmaku comment is 20:10 and preset time offset value 1 is 5 seconds, the appearance time of the first danmaku comment, i.e., the playback time T1, is 20:05. In some other scenarios, it may be understood that the duration of a highlight in a video is short, or that a user is accustomed to sending a danmaku comment at the end of a highlight. In this case, the sending time of the first danmaku comment minus preset time offset value 1 is determined as the playback time T1, thereby reminding other users in advance to pay attention to the highlights in the video. In some other embodiments, the playback time T1 is equal to the sending time of the first danmaku comment plus preset time offset value 2. For example, if the sending time of the first barrage comment is 20:10 and the preset time offset value 2 is 3 seconds, the appearance time of the first barrage comment, i.e., the playback time T1, is 20:08. The preset time offset value 1 and the preset time offset value 2 may be the same or different. In some other scenarios, it may be understood that users are accustomed to holding discussions, etc., before highlights. In this case, the sending time of the first barrage comment plus the preset time offset value 2 is determined as the playback time T1, reminding other users to pay timely attention to the highlights in the video.

The value of the Duration field is the duration ΔT (e.g., 15 seconds) for which the first danmaku comment is continuously displayed on the display. In some examples, the first danmaku comment begins to appear from the rightmost side of the terminal display and then moves from the rightmost side to the leftmost side of the display until it disappears from the display. The entire process from when the first danmaku comment appears from the rightmost side of the display to when it disappears from the leftmost side of the display takes ΔT, e.g., 15 seconds. The duration ΔT may be a preset value or may be set by the user. For example, the user may set a danmaku comment mode, with different danmaku comment modes corresponding to different durations. The values of the Font or Color field, etc., are set by the user when the user enters the content of the first danmaku comment, or are set by default on the terminal 100.

The value of the viewpoint field is the viewpoint corresponding to the video image currently being displayed by terminal 100 when the first danmaku comment was sent, i.e., viewpoint A. In other words, a correspondence is established between the first danmaku comment and viewpoint A.

S402: The terminal 100 transmits the data of the first barrage comment to the barrage server 600.

In some embodiments, after receiving the data of the first barrage comment sent by terminal 100, the barrage comment server reviews the data of the first barrage comment, for example, by reviewing the content and format. After the review is successful, the barrage comment server stores the data of the first barrage comment.

In this way, the danmaku comment server 600 stores the danmaku comment data for the first free viewpoint video transmitted from each viewpoint and uploaded by each user.

It can be understood that the danmaku comment server 600 stores a large amount of danmaku comment data for free viewpoint video at each viewpoint. Subsequently, other terminals (including terminal 100) may request danmaku comment data for a specific free viewpoint video at a specific playback point from the danmaku comment server 600. Furthermore, the danmaku comment data returned by the danmaku comment server 600 includes the content of the danmaku comment, the time at which the danmaku comment appeared, the viewpoint of the danmaku comment, etc. Subsequently, when displaying the danmaku comments, the other terminals may determine different display methods for danmaku comments at different viewpoints.

2. Solution for displaying bullet comments

For example, the method for displaying danmaku comments provided in the embodiments of this application will be described using an example in which a terminal 100 (here, the terminal 100 may be the same as or different from the terminal 100 that transmits the first danmaku comment) plays a first free viewpoint video.

Figure 7 is a schematic flowchart of a method for displaying barrage comments according to an embodiment of the present application. The method includes the following steps:

S701: The terminal 100 requests a video file of the first free viewpoint video from viewpoint B at playback time T2 from the media server 300. In response, the media server 300 returns a video file of the first free viewpoint video from viewpoint B at playback time T2 to the terminal 100.

For example, the user may select to play video from any viewpoint by using the free viewpoint video client/application of the terminal 100. Based on the playback progress of the first free viewpoint video, the terminal 100 requests a video file for the corresponding playback time from the media server 300. For example, the terminal 100 requests a video file of the first free viewpoint video from viewpoint B at playback time T2 from the media server 300.

S702: The terminal 100 further requests danmaku comment data for the first free viewpoint video at playback time T2 from the danmaku comment server 600. In response, the danmaku comment server 600 returns danmaku comment data for the first free viewpoint video at playback time T2 to the terminal 100.

When the terminal 100 is playing back a first free viewpoint video and the danmaku comment function is enabled, the terminal 100 must also request danmaku comment data for the corresponding playback time point from the danmaku comment server 600, based on the playback progress of the first free viewpoint video. For example, the terminal 100 requests danmaku comment data for the first free viewpoint video at playback time point T2 from the media server 300. In this case, it should be noted that the danmaku comment data for the first free viewpoint video at playback time point T2 returned to the terminal 100 by the danmaku comment server 600 includes danmaku comments for each viewpoint, for example, a danmaku comment for viewpoint B, a danmaku comment for viewpoint A, and danmaku comments for other viewpoints.

It should be noted that step S702 may be performed after, before, or simultaneously with step S701, which is not limited in the embodiment of this application.

S703: The terminal 100 calculates the offset angle between viewpoint B and the viewpoint of each danmaku comment.

In some embodiments, the offset angle includes an offset direction and/or an offset angle value. The offset direction includes a left offset direction and a right offset direction. If the offset direction is a left offset direction, this means that viewpoint B needs to be switched to the left when it is switched to the viewpoint of the danmaku comment. If the offset direction is a right offset direction, this means that viewpoint B needs to be switched to the right when it is switched to the viewpoint of the danmaku comment. The offset angle value is the absolute value of the difference between viewpoint B and the viewpoint of the danmaku comment. In other embodiments, the offset angle = viewpoint of the danmaku comment - viewpoint B. When the offset angle is a positive number, this indicates that the danmaku comment is a right-offset danmaku comment. When the offset angle is a negative number, this indicates that the danmaku comment is a left-offset danmaku comment. The definition and calculation of the offset angle are not limited to the embodiments of this application. This specification will be described by using an example where the offset angle = viewpoint of the danmaku comment - viewpoint B. Details will not be provided below.

For example, in the playback interface 613 shown in Figure 6(1), the current playback viewpoint (viewpoint B) is 70 degrees, and the viewpoint of danmaku comment 1 is 60 degrees. In this case, the offset angle between viewpoint B and the viewpoint of danmaku comment 1 is a left offset of 10 degrees. When the viewpoint of danmaku comment 2 is 100 degrees, the offset angle between viewpoint B and the viewpoint of danmaku comment 2 is a right offset of 30 degrees.

In another example, as described above, the free viewpoint video is video captured by a camera around the target object, and thus the free viewpoint video includes images of the target object from multiple viewpoints. As shown in FIG. 8, the current viewer angle (the playback viewpoint of the free viewpoint video, i.e., also referred to as viewpoint B) is 0 degrees. When the transmission viewpoint of danmaku comment 1 (also referred to as the viewpoint of danmaku comment 1) is -30 degrees, the offset angle between viewpoint B and the viewpoint of danmaku comment 1 is a left offset of 30 degrees. When the transmission viewpoint of danmaku comment 2 (also referred to as the viewpoint of danmaku comment 2) is +25 degrees, the offset angle between viewpoint B and the viewpoint of danmaku comment 2 is a right offset of 25 degrees. When the transmission viewpoint of danmaku comment 3 (also referred to as the viewpoint of danmaku comment 3) is -100 degrees, the offset angle between viewpoint B and the viewpoint of danmaku comment 3 is a left offset of 100 degrees. When the sending viewpoint of danmaku comment 4 (also called the viewpoint of danmaku comment 4) is +90 degrees, the offset angle between viewpoint B and the viewpoint of danmaku comment 4 is a right offset of 90 degrees.

It should be noted that FIG. 9 shows free-viewpoint footage captured in a circle ( 360 degrees) around the target object. In other words, the free-viewpoint footage is a 360-degree surrounding footage. Obviously, in some other examples, the free-viewpoint footage may not be captured in a circle around the target object. For example, the free-viewpoint footage may be a 180-degree surrounding footage, a 130-degree surrounding footage, or a 270-degree surrounding footage.

S704: The terminal 100 displays a video image of the first free viewpoint video at viewpoint B, and presents each danmaku comment based on the offset angle between viewpoint B and the viewpoint of each danmaku comment.

It can be understood that when the offset angle between viewpoint B and the viewpoint of the barrage comment is different, the barrage comment is presented in a different manner. For example, differences in presentation manner include different tilting methods (including tilt direction and/or tilt degree) of the text direction of the barrage comment, different rotation methods (including rotation direction and/or rotation degree) of the text of the barrage comment, different fonts, different three-dimensional effects, different movement methods, etc.

In this embodiment, the danmaku comments of the first free viewpoint video at the playback viewpoint (viewpoint B) are presented in a different manner from the danmaku comments at other viewpoints. In this example embodiment, the following policy may be used to present different danmaku comments: When |Viewpoint of danmaku comment - Viewpoint B|≦Threshold A (e.g., Threshold A is 0 degrees or 5 degrees), the danmaku comments are displayed in a first presentation manner. When |Viewpoint of danmaku comment - Viewpoint B|>Threshold A, the danmaku comments are displayed in a second presentation manner, which is different from the first presentation manner. For example, the first presentation manner is such that the text of the danmaku comments is displayed normally (e.g., the text of the danmaku comments is not tilted or rotated, the font is not transformed, and a flat effect is used). In this case, the text direction of the danmaku comments is parallel to a first direction. For example, the first direction is a direction parallel to the upper bezel/lower bezel of the screen of terminal 100. The second presentation method performs display using one or more of tilt, rotation, font deformation, and three-dimensional effect, i.e., the text direction of the barrage comments is a direction obtained by rotating a first direction (e.g., parallel to the upper bezel/lower bezel of the screen of terminal 100) clockwise by a specific angle. In this case, a specific included angle is formed between the text direction of the barrage comments and a bezel on one side of the screen of terminal 100. In another example, in the first presentation method, the barrage comments move in a first direction (e.g., left), and in the second presentation method, the barrage comments move in a second direction (e.g., right). The first direction is different from the second direction.

9(1), the viewpoint of danmaku comment 902 (the viewpoint is 60 degrees) is the same as the playback viewpoint of the first free viewpoint video (viewpoint B, also 60 degrees), and the text of danmaku comment 902 is displayed normally without tilt. The viewpoint of danmaku comment 903 (the viewpoint is 90 degrees), the viewpoint of danmaku comment 904 (the viewpoint is 30 degrees), the viewpoint of danmaku comment 905 (the viewpoint is 120 degrees), and the viewpoint of danmaku comment 906 (the viewpoint is 10 degrees) are all different from the playback viewpoint of the first free viewpoint video, and the text of the four danmaku comments are all displayed in a tilted manner. It may be understood that in some other examples, when displaying danmaku comments, terminal 100 may or may not display viewpoint information of the danmaku comments.

It can be seen that the user can distinguish the danmaku comments of the first free viewpoint video at the playback viewpoint (viewpoint B) from the danmaku comments at other viewpoints based on the presentation format of the danmaku comments. In this way, the user will not be confused or suspicious when the video at the playback viewpoint does not match the content of the danmaku comments at other viewpoints. This improves the viewing experience of the first free viewpoint video.

In other embodiments, when the offset direction between viewpoint B and the viewpoint of the danmaku comment is different, the danmaku comment is presented in a different manner. In one example, when the offset angle between viewpoint B and the viewpoint of the danmaku comment is a left offset, the danmaku comment is displayed in a third presentation manner. When the offset angle between viewpoint B and the viewpoint of the danmaku comment is a right offset, the danmaku comment is displayed in a fourth presentation manner. The third presentation manner is different from the fourth presentation manner.

For example, an example in which the first direction is parallel to the top bezel of the screen of the terminal 100 will be used for explanation. In the third presentation method, the text direction of the danmaku comment is a direction obtained by rotating the first direction counterclockwise by a specific angle. In the fourth presentation method, the text direction of the danmaku comment is a direction obtained by rotating the first direction clockwise by a specific angle. For example, in the first free viewpoint video playback interface 907 shown in FIG. 9(2), the viewpoint of the danmaku comment 902 (the viewpoint is 60 degrees) is the same as the playback viewpoint of the first free viewpoint video (viewpoint B, also 60 degrees), and the text of the danmaku comment 902 is displayed normally without tilt. The offset angle between the viewpoint of the danmaku comment 903 (the viewpoint is 90 degrees) and the playback viewpoint of the first free viewpoint video is a right offset, and the text direction of the danmaku comment 903 is a direction obtained by rotating the first direction clockwise by a specific angle. Similarly, the offset angle between the viewpoint of danmaku comment 905 (the viewpoint is 120 degrees) and the playback viewpoint of the first free viewpoint video is also a right offset, and the text direction of danmaku comment 905 is a direction obtained by rotating the first direction clockwise by a specific angle. The offset angle between the viewpoint of danmaku comment 904 (the viewpoint is 30 degrees) and the playback viewpoint of the first free viewpoint video is a left offset, and the text direction of danmaku comment 904 is a direction obtained by rotating the first direction counterclockwise by a specific angle. Similarly, the offset angle between the viewpoint of danmaku comment 906 (the viewpoint is 10 degrees) and the playback viewpoint of the first free viewpoint video is also a left offset, and the text direction of danmaku comment 906 is a direction obtained by rotating the first direction counterclockwise by a specific angle.

In another example, the third presentation method includes a first mark (e.g., a left arrow) and the fourth presentation method includes a second mark (e.g., a right arrow).

It can be seen that when the offset direction between the viewpoint of a danmaku comment and the playback viewpoint of the first free viewpoint video is different and the presentation method of the danmaku comments is also different, the user may distinguish the offset direction of the viewpoint corresponding to the danmaku comment based on the presentation method of the danmaku comment. In this case, when the user is interested in some danmaku comments (referred to as target danmaku comments), the user may determine the viewpoint switching direction based on the presentation method of the target danmaku comment and switch to the viewpoint corresponding to the target danmaku comment. In other words, different presentation methods of danmaku comments may inform the user of the direction that needs to be switched when the user switches viewpoint, thereby enabling the user to switch viewpoint quickly.

For example, in the playback interface 907 shown in FIG. 9(2), if a user is interested in danmaku comment 903 and wishes to switch to the viewpoint corresponding to danmaku comment 903, the user will know that the viewpoint offset between playback viewpoint B and the viewpoint of danmaku comment 903 is a right offset, based on the presentation method in which danmaku comment 903 is tilted to the right. Thus, the user may invoke the viewpoint ruler and slide to the right on the viewpoint ruler until the viewpoint corresponding to danmaku comment 903 is reached. If a user is interested in danmaku comment 904 and wishes to switch to the viewpoint corresponding to danmaku comment 904, the user will know that the viewpoint offset between playback viewpoint B and the viewpoint of danmaku comment 904 is a left offset, based on the presentation method in which danmaku comment 904 is tilted to the left. Thus, the user may invoke the viewpoint ruler and slide to the left on the viewpoint ruler until the viewpoint corresponding to danmaku comment 904 is reached.

In yet another embodiment, when the offset direction between viewpoint B and the viewpoint of the danmaku comment is the same but the offset angle value is different, the danmaku comment is presented in a different manner. For example, the greater the offset angle between viewpoint B and the viewpoint of the danmaku comment, the greater the degree of tilt present when the danmaku comment is presented, i.e., the greater the tilt angle of the danmaku comment. For example, in the playback interface 908 shown in FIG. 9(3), both danmaku comment 903 and danmaku comment 905 are offset to the right with respect to the playback viewpoint of the first free viewpoint video. When |viewpoint of danmaku comment 903 - viewpoint B|< {viewpoint of danmaku comment 905 - viewpoint B|, then tilt angle a of danmaku comment 903 < tilt angle b of danmaku comment 905. Both danmaku comment 904 and danmaku comment 906 are offset to the right with respect to the playback viewpoint of the first free viewpoint video. When |Viewpoint of danmaku comment 904 - Viewpoint B| < |Viewpoint of danmaku comment 906 - Viewpoint B|, the tilt angle c of danmaku comment 904 < the tilt angle d of danmaku comment 906.

In one example, the tilt angle of the text direction of the barrage comment is directly proportional to the value of the offset angle of the barrage comment. In other words, the tilt angle of the text direction of the barrage comment has a linear increasing relationship based on the value of the offset angle of the barrage comment. In another example, the tilt angle of the text direction of the barrage comment may alternatively have a stepwise increasing relationship based on the value of the offset angle of the barrage comment. For example, the following policies may be used to present different barrage comments:

When |Barrage comment viewpoint - viewpoint B| ≦ threshold B (for example, threshold B is 10 degrees or 15 degrees), the barrage comment will be displayed as normal.

When threshold B < | viewpoint of danmaku comment - viewpoint B | ≦ threshold C (for example, threshold C is 30 degrees or 45 degrees), the text direction of the danmaku comment is the direction obtained by rotating the first direction counterclockwise/clockwise by the first angle.

When threshold C<| viewpoint of danmaku comment - viewpoint B|≦ threshold D (for example, threshold D is 60 degrees), the text direction of the danmaku comment is the direction obtained by rotating the first direction counterclockwise/clockwise by a second angle, where the second angle is greater than the first angle.

...

Table 2 below provides an example of the correspondence between the offset angle of the barrage comment and the tilt angle of the barrage comment.

In conclusion, when the offset angle between viewpoint B and the viewpoint of the danmaku comment is different and the presentation methods of the danmaku comment are different, the user may distinguish the value of the offset angle between viewpoint B and the viewpoint of the danmaku comment based on the presentation method of the danmaku comment. In this case, when the user is interested in some danmaku comments (called target danmaku comments), the user may determine the viewpoint switching amplitude (e.g., the finger sliding distance, or the duration or number of presses of a button on a remote control) based on the presentation method of the target danmaku comment and switch to the viewpoint corresponding to the target danmaku comment. In other words, different presentation methods of the danmaku comments may notify the user of the viewpoint switching amplitude (e.g., the finger sliding distance, or the duration or number of presses of a button on a remote control), thereby allowing the user to switch viewpoints quickly.

It can be understood that in some other embodiments, danmaku comments in the playback interface in all viewpoints, such as danmaku comment 902, danmaku comment 903, danmaku comment 904, danmaku comment 905, and danmaku comment 906 in playback interface 908, may be presented in the same manner, e.g., all are displayed normally. It can be understood that a user may distinguish the danmaku comments of the first free viewpoint video in the playback viewpoint (viewpoint B) from the danmaku comments in the other viewpoints based on the viewpoint conveyed in each danmaku comment. In this way, the user will not be confused or suspicious when the video in the playback viewpoint does not match the content of the danmaku comments in the other viewpoints. This improves the viewing experience of the first free viewpoint video.

As described above, the user may randomly switch the playback viewpoint of the first free viewpoint video. In some embodiments, in the process of switching the viewpoint of the first free viewpoint video, terminal 100 leaves the content and presentation method of the danmaku comments unchanged. After the viewpoint of the first free viewpoint video is switched, terminal 100 reacquires the danmaku comment data acquired after the viewpoint is switched, and determines the display content of the danmaku comments and the specific presentation method for each danmaku comment based on the reacquired danmaku comment data.

For example, FIG. 10A shows a method for displaying barrage comments in a viewpoint switching process according to an embodiment of the present application. The method includes the following steps:

S1001: At time t0, terminal 100 detects that the user has begun to perform a viewpoint switching operation, which indicates switching the viewpoint of the first free viewpoint video.

It should be noted that before step S1001, the terminal 100 may display a video image of the first free viewpoint video from any viewpoint and the corresponding danmaku comments by using the method in steps S701 to S704. In one example, the terminal 100 displays a video image of the first free viewpoint video from viewpoint B at playback time T2 and the danmaku comments corresponding to playback time T2.

The viewpoint switching operation may be, for example, a slide operation performed by the user on the screen of the terminal 100, an operation of tilting the terminal 100 by the user, or an operation of operating a button on a remote control by the user.

For example, if the terminal 100 detects that the user's finger has remained on the viewpoint ruler for longer than a preset duration of 2 (e.g., 3 seconds), the terminal 100 determines that the user has begun to perform a viewpoint change operation and records the current time as time t0. In another example, when the terminal 100 is configured with a device such as a gravity acceleration or IMU device, the orientation of the terminal 100 can be detected. When it is detected that the tilt angle of the terminal 100 with respect to the horizontal plane is greater than or equal to threshold 1, it is determined that the user has begun to perform a viewpoint change operation and records the current time as time t0. In another example, when the terminal 100 receives a signal corresponding to a specific button transmitted by a remote control, it determines that the user has begun to perform a viewpoint change operation and records the current time as time t0.

S1002: From time t0 to time t1, terminal 100 leaves the presentation method of each danmaku comment unchanged.

S1003: At time t1, terminal 100 detects that the viewpoint switching operation has been completed.

For example, after time t0, the terminal continuously detects that the user's finger has slid across the screen. When it detects that the user's finger has been removed from the screen, the terminal determines that the user has completed the viewpoint change operation and records the current time as time t1. In another example, after time t0, the terminal continuously detects the tilt angle of the terminal 100 relative to the horizontal plane. When it detects that the tilt angle of the terminal 100 relative to the horizontal angle is less than threshold 1, it determines that the user has completed the viewpoint change operation and records the current time as time t1. In another example, after time t0, the terminal continuously detects a signal corresponding to a specific button transmitted by the remote control. When a signal corresponding to the specific button is not detected for a preset duration of 3 (e.g., 5 seconds) after the last detection of a signal corresponding to the specific button, it determines that the user has completed the viewpoint change operation and records the current time as time t1.

For example, in the process of a user performing a viewpoint switching operation, i.e., between time t0 and time t1 (excluding time t1), terminal 100 may leave the presentation method of each danmaku comment unchanged, for example, the tilt direction and tilt angle of each danmaku comment unchanged. In other words, regardless of whether the playback viewpoint of the first free viewpoint video changes between time t0 and time t1, terminal 100 always maintains the presentation method of the danmaku comments displayed by terminal 100 at time t0.

It should be noted that in some examples, the playback viewpoint of the video image of the first free viewpoint video changes between time t0 and time t1. For example, terminal 100 may alternatively play back the video image of the first free viewpoint video at another viewpoint (e.g., one or more viewpoints between viewpoint B and viewpoint C) based on a viewpoint switching operation performed by a user (e.g., a sliding distance of a finger, or a number of times or duration of pressing a button on a remote control). In other words, from time t0 to time t1, the playback viewpoint of the first free viewpoint video is switched from viewpoint B to viewpoint C, and the playback viewpoint of the first free viewpoint video covers all or some of the viewpoints from viewpoint B to viewpoint C. For example, when the viewpoint of the first free viewpoint video is switched from 60 degrees to 90 degrees, the viewpoint of the first free viewpoint video covers all angles between 60 degrees and 90 degrees, or covers some angles between 60 degrees and 90 degrees , for example , angles including 60 degrees, 70 degrees, 80 degrees, and 90 degrees.

In some other examples, the playback viewpoint of the video image of the first free viewpoint video does not change between time t0 and time t1, i.e., the terminal 100 continues to play back the video image of the first free viewpoint video at angle B.

It should be further noted that the terminal 100 may determine the danmaku comments to be displayed and the display positions of these danmaku comments based on the playback progress (playback time point) of the first free viewpoint video. In some examples, in the process of a viewpoint switching operation performed by a user, i.e., between time t0 and time t1 (excluding time t1), the terminal 100 continuously plays back the first free viewpoint video. In this case, the playback time point of the video images of the first free viewpoint video changes. In this case, the terminal 100 determines in real time the danmaku comments to be displayed by the terminal 100 and the display positions of the danmaku comments based on the change in the playback time point. However, the presentation manner of each danmaku comment (e.g., the tilt direction and tilt angle of the danmaku comment) remains unchanged. In some other examples, in the process of a viewpoint switching operation performed by a user, i.e., between time t0 and time t1 (excluding time t1), the terminal 100 pauses the playback of the first free viewpoint video. During this period, the playback time point of the video images of the first free viewpoint video does not change. In this case, both the barrage comments displayed by the terminal 100 and the display positions of the barrage comments remain unchanged.

S1004: After time t1, terminal 100 requests a video file of the first free viewpoint video from viewpoint C at playback time T3 from the media server.

In response, the media server returns to the terminal 100 a video file of the first free viewpoint video from viewpoint C at playback time T3.

In some embodiments, after detecting that the user has completed the viewpoint switching operation, the terminal 100 sends a request 1 to the media server, where the request 1 is used to request a video file corresponding to the viewpoint acquired after the switch. In one example, the request 1 includes an identifier of the first free viewpoint video, viewpoint C, and playback time T3. The playback time T3 is related to the playback time T2. For example, the playback time T3 = playback time T2, the playback time T3 = playback time T2 - preset time offset value 1, or the playback time T3 = playback time T2 + preset time offset value 2. The preset time offset value 1 and the preset time offset value 2 may be the same or different.

S1005: After time t1, terminal 100 further requests danmaku comment data corresponding to playback time T3 of the first free viewpoint video from the danmaku comment server.

In response, the barrage comment server returns barrage comment data corresponding to playback time point T3 of the first free viewpoint video to terminal 100.

In some embodiments, after detecting that the user has completed the viewpoint switching operation, the terminal 100 sends a request 2 to the media server, where the request 2 is used to request barrage comment data corresponding to the playback time T3 of the first free viewpoint video. In one example, the request 2 includes an identifier of the first free viewpoint video and the playback time T3. The playback time T3 is related to the playback time T2. For example, the playback time T3 = the playback time T2, the playback time T3 = the playback time T2 - the preset time offset value 1, or the playback time T3 = the playback time T2 + the preset time offset value 2. In some other embodiments, the terminal 100 continuously plays the first free viewpoint video in the process of the user performing the viewpoint switching operation. In this case, the playback time T3 = the playback time T2 + (time t1 - time t0).

It should be noted that step S1005 may be performed after, before, or simultaneously with step S1004.

It should be further noted that this step is optional. In some other embodiments, playback time point T3 = playback time point T2, or playback time point T3 = playback time point T2 - preset time offset value 1. Terminal 100 stores the danmaku comment data corresponding to playback time point T3. In this case, terminal 100 does not need to request the danmaku comment data at playback time point T3 again from the danmaku comment server. As playback of the first free viewpoint video progresses, terminal 100 requests danmaku comment data that exists after playback time point T3 or playback time point T2 from the danmaku comment server.

S1006: The terminal 100 recalculates the offset angle between viewpoint C and the viewpoint of each danmaku comment after the viewpoint is switched.

It should be noted that before the viewpoint is switched, the terminal 100 calculates the offset angle between viewpoint B and the viewpoint of each danmaku comment. After the viewpoint is switched, the terminal 100 calculates the offset angle between viewpoint C and the viewpoint of each danmaku comment. For a method for calculating the offset angle, see the related content in step S703.

S1007: The terminal 100 displays a video image of the first free viewpoint video from viewpoint C at playback time T3, and displays each danmaku comment based on the offset angle between viewpoint C and the viewpoint of each danmaku comment.

For this step, please refer to the relevant content in step S704. Details will not be explained again here.

For example, as shown in FIG. 10B(1), the terminal 100 plays a playback interface 1001 of a first free viewpoint video from a viewpoint at 60 degrees (viewpoint B). The playback interface 1001 further displays danmaku comments from different viewpoints, such as danmaku comment 1002, danmaku comment 1003, danmaku comment 1004, danmaku comment 1005, and danmaku comment 1006. At time point t0, the terminal 100 detects that the user has dragged the viewpoint ruler in the playback interface 1001 (the playback time point in this case is 20:05, i.e., playback time point T2), and at time point t1, the terminal 100 detects that the user's finger has been lifted from the display of the terminal 100. In response to information indicating that the user has performed a viewpoint switching operation (an operation in which the user drags the viewpoint ruler) at time point t1, the terminal 100 switches the playback viewpoint of the first free viewpoint video and displays a playback interface 1007 shown in FIG. 10B(2). It may be noted that in this case, the playback viewpoint of the first free viewpoint video is switched to a 70-degree viewpoint (viewpoint C). Furthermore, the presentation manner of each danmaku comment in the playback interface 1007 is changed. For example, compared with that of each danmaku comment in the playback interface 1001, the presentation manner of the danmaku comment 1002 changes from a normal display to a tilted display. The tilt angles of the danmaku comments 1003 and 1005 decrease (a changes to a', and b changes to b'). The tilt angles of the danmaku comments 1004 and 1006 increase (c changes to c', and d changes to d'). In some examples, the terminal 100 pauses the playback of the first free viewpoint video between time t0 and time t1 (excluding time t1), i.e., the video image displayed by the terminal 100 remains unchanged. In this case, the playback time point of the first free viewpoint video in the playback interface 1007 is 20:05 (playback time point T3 is the same as playback time point T2).

In some other examples, the terminal 100 continuously plays back the first free viewpoint video between time t0 and time t1 (excluding time t1). In this case, in response to information indicating that the user has performed a viewpoint switching operation (an operation in which the user drags the viewpoint ruler) at time t1, the terminal 100 switches the playback viewpoint of the first free viewpoint video and displays the playback interface 1008 shown in FIG. 10B(3). The playback time of the first free viewpoint video in the playback interface 1008 is 20:10 (specifically, playback time T3 = playback time T2 + (time t1 - time t0)).

In conclusion, in this embodiment, in the process of the user switching the viewpoint, i.e., between time t0 and time t1, the terminal 100 does not update the specific presentation format of the barrage comments. This helps the user compare the change in the presentation format of the barrage comments before and after the viewpoint is switched, reduces the processing load of the terminal 100, and reduces the power consumption of the terminal 100.

In some other embodiments, in the process of switching the viewpoint of the first free viewpoint video, the presentation format of each danmaku comment displayed by terminal 100 changes in real time based on the current playback viewpoint of the first free viewpoint video. In other words, step S1002 in FIG. 10A may be replaced by step S1101 in FIG. 11A.

FIG. 11A shows another method for displaying barrage comments in a viewpoint switching process according to an embodiment of the present application. The method includes steps S1001, S1101, and S1003 to S1007.

For step S1001 and steps S1003 to S1007, please refer to the explanation of the corresponding steps in Figure 10A. Details will not be repeated here.

S1101: From time t0 to time t1, terminal 100 calculates in real time the viewpoint offset between the viewpoint of each danmaku comment and the playback viewpoint of the first free viewpoint video, and displays each danmaku comment based on the viewpoint offset calculated in real time.

In some embodiments, the playback viewpoint of the video image of the first free viewpoint video changes between time t0 and time t1. For example, terminal 100 may alternatively play back the video image of the first free viewpoint video at another viewpoint (e.g., one or more viewpoints between viewpoint B and viewpoint C) based on a viewpoint switching operation performed by a user (e.g., a sliding distance of a finger, or a number of times or duration of pressing a button on a remote control). In other words, from time t0 to time t1, the playback viewpoint of the first free viewpoint video is switched from viewpoint B to viewpoint C, and the playback viewpoint of the first free viewpoint video covers all or some of the viewpoints from viewpoint B to viewpoint C. For example, when the viewpoint of the first free viewpoint video is switched from 60 degrees to 90 degrees, the viewpoint of the first free viewpoint video covers all angles between 60 degrees and 90 degrees, or covers some angles between 60 degrees and 90 degrees, for example , angles including 60 degrees, 70 degrees, 80 degrees, and 90 degrees.

From time t0 to time t1, terminal 100 determines that as the playback viewpoint of the first free viewpoint video changes, the presentation manner of each danmaku comment changes accordingly. For example , when the playback viewpoint of the first free viewpoint video changes, terminal 100 calculates in real time an angle offset between the viewpoint of each danmaku comment and the changed playback viewpoint of the first free viewpoint video, and determines the presentation manner of each danmaku comment based on the angle offset calculated in real time.

It should be further noted that the terminal 100 may determine the danmaku comments to be displayed and the display positions of these danmaku comments based on the playback progress (playback time point) of the first free viewpoint video. In some examples, in the process of a user performing a viewpoint switching operation, i.e., between time point t0 and time point t1 (excluding time point t1), the terminal 100 continuously plays back the first free viewpoint video. In this case, the playback time point of the video image of the first free viewpoint video changes. In this case, the terminal 100 determines in real time the danmaku comments to be displayed by the terminal 100 and the display positions of the danmaku comments based on the change in the playback time point. In other words, the content of the danmaku comments displayed by the terminal 100 changes as the playback progresses of the first free viewpoint video. Optionally, in this scenario, between time point t0 and time point t1 (excluding time point t1), the terminal 100 may further request danmaku comment data corresponding to each playback time point in real time from the danmaku comment server based on the playback progress of the first free viewpoint video.

In some other examples, in the process of the user performing a viewpoint switching operation, i.e., between time t0 and time t1 (excluding time t1), terminal 100 pauses the playback of the first free viewpoint video. During this period, the playback time of the video image of the first free viewpoint video does not change. In this case, both the barrage comments displayed by terminal 100 and the display position of the barrage comments remain unchanged.

11B(1), terminal 100 plays a playback interface 1101 of a first free viewpoint video at a viewpoint of 60 degrees (viewpoint B). Playback interface 1101 further displays danmaku comments at different viewpoints, such as danmaku comment 1102, danmaku comment 1103, danmaku comment 1104, danmaku comment 1105, and danmaku comment 1106. At time t0, terminal 100 detects that the user has dragged the viewpoint ruler in playback interface 1101 (the playback time in this case is 20:05, i.e., playback time T2), and at time t1, terminal 100 detects that the user's finger has been lifted from the display of terminal 100.

When the terminal 100 detects that the user has performed a viewpoint switching operation between time t0 and time t1 (excluding time t0 and time t1), the playback viewpoint of the first free viewpoint video displayed by the terminal 100 continuously changes, for example, from 60 degrees to 65 degrees, as shown in the playback interface 1107 shown in FIG. 11B(2). It may be noted that the presentation method of each danmaku comment in the playback interface 1107 is changed. For example, compared with that of each danmaku comment in the playback interface 1101, the presentation method of the danmaku comment 1102 changes from a normal display to a tilted display (tilt angle is e'). The tilt angles of the danmaku comments 1003 and 1005 decrease (a changes to a'' and b changes to b''). The tilt angles of the danmaku comments 1004 and 1006 increase (c changes to c'' and d changes to d'').

At time t1, the terminal 100 further switches the playback viewpoint of the first free viewpoint video, and displays a playback interface 1108 shown in FIG. 11B(3). In this case, it may be noted that the playback viewpoint of the first free viewpoint video is switched to a 70-degree viewpoint (viewpoint C). Furthermore , the presentation manner of each danmaku comment in the playback interface 1108 is again changed from that of each danmaku comment in the playback interface 1107. For example, the tilt angle of the danmaku comment 1102 further increases (e' changes to e) , the tilt angles of the danmaku comments 1003 and 1005 decrease (a'' changes to a', and b'' changes to b') , and the tilt angles of the danmaku comments 1004 and 1006 increase (c'' changes to c', and d'' changes to d').

In conclusion, in this embodiment, in the process of switching viewpoints by the user, the terminal 100 adjusts the presentation method of the danmaku comments in real time based on changes in the playback viewpoint of the first free viewpoint video. This helps the user know the distribution status of the danmaku comments for the first free viewpoint video at each viewpoint, and ultimately indicates to the user to select the switched playback viewpoint.

In some other embodiments, the execution of the viewpoint switching operation may be paused during the process of switching the viewpoint of the first free viewpoint video. In this case, when the viewpoint switching operation is paused, terminal 100 may re-request the danmaku comment data corresponding to the current playback point from the danmaku comment server, and determine the display content of the danmaku comments and the specific presentation method for each danmaku comment based on the re-acquired danmaku comment data.

For example, Figures 12A and 12B are schematic flowcharts of another method for displaying barrage comments in a viewpoint switching process according to an embodiment of the present application. The method includes the following steps:

S1201: At time t0, terminal 100 detects that the user has started to perform a viewpoint switching operation, which indicates switching the viewpoint of the first free viewpoint video.

In one example, the terminal 100 displays the video of the first free viewpoint video from viewpoint B at playback time T2 and the barrage comment data of the first free viewpoint video at playback time T2.

The viewpoint switching operation may be, for example, a sliding operation performed by the user on the screen of the terminal 100, an operation of tilting the terminal 100 by the user, or an operation of pressing a button on a remote control by the user. For example, if the terminal 100 detects that the user's finger remains on the viewpoint ruler for longer than a preset duration of 2 (e.g., 3 seconds), the terminal 100 determines that the user has begun to perform a viewpoint switching operation and records the current time as time t0.

For other details about this step, please refer to the relevant content in step S1001 in Figure 10A. Details will not be explained again here.

S1202: From time t0 to time t2, terminal 100 leaves the presentation method of the displayed barrage comments unchanged.

Time t2 is a point between time t0 and time t1.

For other details about this step, please refer to the relevant details of step S1002 in Figure 10A. Details will not be explained again here.

In some other embodiments, between time t0 and time t2, terminal 100 may alternatively adjust the presentation method of each danmaku comment in real time based on changes in the playback viewpoint of the first free viewpoint video. For related content, refer to the description of step S1101 in FIG. 11A. Details will not be described again here.

S1203: At time t2, terminal 100 detects that the user has paused the execution of the viewpoint switching operation.

For example, after time t0, the terminal continuously detects that the user's finger has slid across the screen. When it detects that the duration during which the user's finger does not slide or leave the screen exceeds preset duration 4 (e.g., 3 seconds), the terminal determines that the user has paused the execution of the viewpoint switching operation and records the current time as time t2.

S1204: After time t2, terminal 100 requests a video file of the first free viewpoint video from viewpoint D at playback time T4 from the media server.

In other words, after detecting that the user has paused the execution of the viewpoint switching operation, the terminal 100 determines the current playback viewpoint of the first free viewpoint video, for example, viewpoint D. Playback time point T4 is relative to playback time point T2. In some examples, when the terminal 100 pauses playback of the first free viewpoint video from time point t0 to time point t2, playback time point T4 = playback time point T2, or playback time point T4 = playback time point T2 - preset time offset value 1. For example, if playback time point T2 is 20:10, playback time point T4 is 20:10 or 20:05. In some other examples, when the terminal 100 continuously plays back the first free viewpoint video from time point t0 to time point T2, playback time point T4 = playback time point T2 + (time point t2 - time point t0). In another example, if playback time point T2 is 20:10, playback time point T4 is 20:15.

S1205: After time t2, terminal 100 further requests danmaku comment data corresponding to playback time T4 of the first free viewpoint video from the danmaku comment server.

S1206: The terminal 100 recalculates the offset angle between viewpoint D and the viewpoint of each danmaku comment after the viewpoint is switched.

S1207: The terminal 100 displays the video image of the first free viewpoint video from viewpoint D at playback time point T4, and displays each danmaku comment based on the offset angle between viewpoint D and the viewpoint of each danmaku comment.

For other details of steps S1204 to S1207, please refer to the relevant details of steps S1004 to S1007 in Figure 10A. Details will not be explained again here.

S1208: At time t1, terminal 100 detects that the viewpoint switching operation has been completed.

For example, after time t2, the device continuously detects that the user's finger is sliding across the screen. When it detects that the user's finger has been removed from the screen, the device determines that the user has completed the viewpoint switching operation and records the current time as time t1.

S1209: After time t1, terminal 100 requests a video file of the first free viewpoint video from viewpoint C at playback time T3 from the media server.

S1210: After time t1, terminal 100 further requests danmaku comment data corresponding to playback time T3 of the first free viewpoint video from the danmaku comment server.

In some examples, playback time T3 is relative to playback time T4. For example, playback time T3 = playback time T4, playback time T3 = playback time T4 - preset time offset value 1, or playback time T3 = playback time T4 + preset time offset value 2. In other examples, playback time T3 = playback time T4 + (time t1 - time t0).

S1211: The terminal 100 recalculates the offset angle between viewpoint C and the viewpoint of each danmaku comment after the viewpoint is switched.

S1212: The terminal 100 displays a video image of the first free viewpoint video from viewpoint C at playback time T3, and displays each danmaku comment based on the offset angle between viewpoint C and the viewpoint of each danmaku comment.

It should be noted that the embodiments shown in Figures 12A and 12B are described using an example in which the viewpoint switching operation is paused only once. It can be understood that the technical solutions provided in the embodiments of this application are also applicable to cases in which the viewpoint switching operation is paused more than once.

For other details of steps S1208 to S1212, please refer to the relevant details of steps S1003 to S1007 in Figure 10A. Details will not be explained again here.

In this embodiment, during the process of switching viewpoints by the user, after the user pauses the viewpoint switching operation, the terminal 100 may adjust the display method of the danmaku comments in real time based on the current playback viewpoint of the first free viewpoint video. This helps the user know the distribution status of the danmaku comments for the first free viewpoint video at the current playback viewpoint, and ultimately indicates to the user to select the switched playback viewpoint.

In other embodiments, the terminal 100 may further predefine the visible viewpoint range of the danmaku comments, or the terminal 100 may provide an option for the user to set the visible viewpoint range of the danmaku comments. The visible viewpoint range of the danmaku comments means that the terminal 100 displays the content of the danmaku comments within the viewpoint range and does not display the content of the danmaku comments outside the viewpoint range.

For example, terminal 100 may predefine the visible viewpoint range of the danmaku comment, or may receive the visible viewpoint range of the danmaku comment set by the user, as follows: The absolute value of the offset angle between the viewpoint of the danmaku comment and the playback viewpoint of the first free viewpoint video is less than angle threshold E (e.g., 45 degrees). In this case, terminal 100 may determine that the visible viewpoint range of the danmaku comment is [playback viewpoint of the first free viewpoint video - angle threshold E, playback viewpoint of the first free viewpoint video + angle threshold E].

In another example, terminal 100 may predefine the visible viewpoint range of the danmaku comment, or may receive the visible viewpoint range of the danmaku comment set by the user, as follows: The absolute value of the offset angle between the viewpoint of the danmaku comment and the playback viewpoint of the first free viewpoint video is less than angle threshold E, and the offset angle is equal to or less than 0 (the offset direction is a left offset). In this case, terminal 100 may determine that the visible viewpoint range of the danmaku comment is [playback viewpoint of the first free viewpoint video - angle threshold E, playback viewpoint of the first free viewpoint video].

In another example, the terminal 100 may predefine the visible viewpoint range of the danmaku comment, or may receive the visible viewpoint range of the danmaku comment set by the user, as follows: The absolute value of the offset angle between the viewpoint of the danmaku comment and the playback viewpoint of the first free viewpoint video is less than the angle threshold E, and the offset angle is equal to or greater than 0 (the offset direction is a right offset). In this case, the terminal 100 may determine that the visible viewpoint range of the danmaku comment is [playback viewpoint of the first free viewpoint video, playback viewpoint of the first free viewpoint video + angle threshold E].

In some other embodiments, terminal 100 may further collect and count the number of danmaku comments outside the visible viewpoint range of the danmaku comments, and display the number of danmaku comments outside the visible viewpoint range to notify the user of the status of the danmaku comments outside the visible viewpoint range. Furthermore, terminal 100 may further count the number of left-offset danmaku comments and the number of right-offset danmaku comments separately for the danmaku comments outside the visible viewpoint range of the danmaku comments.

Alternatively, the terminal 100 may further display a control related to the number of barrage comments outside the visible viewpoint range. For example, a control related to the number of barrage comments outside the visible viewpoint range may indicate the enthusiasm of users participating in discussions about video images outside the visible viewpoint range. In a specific implementation, when the number of barrage comments outside the visible viewpoint range is equal to or greater than a specific number, the control may indicate that viewpoints outside the visible viewpoint range are hot areas, and may be used to prompt users to pay attention to video images outside the visible viewpoint range. In another specific implementation, the terminal 100 displays control 1, which is related to the number of left-offset barrage comments. When the number of left-offset barrage comments is equal to or greater than a specific number, control 1 may indicate that left-offset viewpoints outside the visible viewpoint range are hot areas. The terminal 100 displays control 2, which is related to the number of right-offset barrage comments. When the number of right-offset barrage comments is equal to or greater than a specific number, control 2 may indicate that right-offset viewpoints outside the visible viewpoint range are hot areas.

For example, the visible viewpoint range of the danmaku comments is set as follows: the absolute value of the offset angle between the viewpoint of the danmaku comment and the playback viewpoint of the first free viewpoint video is less than the angle threshold E, the first free viewpoint video is a 360-degree ambient video, the number of left-offset danmaku comments may be calculated using equation (1), and the number of right-offset danmaku comments may be calculated using equation (2).

Number of left offset barrage comments = Σ(-180 degrees < offset angle < - angle threshold E) Equation (1)

Number of Right Offset Danmaku Comments = Σ(Angle Threshold E<Offset Angle<180°) Equation (2)

It can be understood that if the first free-viewpoint image is a 180-degree ambient image, then 180 degrees in equations (1) and (2) are replaced with 90 degrees. If the first free-viewpoint image is a 270-degree ambient image, then 180 degrees in equations (1) and (2) are replaced with 135 degrees.

As shown in Figure 13, when the angle threshold E is set to 45 degrees, the visible viewpoint range of the danmaku comments in the first free viewpoint video is [first free viewpoint video playback viewpoint -45 degrees, first free viewpoint video playback viewpoint +45 degrees].

When the playback viewpoint of the first free viewpoint video is 0 degrees, the visible viewpoint range of the danmaku comments is [-45 degrees, +45 degrees]. Danmaku comment 1 and danmaku comment 2 are located within the visible viewpoint range of the danmaku comments, and terminal 100 displays the contents of danmaku comment 1 and danmaku comment 2. Danmaku comment 3 and danmaku comment 5 are located at [-180 degrees, -45 degrees], outside the visible viewpoint range of the danmaku comments, and are left-offset danmaku comments. In this case, terminal 100 may display prompt information, for example, +2, on the left side of the screen to notify the user that there are two left-offset danmaku comments located outside the visible viewpoint range of the danmaku comments. Danmaku comment 4 and danmaku comment 6 are located at (+45 degrees, +180 degrees), outside the visible viewpoint range of the danmaku comments, and are right-offset danmaku comments. In this case, the terminal 100 may display prompt information, for example, +2, on the right side of the screen to prompt the user that the number of right-offset barrage comments located outside the visible viewpoint range of the barrage comments is two.

As shown in playback interface 1401 in FIG. 14A(1), playback interface 1401 includes danmaku comments within the visible viewpoint range, for example, danmaku comment 1 and danmaku comment 2. Playback interface 1401 further includes control 1402 and control 1403. Control 1402 is used to display the number of danmaku comments that are located outside the visible viewpoint range of the danmaku comments and have a left offset. Control 1403 is used to display the number of danmaku comments that are located outside the visible viewpoint range of the danmaku comments and have a right offset. In some examples, a user may operate control 1402 to display danmaku comments that are located outside the visible viewpoint range of the danmaku comments and have a left offset. For example, in response to detecting that a user has operated control 1402 in playback interface 1401, terminal 100 displays playback interface 1404 shown in FIG. 14A(2). Playback interface 1404 displays danmaku comments that are located outside the visible viewpoint range of the danmaku comments and have a left offset, for example, danmaku comment 3 and danmaku comment 5. It should be noted that the position of each danmaku comment on the display is relative to the current playback time point of the first free viewpoint video and the appearance time of each danmaku comment, and is not relative to the angular offset between the viewpoint of each danmaku comment and the current playback viewpoint of the first free viewpoint video. Playback interface 1404 may further include a control 1405 used to hide danmaku comments outside the visible viewpoint range. Optionally, in response to detecting that the user has operated control 1402 in playback interface 1404, terminal 100 displays playback interface 1406 shown in FIG. 14A(3). Playback interface 1406 displays danmaku comments that are located outside the visible viewpoint range of the danmaku comments and are offset to the right, for example, danmaku comment 4 and danmaku comment 6. Playback interface 1406 may further include a control 1405 used to hide danmaku comments outside the visible viewpoint range.

It should be noted that the barrage comments in playback interface 1401, playback interface 1404, and playback interface 1406 may be presented in the same manner, e.g., may be displayed normally. In some other embodiments, the barrage comments in playback interface 1401, playback interface 1404, and playback interface 1406 may alternatively be different. For example, any one of the barrage comment display methods described above may be used for presentation. The specific presentation method of the barrage comments is not limited here.

In another example, as shown in playback interface 1407 shown in FIG. 14B(1), playback interface 1408 shown in FIG. 14B(2), and playback interface 1409 shown in FIG. 14B(3), the presentation method for each danmaku comment is determined based on the offset angle between the viewpoint of each danmaku comment and the playback viewpoint of the first free viewpoint video. For specific determination methods, refer to the above explanation. Furthermore, for functional control in each playback interface, refer to the explanations related to playback interface 1401 shown in FIG. 14A(1), playback interface 1404 shown in FIG. 14A(2), and playback interface 1406 shown in FIG. 14A(2). Details will not be explained again here.

In some other examples, the terminal 100 may automatically further display hidden danmaku comments outside the visible viewpoint range based on a viewpoint switching operation performed by the user. For example, if the terminal 100 detects that the user has performed an operation to switch the viewpoint to the left in the interface 1401 shown in FIG. 14A(1), the terminal 100 may also display danmaku comments that are located outside the visible viewpoint range of the danmaku comments and that are offset to the left when switching the viewpoint. In another example, if the terminal 100 detects that the user has performed an operation to switch the viewpoint to the right in the interface 1401 shown in FIG. 14A(1), the terminal 100 may also display danmaku comments that are located outside the visible viewpoint range of the danmaku comments and that are offset to the right when switching the viewpoint.

Since the content of danmaku comments within the visible viewpoint range is usually strongly related to the content of the video image currently being viewed by the user in the playback viewpoint, when terminal 100 displays only the content of danmaku comments within the visible viewpoint range and does not display the content of danmaku comments outside the visible viewpoint range, the user can focus on viewing the current video image and the content of danmaku comments related to the current video image, improving the viewing experience. When the user is interested in the content of danmaku comments outside the visible viewpoint range, the content of danmaku comments outside the visible viewpoint range may alternatively be expanded by using corresponding controls to meet the user's viewing requirements in multiple scenarios.

As described above, the user may randomly switch the playback viewpoint of the first free viewpoint video. In some embodiments, in the process of switching the viewpoint of the first free viewpoint video, terminal 100 leaves the number of danmaku comments outside the visible viewpoint range unchanged. After the viewpoint of the first free viewpoint video is switched, terminal 100 reacquires the danmaku comment data acquired after the viewpoint was switched, and determines the content of the danmaku comments within the visible viewpoint range and the number of danmaku comments outside the visible viewpoint range based on the reacquired danmaku comment data.

For example, FIG. 15A shows a method for displaying barrage comments in a viewpoint switching process according to an embodiment of the present application. The method includes the following steps:

S1501: At time t0, terminal 100 detects that the user has begun to perform a viewpoint switching operation, which indicates switching the viewpoint of the first free viewpoint video.

It should be noted that before step S1501, terminal 100 may display a video image of the first free viewpoint video at any viewpoint and the corresponding danmaku comments by using the method in steps S701 to S704. In one example, terminal 100 displays a video image of the first free viewpoint video at viewpoint B at playback time T2 and the danmaku comments corresponding to playback time T2. The danmaku comments corresponding to playback time T2 include the content of the danmaku comments within the visible viewpoint range and the number of danmaku comments outside the visible viewpoint range.

S1502: From time t0 to time t1, the terminal 100 displays the message that the number of barrage comments outside the visible viewpoint remains unchanged.

In some embodiments, between time t0 and time t1, terminal 100 retains the number of danmaku comments outside the visible viewpoint that were displayed by terminal 100 at time t0. In other words, between time t0 and time t1, the number of danmaku comments outside the visible viewpoint that were displayed by terminal 100 remains unchanged.

It should be noted that the video image of the first free viewpoint video displayed by the terminal 100 between time t0 and time t1 may or may not change. This is not limited to the embodiments of this application. For information about changes in the video image of the first free viewpoint video displayed by the terminal 100 between time t0 and time t1, please refer to the description of the related content above. Details will not be described again here.

Between time t0 and time t1, the presentation method of the barrage comments within the visible viewing range displayed by the terminal 100 may or may not change. For specific details, please refer to the above related content. Details will not be described again here.

S1503: At time t1, the terminal 100 detects that the viewpoint switching operation has been completed.

S1504: After time t1, terminal 100 requests a video file of the first free viewpoint video from viewpoint C at playback time T3 from the media server.

In response, the media server returns to the terminal 100 a video file of the first free viewpoint video from viewpoint C at playback time T3.

Playback time T3 is relative to playback time T2. For example, playback time T3 = playback time T2, playback time T3 = playback time T2 - preset time offset value 1, or playback time T3 = playback time T2 + preset time offset value 2.

S1505: After time t1, terminal 100 further requests danmaku comment data corresponding to playback time T3 of the first free viewpoint video from the danmaku comment server.

In response, the barrage comment server returns barrage comment data corresponding to playback time point T3 of the first free viewpoint video to terminal 100.

Playback time T3 is relative to playback time T2. For example, playback time T3 = playback time T2, playback time T3 = playback time T2 - preset time offset value 1, or playback time T3 = playback time T2 + preset time offset value 2.

It should be noted that this step is optional. In some other embodiments, playback time point T3 = playback time point T2, or playback time point T3 = playback time point T2 - preset time offset value 1. Terminal 100 stores the danmaku comment data corresponding to playback time point T3. In this case, terminal 100 does not need to request the danmaku comment data at playback time point T3 again from the danmaku comment server. Instead, as playback of the first free viewpoint video progresses, terminal 100 requests danmaku comment data that exists after playback time point T3 or playback time point T2 from the danmaku comment server.

It should further be noted that step S1505 may be performed after, before, or simultaneously with step S1504.

S1506: The terminal 100 recalculates the number of barrage comments outside the visible viewpoint range after the viewpoint is switched.

In some embodiments, the terminal 100 further recalculates the offset angle between viewpoint C and the viewpoint of each danmaku comment within the visible viewpoint range, and determines the presentation method for each danmaku comment within the visible viewpoint range based on the recalculated offset angle.

S1507: The terminal 100 displays the video image of the first free viewpoint video from viewpoint C at playback time T3, the number of danmaku comments within the visible viewpoint range, and the number of danmaku comments outside the visible viewpoint range.

For other details of this embodiment, please refer to the relevant content of steps S1001 to S1007 in Figure 10A. Details will not be explained again here.

In some other embodiments, in step 1502, the number of danmaku comments outside the visible viewpoint range displayed by terminal 100 may alternatively be changed. For example, when terminal 100 continuously plays back the first free viewpoint video between time t0 and time t1, the playback time points of the first free viewpoint video continuously change. Correspondingly, the number of danmaku comments corresponding to the playback time points of the first free viewpoint video also changes, and the number of danmaku comments outside the visible viewpoint range also changes accordingly.

For example, as shown in FIG. 15B( 1), the terminal 100 plays a video image 1501 of a first free viewpoint video at a 60-degree viewpoint (viewpoint B). The playback interface 1501 includes danmaku comments within the visible viewpoint range, such as danmaku comment 1, danmaku comment 2, and danmaku comment 3. The playback interface 1501 further displays a control 1502 and a control 1503. The control 1502 is used to indicate the number (e.g., 2) of danmaku comments located outside the visible viewpoint range and offset to the left. The control 1503 is used to indicate the number (e.g., 2) of danmaku comments located outside the visible viewpoint range and offset to the right. When the first free viewpoint video has been played up to 20:05, the terminal 100 detects a viewpoint switching operation performed by the user (e.g., sliding the viewpoint ruler to the right). The terminal 100 continuously plays back the first free viewpoint video and switches the playback viewpoint of the first free viewpoint video in real time based on the progress of the user's viewpoint switching operation. For example, in the process of a user performing a viewpoint switching operation, the terminal 100 displays the playback interface 1504 shown in FIG. 15B( 2). In this case, the playback viewpoint of the first free viewpoint video is 65 degrees, and the playback time is 20:15. It can be understood that when the playback time changes to 20:15, the danmaku comments corresponding to the playback time also change accordingly. As shown in the playback interface 1504, the danmaku comments within the current visible viewpoint range change to danmaku comment 2, danmaku comment 3, and danmaku comment 4. The number of danmaku comments outside the visible viewpoint range and offset to the left changes to 3. The number of danmaku comments outside the visible viewpoint range and offset to the right changes to 5. In this case, it should be noted that the danmaku comments in the playback interface 1504 change accordingly as the playback time of the first free viewpoint video changes. In this case, the terminal 100 does not request new danmaku comment data from the danmaku comment server.

Furthermore, after the terminal 100 detects that the user has completed the viewpoint switching operation, the terminal 100 requests danmaku comment data corresponding to the current playback time point from the danmaku comment server. The content of the danmaku comments changes. As shown in the playback interface 1505 of FIG. 15B( 3), in this case, the playback viewpoint of the first free viewpoint video is 70 degrees, the playback time point is 20:15, and the danmaku comments within the visible viewpoint range change from danmaku comment 3 to danmaku comment 7. The number of danmaku comments that are outside the visible viewpoint range and offset to the left changes to 12. The number of danmaku comments that are outside the visible viewpoint range and offset to the right changes to 9.

In some other embodiments, the execution of the viewpoint switching operation may be paused during the process of the user performing the viewpoint switching operation. In this case, when the execution of the viewpoint switching operation is paused, the terminal 100 may re-request the danmaku comment data corresponding to the current playback point from the danmaku comment server, and then update the content of the danmaku comments within the visible viewpoint range and the number of danmaku comments outside the visible viewpoint range based on the re-acquired danmaku comment data.

16A and 16B are schematic flowcharts of another method for displaying barrage comments in a process of switching viewpoints according to an embodiment of the present application. The method includes the following steps:

S1601: At time t0, terminal 100 detects that the user has started to perform a viewpoint switching operation on the first free viewpoint video, and the viewpoint switching operation indicates switching the viewpoint of the first free viewpoint video.

In one example, the terminal 100 displays the video image of the first free viewpoint video from viewpoint B at playback time T2 and the danmaku comment data of the first free viewpoint video at playback time T2. The danmaku comment corresponding to playback time T2 includes the content of the danmaku comments within the visible viewpoint range and the number of danmaku comments outside the visible viewpoint range.

For example, if the terminal 100 detects that the user's finger remains on the viewpoint ruler for longer than a preset duration of 2 (e.g., 3 seconds), the terminal 100 determines that the user has begun to perform a viewpoint switching operation and records the current time as time t0.

S1602: From time t0 to time t2, the terminal 100 leaves the number of displayed barrage comments outside the visible viewpoint unchanged.

In some embodiments, between time t0 and time t1, terminal 100 retains the number of danmaku comments outside the visible viewpoint range that were displayed by terminal 100 at time t0. In other words, the number of danmaku comments outside the visible viewpoint range that were displayed by terminal 100 between time t0 and time t1 remains unchanged. In some other embodiments, the number of danmaku comments outside the visible viewpoint range that are displayed by terminal 100 may alternatively change. For example, when terminal 100 continuously plays back the first free viewpoint video between time t0 and time t1, the playback time points of the first free viewpoint video change continuously. Correspondingly, the number of danmaku comments corresponding to the playback time points of the first free viewpoint video also changes, and the number of danmaku comments outside the visible viewpoint range also changes accordingly.

It should be noted that between time t0 and time t1, the video image of the first free viewpoint video displayed by terminal 100 may or may not change. This is not a limitation in the embodiment of this application. Between time t0 and time t1, the presentation method of the danmaku comments within the visible viewpoint range displayed by terminal 100 may not change. Obviously, the presentation method of the danmaku comments within the visible viewpoint range displayed by terminal 100 may alternatively change. This is not a limitation in the embodiment of this application.

S1603: At time t2, the terminal 100 detects that the user has paused the execution of the viewpoint switching operation.

For example, after time t0, the device continuously detects that the user's finger has slid across the screen. When the device detects that the user's finger has not slid or been removed from the screen, the device determines that the user has paused the execution of the viewpoint switching operation and records the current time as time t2.

S1604: After time t2, terminal 100 requests a video file of the first free viewpoint video from viewpoint D at playback time T4 from the media server.

In other words, after detecting that the user has paused the execution of the viewpoint switching operation, the terminal 100 determines the current playback viewpoint of the first free viewpoint video, for example, viewpoint D. Playback time point T4 is relative to playback time point T2. For example, playback time point T4 = playback time point T2, playback time point T4 = playback time point T2 - preset time offset value 1, or playback time point T4 = playback time point T2 + preset time offset value 2.

S1605: After time t2, terminal 100 further requests danmaku comment data corresponding to playback time T4 of the first free viewpoint video from the danmaku comment server.

S1606: The terminal 100 recalculates the number of barrage comments outside the visible viewpoint range after the viewpoint is switched.

In some embodiments, the terminal 100 further recalculates the offset angle between viewpoint C and the viewpoint of each danmaku comment within the visible viewpoint range, and determines the presentation method for each danmaku comment within the visible viewpoint range based on the recalculated offset angle.

S1607: The terminal 100 displays the video image of the first free viewpoint video from viewpoint D at playback time T4, and the terminal 100 displays the video image of the first free viewpoint video from viewpoint C at playback time T3, the number of danmaku comments within the visible viewpoint range, and the number of danmaku comments outside the visible viewpoint range.

S1608: At time t1, the terminal 100 detects that the viewpoint switching operation has been completed.

For example, after time t2, the terminal continuously detects that the user's finger has slid across the screen. When the terminal detects that the user's finger has been removed from the screen, the terminal determines that the user has completed the viewpoint switching operation and records the current time as time t1.

S1609: After time t1, terminal 100 requests a video file of the first free viewpoint video from viewpoint C at playback time T3 from the media server.

S1610: After time t1, the terminal 100 further requests danmaku comment data corresponding to playback time T3 of the first free viewpoint video from the danmaku comment server.

In some examples, playback time T3 is relative to playback time T4. For example, playback time T3 = playback time T4, playback time T3 = playback time T4 - preset time offset value 1, or playback time T3 = playback time T4 + preset time offset value 2.

S1611: The terminal 100 recalculates the number of barrage comments outside the visible viewpoint range after the viewpoint is switched.

S1612: The terminal 100 displays the video image of the first free viewpoint video from viewpoint C at playback time T3, and the terminal 100 displays the video image of the first free viewpoint video from viewpoint C at playback time T3, the number of danmaku comments within the visible viewpoint range, and the number of danmaku comments outside the visible viewpoint.

For other details of steps S1601 to S1612, refer to the related details of steps S1001 to S1007 in Figure 10A and the related details of steps S1501 to S1507 in Figure 15A. Details will not be explained again here.

16A and 16B are described by using an example in which the viewpoint switching operation is paused only once. It can be understood that the technical solutions provided in the embodiments of this application are also applicable to the case in which the viewpoint switching operation is paused more than once.

Figure 17 is a schematic flowchart of another method for displaying barrage comments in a free viewpoint video according to an embodiment of the present application. This procedure includes the following steps:

S1701: The terminal displays a first interface for a first free viewpoint video at a first playback point.

The first interface includes a video image of a first free viewpoint video from a first viewpoint and a first barrage comment.

For example, the first playback time point is relative to the appearance of the first danmaku comment. For example, the first playback time point is the appearance of the first danmaku comment. Alternatively, the first playback time point is after the appearance of the first danmaku comment and within the duration during which the first danmaku comment is continuously displayed.

The viewpoint offset between the third viewpoint corresponding to the first barrage comment and the first viewpoint is within a first preset angle range. The first preset angle range is the preset visible viewpoint range.

S1702: The terminal receives a first operation, which is an operation to switch the playback viewpoint of the first free viewpoint video from the first viewpoint to the second viewpoint.

S1073: In response to receiving the first operation, the terminal displays a second interface of the first free viewpoint video at the first playback point.

The second interface includes a video image of the first free viewpoint video from a second viewpoint, and the second interface does not include the first barrage comments.

The viewpoint offset between the third viewpoint corresponding to the first danmaku comment and the second viewpoint is outside the first preset angle range. In other words, the viewpoint offset between the third viewpoint corresponding to the first danmaku comment and the second viewpoint is not within the preset visible viewpoint range. It can be understood that the offset angle between the first viewpoint and the third viewpoint corresponding to the first danmaku comment is smaller than the offset angle between the second viewpoint and the third viewpoint corresponding to the first danmaku comment.

In one example, in the process of switching viewpoints, the terminal automatically pauses playback of the first free viewpoint video. Thus, before the user performs the first operation, the terminal plays video images of the first free viewpoint video at the first viewpoint. When the terminal detects that the user has performed the first operation at the first playback time point, it automatically pauses playback of the first free viewpoint video. After the first operation is performed and the playback viewpoint of the first free viewpoint video is switched from the first viewpoint to the second viewpoint, the first free viewpoint video begins to be played back from the first playback time point.

In another example, in the process of switching viewpoints, the terminal continuously plays back the first free viewpoint video. In this case, before performing the first operation, the user may first manually pause the playback of the first free viewpoint video. In this case, the first free viewpoint video remains at the first playback point. Then, after the user performs the first operation and the terminal switches the playback viewpoint of the first free viewpoint video from the first viewpoint to the second viewpoint, the video image of the first free viewpoint video still remains at the first playback point.

Alternatively, after the first operation is performed and the terminal switches the playback viewpoint of the first free viewpoint video from the first viewpoint to the second viewpoint, playback begins from a playback point after the first playback point. Alternatively, the user may manually drag the playback progress bar to adjust the playback point back to the first playback point.

Optionally, the second interface includes a second danmaku comment, and the offset angle between the second viewpoint and a viewpoint corresponding to the second danmaku comment is within a first preset angle range. It may be understood that the playback time of the second danmaku comment is relative to the first playback time.

In some other embodiments, the first interface displayed by the terminal further includes a first control, the first control being associated with a number of first danmaku comments, the number of first danmaku comments being the number of danmaku comments obtained when the offset angle between the viewpoint of the danmaku comments and the playback viewpoint of the terminal is within a second preset angle range, the second preset angle range not overlapping with the first preset angle range. The terminal displays a third interface upon receiving a third operation on the first control, the third interface including the first danmaku comments. In other words, the user may expand the danmaku comments hidden by the terminal by using the first control.

An embodiment of this application further provides an apparatus. The apparatus is included in a terminal, and has a function that implements the behavior of the terminal in any of the methods in the above embodiments. The function may be implemented by hardware, or by hardware that executes corresponding software. The hardware or software includes at least one module or unit corresponding to the above function, such as a detection module or unit, a display module or unit, a determination module or unit, and a calculation module or unit.

An embodiment of this application further provides a computer storage medium containing computer instructions. When the computer instructions are executed on a terminal, the terminal is enabled to perform any of the methods in the above embodiments.

An embodiment of this application further provides a computer program product. When the computer program product is executed on a computer, the computer is enabled to execute any of the methods in the above-described embodiments.

An embodiment of this application further provides a graphical user interface on a terminal. The terminal includes a display, a camera, a memory, and one or more processors. The one or more processors are configured to execute one or more computer programs stored in the memory, and the graphical user interface includes a graphical user interface that is displayed when the terminal executes any of the methods in the above embodiments.

It can be understood that to achieve the above functions, a terminal or the like includes corresponding hardware structures and/or software modules for executing the functions. In combination with the examples described in the embodiments disclosed in this specification, those skilled in the art should easily recognize that the units, algorithms, and steps in the embodiments of this application may be realized by hardware or a combination of hardware and computer software. Whether the functions are performed by hardware or by hardware driven by computer software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to achieve the described functions for each specific application, but the implementation method should not be considered to exceed the scope of the embodiments of the present invention.

In embodiments of this application, a terminal or the like may be divided into functional modules based on the above-mentioned method examples. For example, each functional module may be obtained through division based on its corresponding function, or two or more functions may be integrated into one processing module. The integrated module may be implemented in the form of hardware or in the form of a software functional module. It should be noted that in embodiments of the present invention, the division into modules is an example and is merely a logical division of function, and other divisions may be used in actual implementation.

The above description of the implementation method allows those skilled in the art to clearly understand that the above division of functional modules is taken as an illustrative example for the purpose of convenient and concise description. In actual applications, the above functions can be assigned to different modules and implemented as needed; that is, the internal structure of the device is divided into different functional modules to implement all or part of the above functions. For detailed operation processes of the above systems, devices, and units, please refer to the corresponding processes in the above method embodiments, and the details will not be described again here.

The functional units in the embodiments of this application may be integrated into a single processing unit, or each unit may exist physically independently, or two or more units may be integrated into a single unit. The integrated units may be realized in the form of hardware or in the form of software functional units.

When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, the integrated unit may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application may essentially, or all or part of the portions contributing to the prior art or the technical solutions may be realized in the form of a software product. A computer software product is stored in a storage medium and includes several instructions for instructing a computer device (which may be a personal computer, a server, or a network device) or a processor to perform all or part of the steps of the methods described in the embodiments of this application. The above storage medium includes any medium capable of storing program code, such as a flash memory, a removable hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

The above description is merely a specific implementation of this application and is not intended to limit the scope of protection of this application. Any modifications or substitutions within the technical scope disclosed in this application shall be included in the scope of protection of this application. Therefore, the scope of protection of this application shall be subject to the scope of protection of the claims.

Claims (19)

A method for displaying barrage comments of a free viewpoint video, which is applied to a terminal, comprising:
a step of displaying a first interface of a first free viewpoint video at a first playback time point, the first interface including a video image of the first free viewpoint video at a first viewpoint and a first danmaku comment, an offset angle between the first viewpoint and a third viewpoint corresponding to the first danmaku comment being within a first preset angle range, the first interface further including a first control, the first control being related to the number of first danmaku comments, the number of first danmaku comments being the number of danmaku comments obtained when the offset angle between the viewpoint of the danmaku comment and a playback viewpoint of the terminal being within a second preset angle range, the second preset angle range not overlapping with the first preset angle range ;
receiving a first operation, the first operation being an operation for switching a playback viewpoint of the first free viewpoint video from the first viewpoint to a second viewpoint;
and a step of displaying a second interface of the first free viewpoint video at the first playback point in time in response to receiving the first operation, the second interface including a video image of the first free viewpoint video at the second viewpoint, and the second interface not including the first barrage comment. The method of claim 1, wherein the offset angle between the first viewpoint and the third viewpoint corresponding to the first danmaku comment is smaller than the offset angle between the second viewpoint and the third viewpoint corresponding to the first danmaku comment. Before receiving the first operation,
The method of claim 1 , further comprising receiving a second operation to pause playback of the first free viewpoint video. The method of claim 1 , wherein an offset angle between the second viewpoint and the third viewpoint corresponding to the first barrage comment is outside the first preset angle range. The method of claim 4, wherein the second interface includes a second danmaku comment, and the offset angle between the second viewpoint and a viewpoint corresponding to the second danmaku comment is within the first preset angle range. The method of claim 5, wherein the transmission time of the first danmaku comment is relative to the first playback time, and the transmission time of the second danmaku comment is relative to the first playback time. receiving a third operation for the first control;
2. The method of claim 1 , further comprising: in response to receiving the third operation, displaying a third interface, the third interface including the first barrage comment. In response to receiving the first operation, after displaying a second interface of the first free viewpoint video at the first playback point in time,
receiving a fourth operation, the fourth operation being an operation of sending a third barrage comment;
2. The method of claim 1, further comprising: in response to receiving the fourth operation, transmitting data of the third danmaku comment to a danmaku comment server, wherein the data of the third danmaku comment includes a fourth perspective. The method of claim 1, wherein the first interface further includes a fourth barrage comment, the fourth barrage comment corresponding to the first viewpoint. 10. The method of claim 9 , wherein in the first interface, a first style of the first barrage comment is different from a second style of the fourth barrage comment. before displaying a first interface of a first free viewpoint video at a first playback time point;
11. The method of claim 10, further comprising determining to display the first danmaku comment in the first style in the first interface based on a first offset angle between the first viewpoint and the third viewpoint corresponding to the first danmaku comment. 12. The method of claim 11, wherein, when the first offset angle is a positive value, the first style is a direction in which the text direction of the barrage comments is rotated clockwise by the first angle with respect to a first direction, the first direction being parallel to a first bezel of a screen of the terminal, and when the first offset angle is a negative value , the first style is a direction in which the text direction of the barrage comments is rotated counterclockwise by the first angle with respect to the first direction. receiving a fifth operation for switching the viewpoint;
displaying a fifth interface in the process of the fifth operation, where the fifth interface and the first interface are the same;
The method of claim 1 , further comprising: after the process of the fifth operation, displaying a sixth interface of the first free viewpoint video at a second playback point in time. receiving a sixth operation for switching the viewpoint;
displaying a seventh interface in the sixth operation process, the seventh interface including a video image of the first free viewpoint video at a third playback point in time;
2. The method of claim 1, further comprising: after the process of the sixth operation, displaying an eighth interface of the first free viewpoint video at a second playback point in time, wherein the third playback point is after the first playback point in time and before the second playback point in time. 15. The method of claim 14 , wherein the seventh interface further includes a fifth barrage comment, and the first interface further includes no fifth barrage comment. 15. The method of claim 14, wherein the seventh interface further includes the first barrage comment, and further comprising: a display style of the first barrage comment in the seventh interface that is different from a display style of the first barrage comment in the first interface. A terminal including a processor, a memory, and a touchscreen,
The memory and the touch screen are coupled to the processor, the memory is configured to store computer program code, the computer program code includes computer instructions, and when the processor reads the computer instructions from the memory, the terminal is enabled to execute the method for displaying barrage comments in free viewpoint video described in any one of claims 1 to 16 . A computer-readable storage medium containing computer instructions,
A computer-readable storage medium, the computer instructions being, when executed on a terminal, capable of causing the terminal to execute the method for displaying barrage comments in free viewpoint video described in any one of claims 1 to 16 . A computer program which, when executed by a processing device, causes the processing device to carry out a method according to any one of claims 1 to 16 .