JP7613878B2 - Information processing device and computer program - Google Patents

Description

The present disclosure relates to data processing technology, and more particularly to an information processing device and a computer program relating to content viewing.

With the spread of broadband communications, various video distribution services are being provided (see, for example, Patent Document 1).

JP 2020-162881 A

In addition to the image of the content, the sound of the content is also an important factor in determining the quality of the user experience when viewing the content. The inventors felt that there was room for improvement in the sound provided to users who simultaneously view multiple distributed contents in order to improve the quality of the user experience.

This disclosure has been made in light of these issues, and one objective is to provide technology that improves the quality of the user experience when viewing distributed content.

In order to solve the above problem, an information processing device according to one aspect of the present disclosure includes an acquisition unit that acquires data of multiple contents distributed from a server, a display control unit that displays one of the multiple contents on a relatively large main screen and the other contents on a relatively small sub-screen, and changes the content that was previously displayed on the main screen to be displayed on the sub-screen in response to a user operation, and changes the content that was previously displayed on the sub-screen to be displayed on the main screen, and an audio output control unit that switches the audio output from the speaker from the audio of the content displayed on the main screen to the audio of the content displayed on the sub-screen in response to a user operation.

Another aspect of the present disclosure is an information processing system. This information processing system includes a broadcasting device that distributes content data via a broadcast network, a server that distributes content data including video of performers of the content via the Internet, a viewer terminal that displays the video of the performers of the content distributed from the server, and a performer terminal that presents information to the performers. When a comment from the viewer is input, the viewer terminal transmits the comment data, and the performer terminal displays the comment transmitted from the viewer terminal.

In addition, any combination of the above components, or any conversion between the expressions of this disclosure as a system, method, computer program, or recording medium on which a computer program is readably recorded, is also valid as an aspect of this disclosure.

The technology disclosed herein can improve the quality of the user experience when viewing distributed content.

FIG. 1 is a diagram illustrating a configuration of a communication system according to an embodiment. FIG. 2 is a block diagram showing functional blocks of the user device of FIG. 1; FIG. 13 is a diagram showing an example of an App screen. FIG. 13 is a diagram showing an example of an App screen. FIG. 13 is a diagram showing an example of an App screen. FIG. 11 is a diagram showing an example of transition of an App screen. FIG. 13 is a diagram showing an example of an App screen.

The server of the communication system of the embodiment provides a plurality of live contents (electronic contents) relating to the same event (e.g., a baseball game between team A and team B) to the viewer's terminal. The live content of the embodiment is video data showing an ongoing event, and includes video data and audio data. The viewer's terminal displays one of the plurality of live contents on a large main screen, and displays the other live contents on a small sub-screen. The viewer's terminal is configured to be able to switch the audio output from the speaker between the audio of the live content being displayed on the main screen and the audio of the live content being displayed on the sub-screen.

At least one live content to be distributed is distributed via the Internet and is also broadcast to the public via a broadcast network. The communication system of the embodiment is configured so that comments entered by the viewer on the viewer's terminal are displayed on the terminal of the performer of the live content. This allows interactive communication between the performer of the live content for public broadcast and the viewer, improving the quality of the user experience of viewing the live content.

Figure 1 shows the configuration of a communication system 10 according to an embodiment. The communication system 10 is an information processing system that distributes multiple live contents to viewers. The communication system 10 includes user devices 12a, 12b, 12c, and 12d (collectively referred to as "user devices 12"), camera devices 14a, 14b, 14c, and 14d (collectively referred to as "camera devices 14"), a video distribution server 16, a microphone 20, a performer terminal 22, a broadcasting device 24, and a radio device 28.

The multiple user devices 12 are information processing devices operated by multiple viewers (hereinafter also referred to as "users") who watch live content distributed from the video distribution server 16. Each user device 12 displays the multiple live contents distributed from the video distribution server 16 on a display unit (such as a liquid crystal display). Each user device 12 may be, for example, a PC, a smartphone, or a tablet terminal.

Each of the multiple camera devices 14 captures multiple different live contents related to the same event. For example, camera device 14a captures an ongoing baseball game in the manner of a normal baseball broadcast. Camera device 14b captures the performers (studio) of a radio program that is commenting on or explaining the baseball game. Camera device 14c captures the cheering squad that is supporting the baseball game. Camera device 14d is placed in the seating area (e.g., VIP seats) of the stadium where the baseball game is being played, and captures the baseball game from the seating area. Each camera device 14 includes a microphone function. At least some of the multiple camera devices 14 may be spherical cameras (in other words, 360° cameras).

The video distribution server 16 is an information processing device that distributes multiple live content data output from multiple camera devices 14 to the user device 12 via the Internet 18 in response to a request from the user device 12. In the embodiment, the video distribution server 16 transmits multiple live content data in parallel by streaming using HTTP live streaming, which is an HTTP-based media streaming protocol.

The video distribution server 16 of the embodiment simultaneously distributes data of four live contents. The data of the four live contents includes the following: (1) Video and audio of an ongoing baseball game. (2) Video and audio of performers (in the studio) of a radio program providing commentary or commentary on the baseball game. (3) Video and audio of a cheering squad supporting the baseball game. (4) Video and audio captured in the stands of the stadium where the baseball game is being played.

In a studio where a radio program that reports or explains a baseball game is produced, a microphone 20 and a performer terminal 22 are installed in addition to the camera device 14b. The microphone 20 captures the audio of the radio program that reports or explains a baseball game (the audio of the performers of the program, which can also be said to be the audio of the studio). The broadcasting device 24 is a device that distributes content data via the broadcasting network 26. In the embodiment, the broadcasting device 24 publicly transmits the audio of the radio program captured by the microphone 20 via the broadcasting network 26 (radio broadcasting network). The radio device 28 plays back and outputs the audio of the radio program transmitted from the broadcasting device 24.

The performer terminal 22 is an information terminal that presents information to the performers of the radio program. The performer terminal 22 may be a PC, a smartphone, or a tablet terminal. The performer terminal 22 includes a display unit (such as a liquid crystal display). The performer terminal 22 displays the user's comments transmitted from the user device 12 on the display unit.

Figure 2 is a block diagram showing the functional blocks of the user device 12 in Figure 1. The multiple functional blocks shown in the block diagram in this specification can be configured in hardware with circuit blocks, memory, and other LSIs, and in software, are realized by a CPU executing a program loaded into memory. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms using only hardware, only software, or a combination of both, and are not limited to any one of them.

The user device 12 includes a control unit 30, a communication unit 32, a display unit 34, a speaker 36, and a storage unit 41. The control unit 30 executes various data processing related to content viewing. The communication unit 32 communicates with an external device according to a predetermined communication protocol. The control unit 30 transmits and receives data to and from the video distribution server 16 via the communication unit 32.

The display unit 34 displays image information (e.g., video of live content, etc.). The display unit 34 is, for example, a liquid crystal display. The speaker 36 outputs audio information (e.g., audio of live content). The storage unit 41 stores data (video data and audio data) of multiple live contents provided by the video distribution server 16, which is data acquired by the content acquisition unit 40 described below.

The control unit 30 includes an operation acquisition unit 38, a content acquisition unit 40, a display control unit 42, an audio output control unit 44, a comment transmission unit 46, and a heart transmission unit 48. A computer program (hereinafter also referred to as a "video viewing App") in which modules corresponding to these multiple functional blocks are implemented may be downloaded from a specified server via a network and installed in the storage of the user device 12. The video viewing App may also be stored in a computer-readable recording medium, and may be installed in the storage of the user device 12 via the recording medium. The CPU of the user device 12 may fulfill the functions of the multiple functional blocks by reading the video viewing App into the main memory and executing it.

The operation acquisition unit 38 detects operations input by the user to the video viewing App. The content acquisition unit 40 requests the video distribution server 16 to provide live content in response to the user's operation. The content acquisition unit 40 acquires data (video data and audio data) of multiple live contents distributed from the video distribution server 16. The content acquisition unit 40 stores the data of multiple live contents distributed from the video distribution server 16 in the memory unit 41.

The display control unit 42 displays a video viewing App screen (hereinafter also referred to as the "App screen") on the display unit 34 and controls the display of live content on the App screen. The display control unit 42 reads out video data of multiple live content items stored in the memory unit 41 and displays the videos of the multiple live content items in parallel on the display unit 34.

The display control unit 42 displays the video of one of the multiple live contents on the main screen, which is relatively large. At the same time, the display control unit 42 displays the video of the other live contents that are not displayed on the main screen on the sub-screen, which is relatively small. The main screen can be said to be a video display area that occupies a relatively large area on the App screen, and the sub-screen can be said to be a video display area that occupies a relatively small area on the App screen.

In addition, in response to a user operation, the display control unit 42 changes the live content that was displayed on the main screen until the operation was input to be displayed on the sub-screen. At the same time, the display control unit 42 changes the live content that was displayed on the sub-screen until the operation was input to be displayed on the main screen.

The audio output control unit 44 reads out the audio data of one live content from the audio data of the multiple live contents stored in the storage unit 41, and outputs the audio of that live content from the speaker 36. In response to a user operation, the audio output control unit 44 switches the audio output from the speaker 36 from the audio of the live content displayed on the main screen to the audio of the live content displayed on the sub-screen.

Figure 3 shows an example of App screen 50 (App screen 50a). App screen 50a includes a main screen 52, a sub-screen 54, a sub-screen 56, and a sub-screen 58. The main screen 52 displays live video of a baseball game in progress. The sub-screen 54 displays video of performers (in the studio) of a radio program providing commentary or commentary on the baseball game. The sub-screen 56 displays video of a cheering squad supporting the baseball game. The sub-screen 58 displays video captured in the stands of the stadium where the baseball game is being played.

The display control unit 42 displays comments entered by a user who owns one user device 12 (own device) and comments entered by users who own other user devices 12 (other devices) in the comment area 60 of the main screen 52. The display control unit 42 displays a comment input area 62, a send button 64, a heart button 66, a voice button 68, and an enlarge button 70. The display control unit 42 displays a switch button 72 on each of the sub-screens 54, 56, and 58.

The comment input area 62 is a text area in which the user inputs a comment string. When a selection operation (such as pressing) is input on the send button 64, the comment sending unit 46 sends the comment string entered in the comment input area 62 to the video distribution server 16. The video distribution server 16 transmits comment data sent from a certain user device 12 to other user devices 12 and the performer terminal 22. The display control unit 42 displays the comment sent from the video distribution server 16 in the comment area 60 of the main screen 52.

When a selection operation (such as pressing) is input for the heart button 66, the heart sending unit 48 sends the fact that the heart button 66 has been selected to the video distribution server 16. The video distribution server 16 stores the total number of times the heart button 66 has been selected by multiple users (referred to as the "total number of hearts"). When the video distribution server 16 is notified by a certain user device 12 that the heart button 66 has been selected, it notifies the multiple user devices 12 and the performer terminal 22 of the fact that the heart button 66 has been selected and the total number of hearts. When the display control unit 42 is notified by the video distribution server 16 that the heart button 66 has been selected, it displays a heart mark 67 on the main screen 52. The display control unit 42 also displays the total number of hearts notified by the video distribution server 16 on the main screen 52. Similarly, the performer terminal 22 also displays the heart mark 67 and the total number of hearts on the display unit.

When a selection operation (such as pressing) is input for the switching button 72 of any of the sub-screens 54, 56, and 58, the display control unit 42 changes the App screen 50 so that the image of the live content displayed on the sub-screen on which the selected switching button 72 is located (here referred to as the "specific sub-screen") is displayed on the main screen 52. At the same time, the display control unit 42 changes the App screen 50 so that the image of the live content that was previously displayed on the main screen 52 is displayed on the specific sub-screen.

Figure 4 also shows an example of the App screen 50 (App screen 50b). The App screen 50b in the figure is an App screen that is displayed when the switching button 72 of the sub-screen 54 in the App screen 50a in Figure 3 is selected. The main screen 52 of the App screen 50b displays a video of a radio program performer (in the studio) who is commenting or explaining the ongoing baseball game. The sub-screen 54 of the App screen 50b displays a live video of the baseball game. That is, the display contents are switched between the main screen 52 and the sub-screen 54 in Figures 3 and 4. According to the video viewing App of the embodiment (the user device 12 in which the video viewing App is installed), the user can arbitrarily select the live content to be displayed on the main screen and the sub-screen, improving the quality of the user experience of viewing live content.

When a selection operation (such as pressing) is input for the enlarge button 70 provided on the main screen 52 of the App screen 50, the display control unit 42 changes the image of the live content that was displayed on the main screen 52 before the operation was input so that it is displayed on the entire App screen 50 (i.e., full screen). At the same time, the display control unit 42 hides the images of the multiple live contents that were displayed on the multiple sub-screens before the operation was input. Figure 5 also shows an example of the App screen 50 (App screen 50c). The App screen 50c in the figure shows the App screen that is displayed when the enlarge button 70 of the main screen 52 in the App screen 50a in Figure 3 is selected.

When displaying one live content full screen, the display control unit 42 displays a reduce button 74 in the display area of that live content. When a selection operation (such as pressing) is input for the reduce button 74, the display control unit 42 displays the live content displayed full screen on the main screen 52, and returns to the App screen 50 where other live content that was not displayed is displayed on the sub-screen. For example, when the reduce button 74 is selected on the App screen 50c in FIG. 5, the display control unit 42 returns to the display of the App screen 50a in FIG. 3.

Figure 6 shows an example of the transition of App screen 50. When the switch button 72 of sub-screen 54 is selected on App screen 50a of Figure 3, the screen transitions to App screen 50b of Figure 4. When the switch button 72 of sub-screen 54 is selected on App screen 50b of Figure 4, the screen transitions to App screen 50a of Figure 3. When the enlarge button 70 of main screen 52 is selected on App screen 50a of Figure 3, the screen transitions to App screen 50c of Figure 5. When the reduce button 74 is selected on App screen 50c of Figure 5, the screen transitions to App screen 50a of Figure 3.

Figure 7 shows an example of App screen 50 (App screen 50a). The audio output control unit 44 outputs the audio of the live content displayed on the main screen 52 from the speaker 36 among the audio of the multiple live contents displayed on the App screen 50 as the default setting of the video viewing App. When a selection operation (pressing, etc.) is input on the audio button 68 on the main screen 52 of the App screen 50, the display control unit 42 displays an object (audio selection area 76) on the main screen 52 for allowing the user to select the audio to be output. The audio output control unit 44 outputs the audio of the live content selected in the audio selection area 76 from the speaker 36.

The selection option "Main" in the audio selection area 76 is for selecting the audio of the live content being displayed on the main screen 52. The selection option "Sub1" is for selecting the audio of the live content being displayed on the sub-screen 54. The selection option "Sub2" is for selecting the audio of the live content being displayed on the sub-screen 56. The selection option "Sub3" is for selecting the audio of the live content being displayed on the sub-screen 58.

For example, when "Sub1" is selected in the audio selection area 76 of the App screen 50a in FIG. 7, the audio output control unit 44 switches the audio to be output from the audio of the live content being displayed on the main screen 52, which is the default, to the audio of the live content being displayed on the sub-screen 54. In this way, according to the video viewing App of the embodiment (user device 12 on which the video viewing App is installed), it is possible to output the audio that the user desires (e.g., the audio of the live content being displayed on the sub-screen) from the audio of multiple live contents being simultaneously distributed from the video distribution server 16. This can improve the quality of the user experience when viewing the live content.

Furthermore, when the image of the live content (herein referred to as the "first content") displayed on the main screen is displayed in full screen while the audio of the live content (herein referred to as the "second content") displayed on the sub screen is being output, the audio output control unit 44 continues to output the audio of the second content that became hidden when the first content was displayed in full screen. In other words, when the audio output of the second content on the sub screen has been selected in advance, the audio output control unit 44 maintains the audio output of the second content without resetting the audio to be output, even if the first content on the main screen is displayed in full screen.

For example, suppose that the audio of the live content on the sub-screen 54 is selected on the App screen 50a in FIG. 7, and then the enlarge button 70 on the main screen 52 is selected. In this case, the App screen 50 transitions to the App screen 50c shown in FIG. 5. Meanwhile, the audio output on the App screen 50c after the transition is maintained as the audio of the live content displayed on the sub-screen 54 before the enlarge button 70 was selected (in other words, before full-screen display). In this way, by providing the audio of the second content desired by the user (e.g., the audio of a radio program related to the baseball game, the audio of the cheering squad, etc.) while the video of the first content desired by the user (e.g., a video of a regular baseball game broadcast) is displayed in full screen, a viewing experience that meets the user's wishes can be provided.

Next, a configuration for synchronizing the playback timing of a plurality of live contents will be described.
In the communication system 10 of the embodiment, video and audio of four live contents are streamed from the video distribution server 16 to the user device 12 via the Internet 18. Therefore, there may be a difference in timing when the data of each live content (data including video and audio, hereinafter also referred to as "chunk") arrives at the user device 12. In this case, if playback is started sequentially from the live contents whose chunks have been acquired, a time difference will occur in the display of the multiple live contents on the App screen 50.

The display control unit 42 therefore starts displaying the video of the multiple live contents on the main screen and the sub screen when the data (chunks) of all the live contents that are distributed simultaneously have been acquired. In other words, the display control unit 42 refrains from starting to display the live content that was acquired first until the data (chunks) of all the live contents that are distributed simultaneously have been acquired. This makes it possible to avoid a time lag occurring in the video of the multiple live contents that are played and displayed on the App screen.

Specifically, the storage unit 41 stores the identification information of each of the four live contents distributed from the video distribution server 16, and the display control unit 42 reads out and grasps the identification information of each live content from the storage unit 41. Furthermore, each chunk of the four live contents distributed from the video distribution server 16 includes the identification information of its own live content. The display control unit 42 determines whether or not all chunks of the four live contents have been acquired by comparing the identification information included in the chunks stored in the storage unit 41 with the identification information of each of the four live contents stored in advance.

In addition, each chunk of the four live contents includes time information (which can also be considered as information indicating the playback timing). When the display control unit 42 determines that all chunks of the four live contents have been acquired, it refers to the time information included in the chunks of each live content and starts the display process from the chunks of each live content that indicate the same time position (same playback timing).

When the display of multiple live content images starts on the App screen 50, the audio output control unit 44 starts audio output for one of the multiple live content images (the live content displayed on the main screen by default).

Next, a configuration for realizing interactive communication between performers of live content and viewers will be described.
1, the broadcasting device 24 transmits audio data of a radio program that provides commentary or commentary on an ongoing baseball game via a broadcasting network 26. Meanwhile, the video distribution server 16 distributes data of the radio program, including video of the performers of the radio program (live content data), via the Internet 18.

The display control unit 42 of the user device 12 displays the video of the performers of the radio program distributed from the video distribution server 16 on the App screen 50 (e.g., the sub-screen 54 in FIG. 3). When a user comment is entered on the App screen 50, the comment sending unit 46 of the user device 12 sends the comment data to the video distribution server 16. The video distribution server 16 sends the comment data sent from the user device 12 to the performer terminal 22. The performer terminal 22 displays the comment sent from the user device 12 on the display unit.

When the heart button 66 is selected on the App screen 50, the heart sending unit 48 of the user device 12 transmits this to the video distribution server 16. The video distribution server 16 notifies the performer terminal 22 that the heart button 66 has been selected on the user device 12. In response to the notification from the video distribution server 16, the performer terminal 22 displays a heart mark 67 on the display unit.

With this configuration, reactions (such as comments and pressing of the heart button 66) of users watching a radio program using the video viewing app can be fed back to the radio program performers, supporting interactive communication between the radio program performers and users. For example, a radio program distributed to a radio device 28 via a broadcast network 26 may contain commercials (advertising broadcasts), whereas a radio program distributed to a user device 12 via the Internet 18 may not contain commercials. In this case, the radio program performers can respond to comments from users during commercials in the radio program, thereby realizing communication with users of the video viewing app without interfering with the public broadcast radio program.

The present disclosure has been described above based on examples. These examples are merely illustrative, and those skilled in the art will understand that various modifications are possible in the combination of each component or each processing process, and that such modifications are also within the scope of the technology of the present disclosure.

In the above embodiment, a heart mark is used as an indication of the user's reaction, but the mark is not limited to this. In other words, a star mark or a double circle mark (◎) can also be used, and any mark can be applied as long as it indicates the viewer's reaction (e.g. interest, favorability, etc.).

In the above embodiment, the video distribution server 16 distributed multiple live contents related to the same event to the user device 12. As a variation, the video distribution server 16 may distribute multiple live contents related to different events to the user device 12. The video distribution server 16 may also distribute electronic content other than live content (e.g., video data including pre-recorded video and pre-recorded audio) to the user device 12. The technology described in the embodiment can be applied to this variation as well, and the quality of the user experience of viewing the content can be improved.

In the above embodiment, at least one of the multiple live contents distributed from the video distribution server 16 via the Internet 18 is a radio program transmitted via a radio broadcasting network. As a variation, at least one of the multiple live contents distributed from the video distribution server 16 via the Internet 18 may be a television program transmitted via a television broadcasting network. Even in this variation, the technology described in the embodiment can be applied, and interactive communication between the performers and viewers of a television program for public broadcasting can be realized.

Any combination of the above-mentioned examples and modifications is also useful as an embodiment of the present disclosure. A new embodiment resulting from the combination will have the combined effects of each of the examples and modifications. It will also be understood by those skilled in the art that the functions to be performed by each of the constituent elements recited in the claims can be realized by each of the constituent elements shown in the examples and modifications alone or in combination with each other.

10 Communication system, 12 User device, 14 Camera device, 16 Video distribution server, 22 Performer terminal, 24 Broadcast device, 40 Content acquisition unit, 42 Display control unit, 44 Audio output control unit.

Claims (4)

An acquisition unit that acquires data of a plurality of contents distributed from a server;
a display control unit that displays one of the plurality of contents on a main screen having a relatively large size and displays the other contents on a sub-screen having a relatively small size;
an audio output control unit that switches the audio to be output from a speaker from the audio of the content displayed on the main screen to the audio of the content displayed on the sub-screen in response to a user operation;
Equipped with
the display control unit changes the first content displayed on the main screen to be displayed on a full screen in response to a user operation;
when the first content displayed on the main screen is changed to a full-screen display while the audio of the second content displayed on the sub-screen is being output, the audio output control unit continues to output the audio of the non-displayed second content.
Information processing device. the display control unit changes the content that has been displayed on the main screen to be displayed on the sub-screen in response to a user operation, and changes the content that has been displayed on the sub-screen to be displayed on the main screen.
2. The information processing device according to claim 1. when data of all of the plurality of contents has been acquired, the display control unit starts displaying the plurality of contents on the main screen and the sub-screen.
The information processing device according to claim 1 . A function to obtain data of multiple contents distributed from the server,
a function of displaying one of the plurality of contents on a main screen having a relatively large size and displaying the other contents on a sub-screen having a relatively small size;
a function of switching the sound output from a speaker from the sound of the content displayed on the main screen to the sound of the content displayed on the sub-screen in response to a user operation;
A computer program for causing a computer to realize the above,
The function of displaying further changes the first content displayed on the main screen to be displayed on a full screen in response to a user operation;
the switching function continues outputting the audio of the non-displayed second content when the first content displayed on the main screen is changed to a full-screen display while the audio of the second content displayed on the sub-screen is being outputted;
Computer program.

Images