JP7790987B2 - Video distribution device, display control device, video distribution system, video distribution method and program - Google Patents

Description

The present invention relates to a video distribution device, a display control device, a video distribution system, a video distribution method, and a program.

In recent years, communication bandwidth has expanded, and services exist that allow multiple videos to be viewed simultaneously. Users can freely change the position, size, etc. of the videos displayed simultaneously, allowing them to select a variety of views based on their interests. Patent Document 1 describes technology that allows users to select and view any video from multiple videos that are distributed simultaneously.

Japanese Patent Application Laid-Open No. 2019-33362

However, in the above-mentioned conventional examples, users are not necessarily familiar with selecting the display method, i.e., when and how to display and view multiple videos, and may not be able to fully enjoy the benefits of being able to display multiple videos.

The present invention aims to provide a video distribution device and video distribution system that allows users to enjoy the benefits of being able to display multiple videos, even if they are not familiar with displaying multiple videos.

The present invention employs the following configuration:
A video distribution device that distributes videos to a plurality of display control devices,
an acquisition means for acquiring a plurality of moving images generated by simultaneously capturing images using a plurality of imaging devices;
a distribution means for distributing the plurality of moving images as a set to the plurality of display control devices;
a receiving means for receiving, from the first display control device, display mode information indicating how the plurality of moving images included in the set distributed to the first display control device are arranged and played back by a user of the first display control device;
a transmitting means for transmitting the display mode information received from the first display control device to a second display control device different from the first display control device in response to a request from the second display control device,
The second display control device is a video distribution device that displays and plays the plurality of videos included in the set simultaneously on a screen in an arrangement according to the display mode information.

The present invention also employs the following configuration:
A display control device,
a receiving means for receiving, as a set, a plurality of moving images generated by simultaneously capturing images using a plurality of imaging devices from a moving image distribution device;
a display/playback means for controlling the plurality of moving images to be simultaneously displayed on a screen and played back;
a change means for changing the arrangement of the plurality of moving images included in the set in response to a user operation;
a generating means for generating display mode information indicating how the plurality of moving images included in the set are arranged and played back by the user ;
and a transmitting unit configured to transmit the generated display mode information to the video distribution device.

The present invention also employs the following configuration:
A display control device,
a receiving means for receiving, as a set, a plurality of moving images generated by simultaneously capturing images using a plurality of imaging devices from a moving image distribution device;
a display/playback means for controlling the plurality of moving images to be simultaneously displayed on a screen and played back;
a requesting means for requesting, from the video distribution device, display mode information indicating how the plurality of videos included in the set distributed to the other display control device are arranged and played back by a user of the other display control device;
an acquisition means for acquiring the requested display mode information;
and a display playback means for controlling the display and playback of the plurality of moving images included in the set simultaneously on the screen in an arrangement according to the display mode information.

The present invention also employs the following configuration:
A video distribution system including a video distribution device and a plurality of display control devices,
The video distribution device includes:
an acquisition means for acquiring a plurality of moving images generated by simultaneously capturing images using a plurality of imaging devices;
a distribution means for distributing the plurality of moving images as a set to the plurality of display control devices;
a receiving means for receiving, from a first display control device among the plurality of display control devices , display mode information indicating how the plurality of videos included in the set distributed to the first display control device are arranged and played back by a user of the first display control device;
a transmitting means for transmitting the display mode information received from the first display control device to the second display control device in response to a request from the second display control device, the second display control device being different from the first display control device, among the plurality of display control devices;
The display control device
a display/playback means for controlling the plurality of moving images to be simultaneously displayed and played back on a screen;
The first display control device
a change means for changing the arrangement of the plurality of moving images included in the set in response to a user operation;
a generating means for generating display mode information indicating how the plurality of moving images included in the set are arranged and played back by the user ;
a transmitting means for transmitting the generated display mode information to the video distribution device;
The second display control device
receiving means for receiving the display mode information from the video distribution device;
The display playback means of the second display control device controls the display and playback of the plurality of moving images included in the set on a screen at the same time in an arrangement according to the display mode information.
The video distribution system is characterized by the above.

According to the present invention, when a user watches multiple videos, they can enjoy the benefits of being able to display multiple videos even if they are not familiar with displaying multiple videos.

1 is a diagram illustrating an example of the overall configuration of a video distribution system according to a first embodiment. 1 is a diagram illustrating a hardware configuration of a video distribution device according to a first embodiment. 1 is a diagram illustrating an external appearance of a display control device according to a first embodiment. FIG. 1 is a diagram illustrating a hardware configuration of a display control device according to a first embodiment. 10 is a flowchart illustrating a video distribution process according to the first embodiment. 10 is a flowchart illustrating a display registration process according to the first embodiment. 4 is a flowchart showing a playback process of the display control device according to the first embodiment. FIG. 4 is a diagram showing a display mode management table according to the first embodiment. 3 is a diagram illustrating an example of a video display in the display control device according to the first embodiment.

The following describes preferred embodiments of the present invention with reference to the drawings. Note that, unless otherwise specified, the numerical values and other examples used to concretize the description are not limited to these. Furthermore, the present invention is not limited to the following embodiments, and can be modified as appropriate without departing from the spirit of the invention. For example, the configuration of the device to which the present invention is applied and the various configurations of the following embodiments may be modified or changed as appropriate depending on various conditions.

<Embodiment 1>
1 is a diagram illustrating an example of the overall configuration of a video distribution system according to embodiment 1. In this embodiment, multiple videos are distributed from a video distribution device 21, and the viewing method (viewing method) for each video in a first display control device 31 is shared with a second display control device 32. A user of the second display control device 32 can display and view multiple videos in the same way as the first display control device 31. This allows a user who is not accustomed to viewing multiple videos simultaneously or who is not familiar with the operation method to enjoy the benefit of being able to display multiple videos simultaneously.

The video distribution system according to this embodiment includes an image capture device 11, an image capture device 12, a video distribution device 21, a first display control device 31, and a second display control device 32. While FIG. 1 shows two image capture devices and two display control devices as an example, the system configuration is not limited to this and may include more image capture devices and display control devices. Video captured by each image capture device is distributed to each display control device via the video distribution device 21. The devices are connected via any network, and communication may be either wired or wireless, using any communication standard.

The imaging devices 11 and 12 are imaging devices that capture video (moving images), such as digital video cameras. The video captured by each imaging device is, for example, multi-view video obtained by simultaneously capturing the same location from different directions (orientations). When the multi-view video is displayed and played back, viewers can view the same scene or the same subject from different viewpoints. Alternatively, the video captured by each imaging device may be, for example, multi-directional video obtained by simultaneously capturing images from different directions from the same location. When the multi-directional video is displayed and played back, viewers can view the video from different viewpoints from a single location. The video captured by each imaging device is transmitted to the video distribution device 21. Note that each imaging device may transmit the video being captured to the video distribution device 21 in real time, or may transmit recorded video.

The video distribution device 21 distributes video data including a set of multiple videos received from each imaging device to the first display control device 31 and the second display control device 32. The video distribution device 21 transmits the videos received from each imaging device in real time. Note that pre-recorded videos may also be transmitted. Furthermore, the video distribution device 21 may distribute the videos distributed in real time so that they can be viewed even after the real-time distribution.

The video distribution device 21 also has a function of transmitting and receiving display mode information, which is information relating to the display mode of the multiple videos included in one set in each display control device, and sharing the viewing method of the multiple videos among multiple devices. The display mode information is information that indicates how each frame of each of the multiple videos is displayed and played on each of the multiple devices. The display mode information includes at least one of the following: whether each of the multiple videos is displayed or not, its display position on the screen, the size displayed on the screen, the range displayed (display angle of view), the order of overlap with other videos (overlapping order), device identification information, and playback speed. The display mode information also includes video identification information for identifying the corresponding set of videos. For example,
When one video is displayed full screen, the display mode information may store data indicating that "the video is displayed full screen, and the remaining videos are not displayed." Furthermore, when one video is displayed full screen, the display mode information may store data indicating that "the video is displayed full screen and in the foreground." Furthermore, the display mode information does not need to store data indicating the display mode of all of the multiple videos included in a set, but it is sufficient to store data indicating the display mode of at least one of the multiple videos included in a set.

The first display control device 31 and the second display control device 32 are devices (electronic devices) connected to the video distribution device 21 and capable of simultaneously displaying multiple videos included in a set received from the video distribution device 21. Each device may be, for example, a smartphone, a tablet terminal, a mobile computer, or a television. Each device can change the display layout of each video included in the set (e.g., the displayed area of each video, the display position of each video on the screen, the size of each video on the screen, and whether each video is displayed or not). Each device may also be capable of changing the playback speed of each video. In this embodiment, the first display control device 31 corresponds to the "first electronic device" and is described as a device that transmits display mode information to the video distribution device 21. The second display control device 32 corresponds to the "second electronic device" and is described as a device that references the viewing method of multiple videos on other devices. Note that one device may have both the function of transmitting display mode information and the function of receiving and referencing display mode information from other devices.

Figure 2 is a diagram showing the hardware configuration of the video distribution device 21 according to this embodiment. The CPU 101, memory 102, non-volatile memory 103, display 104, operation unit 105, recording medium I/F 106, external I/F 108, communication I/F 109, and audio output unit 111 are connected to the internal bus 150. Each component can exchange data with each other via the internal bus 150.

CPU 101 is a control unit that controls the entire video distribution device 21 and has at least one processor or circuit. CPU 101 controls each part of the video distribution device 21 in accordance with the programs stored in non-volatile memory 103, using memory 102 as work memory.

Memory 102 is a work memory used for temporary storage by CPU 101, such as RAM.

Non-volatile memory 103 stores image data, audio data, various programs for operating CPU 101, and the like. Non-volatile memory 103 may be, for example, flash memory or ROM.

The display 104 is a display unit that displays images and GUI screens that constitute a GUI (Graphical User Interface) under the control of the CPU 101. Note that the video distribution device 21 itself may only have an interface for outputting video signals to be displayed on the display 104, and the display 104 may be configured as an external monitor (such as a television).

The operation unit 105 is an input device for accepting user operations. The operation unit 105 includes, for example, a character information input device such as a keyboard, a pointing device such as a mouse or touch panel, buttons, dials, a joystick, a touch sensor, a touch panel, etc.

The recording medium I/F 106 reads data from and writes data to the recording medium 107 under the control of the CPU 101.

The recording medium (recording unit) 107 records data such as the video data distributed by the video distribution device 21 and display mode information for each device. The recording medium 107 can be, for example, a memory card, CD, DVD, or HDD.

The external I/F 108 is an interface that connects to external devices via a wired cable or wirelessly, and performs input and output of video and audio signals.

The communication I/F 109 is an interface for communicating with external devices and the Internet 110 to send and receive various data such as files and commands.

The audio output unit 111 outputs audio from video and music data, operation sounds, ringtones, various notification sounds, etc. The audio output unit 111 includes an audio output terminal 111a for connecting earphones, etc., and a speaker 111b. Audio may also be output using a device connected via wireless communication.

Figure 3A is an external view of the first display control device 31. The display 205 is a display unit that displays videos, images, and various information received from the video distribution device 21. The display 205 is integrally configured with the touch panel 206a, and is capable of detecting touch operations on the display surface of the display 205. As shown in the figure, the operation unit 206 includes the touch panel 206a and operation units 206b, 206c, 206d, and 206e. The operation unit 206b is a power button that accepts operations to turn the power of the first display control device 31 on and off. The operation units 206c and 206d are volume buttons that increase and decrease the volume of the audio output from the audio output unit 222. The operation unit 206e is a home button for displaying the home screen on the display 205. The audio output terminal 212a is an earphone jack that outputs audio to earphones, an external speaker, etc. The speaker 212b is a built-in speaker that produces audio.

Here, the CPU 201 can detect the following operations or states on the touch panel 206a:

A finger or pen that has not been touching the touch panel 206a touches the touch panel 206a again, that is, the start of touching (hereinafter referred to as touch-down).
The touch panel 206a is in a state where a finger or a pen is touching the touch panel 206a (hereinafter referred to as Touch-On).
A finger or pen is moved while touching the touch panel 206a (hereinafter referred to as Touch-Move).
The finger or pen that has been touching the touch panel 206a is removed from the touch panel 206a, that is, the touch ends (hereinafter referred to as "touch-up").
A state in which nothing is touching the touch panel 206a (hereinafter referred to as Touch-Off)

When a touch down is detected, a touch on is also detected at the same time. After a touch down, a touch on is usually continued to be detected unless a touch up is detected. If a touch move is detected, a touch on is also detected at the same time. Even if a touch on is detected, a touch move is not detected unless the touch position has moved. When it is detected that all fingers or pens that were touching have touched up, a touch off is detected.

These operations and states, as well as the position coordinates of the finger or pen touching the touch panel 206a, are notified to the CPU 201 via the internal bus, and the CPU 201 determines what operation (touch operation) was performed on the touch panel 206a based on the notified information. For touch moves, the movement direction of the finger or pen moving on the touch panel 206a can also be determined for each vertical and horizontal component on the touch panel 106a based on changes in the position coordinates. If a touch move of a predetermined distance or more is detected, it is determined that a slide operation has been performed. An operation in which a finger is touched on the touch panel 206a, moved quickly for a certain distance, and then released is called a flick. In other words, a flick is an operation in which the finger is quickly traced across the touch panel 206a as if flicking it. If a touch move of a predetermined distance or more at a predetermined speed or more is detected and a touch up is then detected, it can be determined that a flick has been performed (it can be determined that a flick occurred following a slide operation). Furthermore, a touch operation in which multiple points (for example, two points) are touched simultaneously and the touch positions are brought closer together is called a pinch in, while a touch operation in which the touch positions are moved farther apart is called a pinch out. Pinch out and pinch in are collectively referred to as a pinch operation (or simply pinch). The touch panel 206a may be of any of a variety of touch panel types, including resistive film, capacitive, surface acoustic wave, infrared, electromagnetic induction, image recognition, and optical sensor types. There are types that detect a touch by contact with the touch panel, and types that detect a touch by the approach of a finger or pen to the touch panel, but either type is acceptable.

Figure 3B is a diagram showing the hardware configuration of the first display control device 31 according to this embodiment. Connected to the internal bus 250 are the CPU 201, memory 202, non-volatile memory 203, image processing unit 204, display 205, operation unit 206, recording medium I/F 207, external I/F 209, communication I/F 210, and audio output unit 212. Here, we will omit the configuration common to the video distribution device and the configuration described in the external view above, and will focus on the image processing unit 204.

Under the control of the CPU 201, the image processing unit 204 performs various image processing operations on images stored in the non-volatile memory 203 or recording medium 208, video signals acquired via the external I/F 209, and images acquired via the communication I/F 210. The image processing operations performed by the image processing unit 204 include A/D conversion, D/A conversion, image data encoding, compression, decoding, enlargement/reduction (resizing), noise reduction, and color conversion. The image processing unit 204 may be configured as a dedicated circuit block for performing specific image processing operations. Alternatively, depending on the type of image processing, the CPU 201 may perform image processing according to a program without using the image processing unit 204. The second display control device 32 also has a hardware configuration similar to that of the first display control device 31. Hereinafter, the configuration of Figure 3 will be referred to as the configuration of the second display control device 32 as appropriate.

(Video distribution processing)
4 is a flowchart relating to the video distribution device 21 in this embodiment. Hereinafter, the process of distributing multiple videos and the process of sharing a viewing method by the video distribution device 21 will be described with reference to FIG.

In step S401, the CPU 101 acquires video data stored in the recording medium 107 via the recording medium I/F 106, and starts distributing the video data to each device including the first display control device 31 and the second display control device 32. In this embodiment, the CPU 101 also functions as a video distribution unit that distributes a set of multiple videos simultaneously captured by multiple imaging devices. The multiple videos included in one set are associated with one common piece of video identification information,
The multiple videos are distributed together with the video identification information.

In step S402, CPU 101 determines whether display mode information, which is information regarding the display mode for each specified frame of each of the multiple videos included in a set, has been received from any of the destination devices. In this embodiment, CPU 101 also functions as a display mode information acquisition unit that acquires display mode information. If display mode information has been received, the process proceeds to step S403; if not, the process proceeds to step S404. Note that, as an example, this embodiment describes the processing when display mode information is received from the first display control device 31, but the device from which the video distribution device 21 receives display mode information is not limited to the first display control device 31. Furthermore, the device from which the video distribution device 21 can receive display mode information is not limited to one, but may be multiple, and the information may be received from multiple devices.

The display mode information includes at least one of the following: whether each of the multiple videos is displayed or not, their display position on the screen, their size on the screen, the area on the screen where they are displayed, the overlapping relationship with other videos, device identification information (identification information of the electronic device), and playback speed. Device identification information is assigned to each device that receives the video distribution device 21 and is used to identify the device.

In step S403, the CPU 101 associates device identification information and display mode information for each device (electronic device) received in step S402 and records them on the recording medium 107. That is, in this embodiment, the recording medium 107 also functions as a recording unit that records the display mode information acquired by the display mode information acquisition unit (CPU 101). Here, the recording unit (recording medium 107) preferably records the display mode information in association with a base frame for managing a set of multiple videos. In this embodiment, the storage unit stores the display mode information in association with a time code value indicating the elapsed time of the video distributed by the video distribution device 21. Note that the base frame is not limited to this; for example, the number of frames of a predetermined video among the multiple videos may be used. Furthermore, the number of frames of independent frames (I-frames) generated by intra-frame coding processing and that do not require information on other video frames during decoding may be used as the base frame. In this way, by recording information for managing multiple videos in association with the display mode information, it is possible to record information on frames whose playback speed has been changed or frames that are being repeatedly viewed on each device. The display mode information may also include data indicating the preferences of the device user (such as which subject in the video the user is paying attention to, what aspect of the video the user is interested in, etc.).

FIG. 7 is a display mode management table showing an example of display mode information recorded on the recording medium 107. The display mode management table has fields 701 to 709. Field 701 stores a time code for managing multiple videos. The time code value is expressed in units of "hour/minute/second/frame." Field 702 stores the number of frames for each video. Field 703 stores the overlapping order, which indicates the chronological order (order) of each video. Here, the smaller the overlapping order value, the more the video is displayed in the foreground. Field 704 stores the display/non-display status of each video. Field 705 stores the display position, which indicates the position on the screen where each video is displayed. Here, the display position is indicated by the coordinates of the upper left corner of each video on the screen. Note that the display position may also be indicated using the coordinates of the center of each video or other coordinates. Field 706 stores a value indicating the size at which each video is displayed on the screen. Here, the size of each video is indicated by width and height. The size of each video may be represented by other values, such as the upper left and lower right coordinates of each video. Fields 707 and 708 store information about the display angle of view, indicating which portion of each video is displayed on the screen. Field 707 stores the position of the portion of each video displayed on the screen. Here, the position of the portion of each video displayed on the screen is indicated by the upper left coordinate of the displayed portion. Field 708 stores the size of each video displayed on the screen. Note that, like fields 705 and 706, the values shown in fields 707 and 708 may be different values. Field 709 stores device identification information that can identify each device. The device identification information may be, for example, a MAC address, but may also be other information such as an account name on the Internet or a device serial number. The video distribution device 21 records the display mode information received from each device on the recording medium 107.

In step S404, the CPU 101 determines whether or not a request to acquire a list of other devices that share display mode information has been received from the second display control device 32. In this embodiment, the CPU 101 also serves as an acquisition request receiving unit that accepts a request to acquire display mode information. If a request has been received, the process proceeds to step S405; if not, the process proceeds to step S406.

In step S405, the CPU 101 obtains the identification information of the device that received the display mode information stored on the recording medium 107, creates a list of devices, and records the list in the memory 102.

In step S406, the CPU 101 retrieves the device list recorded in step S405 from the memory 102 and transmits it to the second display control device 32 via the communication I/F 109. That is, in this embodiment, the CPU 101 also functions as an information transmission unit that transmits the identification information recorded in the recording unit to the second electronic device (second display control device 32). Here, the transmitted list may include not only the identification information of the device that received the display mode information, but also evaluation information by other users who referenced the display mode information of multiple videos. The evaluation information may be, for example, the number of users (second electronic devices) who referenced the viewing method of the device. Alternatively, the number of times each display mode information is selected from the list and an acquisition request is received may be counted and used as the evaluation information. The evaluation information may also be an index indicating a favorability rating, such as the number of high ratings or scores, given by users who referenced the display mode information. Furthermore, the identification information of devices may be selected so that the transmitted list includes the top multiple users with the highest ratings. This allows the user of the second display control device 32 to more easily select device identification information that is useful for reference.

In step S407, the CPU 101 determines whether a request to acquire display mode information for a device specified in the list has been received from the second display control device 32 connected via the communication I/F 109. In this embodiment, an example will be described in which the first display control device 31 is specified and the acquisition request includes identification information for the first display control device 31. If an acquisition request has been received, the process proceeds to step S408; if not, the video distribution process ends.

In step S408, the CPU 101 transmits display mode information of the first display control device 31 to the second display control device 32 via the communication I/F 109. In this manner, the information transmission unit (CPU 101) may transmit display mode information in response to an acquisition request. In this embodiment, the information transmission unit transmits display mode information of the electronic device corresponding to the identification information included in the acquisition request to the second display control device 32. The transmitted display mode information may include, for example, information reproducing the display mode of the first display control device 31 for the current frame being viewed by the second display control device 32. Note that the display mode information may include display mode information for not only the current frame but also previous and future frames. This allows the user of the second display control device 32 to view multiple videos in the same way as the first display control device 31. Furthermore, for example, the display mode information transmitted in step S408 may include information including a list display that associates preview images showing the layout of the multiple videos with the time when the layout was changed. This allows the user of the second display control device 32 to know in advance when and how the videos will be displayed.

(Display mode information registration process)
FIG. 5 is a flowchart showing a display registration process for transmitting, to the video distribution device 21, display mode information of the first display control device 31 (first electronic device) capable of simultaneously displaying multiple videos.

In step S501, the CPU 201 performs initialization processing. This initialization processing includes disabling the sharing setting of display mode information and initializing the display mode.

In step S502, the CPU 201 receives multiple videos from the video distribution device 21 via the communication I/F 210. In this embodiment, the CPU 201 also functions as a video receiving unit that receives multiple videos.

In step S503, the CPU 201 plays the multiple videos received in step S502. The multiple videos are displayed on the display 205 (display unit) using the initial settings of the first display control device 31.

The CPU 201 changes the image display mode in response to, for example, a user operation on the operation unit 206 (an operation to change the image display mode). In step S504, the CPU 201 determines whether an operation to change the image display mode has been performed. In this embodiment, the CPU 201 also functions as a display control unit that can simultaneously display multiple images on the display unit and switch between the display modes of the multiple images on the display unit. If a change operation has been performed, the process proceeds to step S505; if a change operation has not been performed, the process proceeds to step S509.

In step S505, the CPU 201 obtains the sharing settings for the display mode information.

In step S506, the CPU 201 determines whether the "display mode information sharing setting" acquired in step S505 is valid or invalid. In this embodiment, the operation unit 206 also functions as a sharing setting acceptance unit that accepts the display mode information sharing setting from the user. If the sharing setting is "valid," the process proceeds to step S507; if it is "invalid," the process proceeds to step S509.

In step S507, the CPU 201 acquires display mode information for each frame of the multiple videos acquired in step S502. The display mode information includes at least one of the following: whether each of the multiple videos is displayed or not, its display position on the screen, its size on the screen, its range on the screen (display angle of view), its overlapping relationship with other videos (overlapping order), device identification information, and playback speed. It is preferable that the frames from which display mode information is acquired include frames from the current point onward that are being played back by the first display control device 31. It is also possible to acquire display mode information for frames prior to the current point. In this case, a selection operation may be further set to enable acquisition of display mode information for previous frames in response to a user selection.

In step S508, the CPU 201 transmits display mode information associated with each frame to the video distribution device 21 via the communication I/F 210. In this embodiment, the CPU 201 also functions as a display mode information transmission unit that transmits the display mode information to the video distribution device 21. The transmitted display mode information may be information associated with a frame that serves as a base for managing multiple videos. In this embodiment, the CPU 201 transmits display mode information associated with a time code value to the video distribution device 21. The CPU 201 sequentially transmits the display mode information to the video distribution device 21. The CPU 201 may transmit the display mode information all at once or for each predetermined frame. The predetermined frame may be determined based on the elapsed time or the number of I frames of the received video. In this way, the display mode information transmission unit transmits the display mode information to the video distribution device 21 if the sharing setting acquired in step S506 is valid. This allows the display mode information to be shared with other devices while reflecting the user's intention regarding the sharing of the display mode information. Furthermore, the display mode information may be set to a common value for each predetermined frame.

In step S509, the CPU 201 determines whether an operation to change the "display mode information sharing settings" has been performed on the operation unit 206. If an operation to change the settings has been performed, the process proceeds to step S510; if no changes have been made, the process proceeds to step S511.

In step S510, the CPU 201 determines the result of the change made in step S509. If the change result is "valid," the process proceeds to step S511; if the change result is "invalid," the process proceeds to step S512.

In step S511, the CPU 201 changes the setting for "Display mode information sharing setting" from "Disabled" to "Enabled," and then proceeds to step S513.

In step S512, the CPU 201 changes the setting for "Display mode information sharing setting" from "Enabled" to "Disabled," and then proceeds to step S513.

In step S513, the CPU 201 determines whether an end operation has been performed on the operation unit 206. If an end operation has been performed, the display mode information registration process ends.

(Display mode information reception process)
FIG. 6 is a flowchart of the playback process of the second display control device 32.

In step S601, the CPU 201 initializes the display mode in the second display control device.

In step S602, the CPU 201 receives a set of multiple videos from the video distribution device 21 via the communication I/F 210. That is, in this embodiment, the CPU 201 also functions as a video receiving unit that receives multiple videos.

In step S603, the CPU 201 plays back the multiple images received in step S502. The multiple images included in one set are displayed on the display 205 (display unit) using the initial settings of the first display control device 31.

In step S604, the CPU 201 determines whether an operation to obtain display mode information for another device has been performed on the operation unit 206. If an operation to obtain display mode information for another device has been performed, the process proceeds to step S605; if not, the process proceeds to step S610.

In step S605, the CPU 201 sends a request to obtain a list of other devices that share the display mode information to the video distribution device 21 via the communication I/F 210. That is, in this embodiment, the CPU 201 also functions as an acquisition request sending unit that sends a request to obtain display mode information to the video distribution device 21.

In step S606, the CPU 201 receives a list of other devices that share the display mode information from the video distribution device 21 via the communication I/F 210. That is, in this embodiment, the CPU 201 also functions as an information receiving unit that receives identification information of other devices (electronic devices) from which the video distribution device 21 has obtained display mode information.

In step S607, the operation unit 206 receives a selection instruction from the user to select one of the identification information included in the list acquired in step S606. That is, in this embodiment, the operation unit 206 also functions as a selection instruction receiving unit. Here, the list of other devices received in step S606 may be presented to the user, for example, by displaying a list of user names corresponding to the device's identification information on the display 205. Note that the display method is not limited, and for example, users with a large number of shared users, users with high ratings from other users, or users who have previously been referenced may be prominently displayed. This allows the user to refer to the information when making a selection instruction in step S607. The acquisition request sending unit (CPU 201) sends a request to acquire display mode information for the device selected by the operation unit 206 to the video distribution device 21 via the communication I/F 210. The transmitted acquisition request includes the identification information accepted by the selection instruction receiving unit. In this embodiment, it is assumed that the first display control device 31 is selected.

In step S608, the CPU 201 receives display mode information for the first display control device 31 selected in step S607 from the video distribution device 21 via the communication I/F 210. In this way, the information receiving unit (CPU 201) may receive display mode information in response to an acquisition request, and in this embodiment, it receives display mode information for the electronic device corresponding to the identification information included in the acquisition request from the video distribution device 21. The received display mode information includes whether or not multiple videos are displayed (display/non-display), their display position on the screen, their displayed size, the displayed range (display angle of view), the overlapping relationship (overlapping order), and device identification information. Note that the display mode information may include display mode information for frames from the current frame onward that the second display control device 32 is viewing, as well as display mode information for frames prior to the reception of the request to acquire display mode information.

In step S609, the CPU 201 plays back the multiple videos using the display mode information received in step S608. That is, in this embodiment, the CPU 201 is capable of switching the display mode of the multiple videos and also functions as a display control unit that displays the multiple videos on the display unit based on the display mode information received by the information receiving unit. Here, the display mode of the multiple videos in the second display control device 32 may be controlled, for example, based on the received viewing information so that they are played back in the same display mode as in the first display control device 31. Furthermore, after playback begins using the display mode information of the first display control device 31, the display mode of the multiple videos may be further changed in response to a user selection operation. Furthermore, after playback in the display mode of a selected device, it may be possible to reselect another device from the list. This allows the user to watch the multiple videos in a display mode more suited to the user's preferences. Note that if the received display mode information includes a frame for which display mode information is not shared, the display mode may be controlled, for example, to maintain the display mode of the immediately preceding frame.

In step S610, the CPU 201 determines whether an end operation has been performed on the operation unit 206. If an end operation has been performed, the playback process ends.

Figure 8 illustrates an example of video distributed by the video distribution device 21 and multiple videos displayed on the second display control device 32 in this embodiment. Figures 8(a) and 8(b) show video 1 and video 2 distributed by the video distribution device 21. Figures 8(c) to 8(f) are display examples of images displayed on the second display control device 32 for each time code in Figure 7. Note that Figures 8(c) to 8(f) are merely examples, and the layouts that can be displayed on each device are not limited to these. In the following description, the resolutions of Figures 8(a) and 8(b) and the resolution of the display 205 of the display control device 32 are each assumed to be 100x100.

Figure 8 (c) shows an example display when the time code is "12:34:56:01." Multiple videos distributed from the video distribution device 21 are displayed on the display 205 of the second display control device 32 according to the settings of record 710 in the table of Figure 7. As can be read from the values of fields 702 to 706, video 1 displays an image with 100 frames in an area of width 50 and height 100 from coordinates (X, Y) = (0, 0) on the display 205. Furthermore, as can be read from the values of fields 707 and 708, of the image with 100 frames, an area of width 50 and height 100 in video 1 is displayed from coordinates (X, Y) = (0, 0). Similarly, video 2 is displayed in an area of display 205 with a width of 50 and a height of 100 from coordinates (X, Y) = (50, 0) in video 2, of the image with 50 frames, with a width of 50 and a height of 100.

Figure 8 (d) shows an example display when the time code is "12:34:56:02". In this case, according to the settings in record 711, an image with 101 frames for video 1 and an image with 50 frames for video 2 are displayed on display 205. Here, the smaller the value of the overlap order in field 703, the more the image is displayed in front. Therefore, the overlap order value for each image is "1" for video 1 and "2" for video 2, so video 1 is displayed in front of video 2. Furthermore, the values in fields 706 and 708 for video 1 indicate that video 1 with a width of 100 and a height of 100 should be displayed in an area of display 205 with a width of 50 and a height of 50. Therefore, video 1 is displayed in a reduced size on display 205.

Figure 8 (e) shows an example of the display when the time code is "12:34:56:03". In this case, according to the settings in record 712, an image with 102 frames of video 1 and an image with 51 frames of video 2 are displayed on display 205. Furthermore, the overlay order values for each video are "2" for video 1 and "1" for video 2, so video 2 is displayed in front of video 1. The values in fields 706 and 708 for video 2 indicate that video 2 with a width of 25 and a height of 25 is to be displayed in an area of display 205 with a width of 50 and a height of 50. Therefore, video 2 is enlarged and displayed on display 205.

8(f) shows an example of a display when the time code is "12:34:56:04." In this case, in accordance with the setting of record 713, the "Show/Hide" field 704 for Video 2 is set to "Hide," so only Video 1 is displayed on the display 205.

Furthermore, the values in field 702 for Video 1 and Video 2 indicate that Video 1 and Video 2 are being played with different frames and at different frame rates. In this way, the playback speed can be changed and information such as the frames being replayed can be shared.

As described above, in this embodiment, multiple videos are distributed from a video distribution device, and the viewing method for the multiple videos is shared between devices capable of simultaneously displaying multiple videos. This allows users who are unfamiliar with the viewing and operation methods for multiple videos to view multiple videos in the same way as other users, and enjoy the benefits of being able to display multiple videos.

<Modification>
In this embodiment, the size of each video (FIG. 7, fields 706, 708) is expressed by width and height, but the size may be expressed using other values, such as a combination of the top left coordinate and the bottom right coordinate within each video, or a combination of the bottom left coordinate and the top right coordinate. Furthermore, the display method for multiple videos is not limited to a rectangle, and they may be displayed in other shapes (circle, ellipse, etc.). Furthermore, the information shared as the above-described display mode information is merely exemplary, and other information may also be included. For example, information such as the movement and animation when switching the size and arrangement of multiple videos may also be included in the display mode information, and the movement and animation when switching may be controlled so that they are displayed in the same way on other devices.

<Other embodiments>
Although the present invention has been described in detail above based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments may be combined as appropriate.

The present invention can also be realized by a process in which a program implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a recording medium, and one or more processors in the system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. Therefore, the program code itself, supplied to and installed on a computer to implement the functional processing of the present invention, also embodies the present invention. In other words, the computer program itself for implementing the functional processing of the present invention is also included in the present invention. In this case, the program may take any form, such as object code, a program executed by an interpreter, or script data supplied to an OS, as long as it has the program functionality. Recording media for providing the program may include, for example, a hard disk, a magnetic recording medium such as a magnetic tape, an optical/magneto-optical recording medium, or a non-volatile semiconductor memory. Another possible method for providing the program is to store the computer program implementing the present invention on a server on a computer network, and then download the computer program to a connected client computer.

21: Video distribution device, 31: First display control device, 32: Second display control device, 101: CPU, 107: Recording medium

Claims (16)

A video distribution device that distributes videos to a plurality of display control devices,
an acquisition means for acquiring a plurality of moving images generated by simultaneously capturing images using a plurality of imaging devices;
a distribution means for distributing the plurality of moving images as a set to the plurality of display control devices;
a receiving means for receiving, from the first display control device, display mode information indicating how the plurality of moving images included in the set distributed to the first display control device are arranged and played back by a user of the first display control device;
a transmitting means for transmitting the display mode information received from the first display control device to a second display control device different from the first display control device in response to a request from the second display control device,
The second display control device is a video distribution device characterized in that it displays and plays the plurality of videos included in the set simultaneously on a screen in an arrangement according to the display mode information. The video distribution device described in claim 1, characterized in that the multiple videos included in the set are generated by using the multiple imaging devices to capture images of a specified location from different directions. The video distribution device described in claim 1, characterized in that the multiple videos included in the set are generated by using the multiple imaging devices to capture images in different directions from a predetermined location. The video distribution device according to any one of claims 1 to 3, characterized in that the second display control device requests one of a plurality of pieces of display mode information including the display mode information received from the first display control device , and the transmitting means transmits the requested display mode information to the second display control device. The video distribution device according to any one of claims 1 to 4, further comprising a counting means for counting the number of times each of a plurality of pieces of display mode information including the display mode information received from the first display control device is requested by the second display control device. A video distribution device as described in any one of claims 1 to 5 , characterized in that the display mode information indicates, for at least one of the multiple videos included in the set , at least one display mode among the display position on the screen, the size displayed on the screen, the range displayed on the screen, and the overlap relationship with other videos. The video distribution device described in claim 6, characterized in that the display mode information indicates the display mode for each frame of at least one of the multiple videos included in the set. A display control device,
a receiving means for receiving, as a set, a plurality of moving images generated by simultaneously capturing images using a plurality of imaging devices from a moving image distribution device;
a display/playback means for controlling the plurality of moving images to be simultaneously displayed on a screen and played back;
a change means for changing the arrangement of the plurality of moving images included in the set in response to a user operation;
a generating means for generating display mode information indicating how the plurality of moving images included in the set are arranged and played back by the user ;
A display control device comprising: a transmission means for transmitting the generated display mode information to the video distribution device. A display control device,
a receiving means for receiving, as a set, a plurality of moving images generated by simultaneously capturing images using a plurality of imaging devices from a moving image distribution device;
a display/playback means for controlling the plurality of moving images to be simultaneously displayed on a screen and played back;
a requesting means for requesting, from the video distribution device, display mode information indicating how the plurality of videos included in the set distributed to the other display control device are arranged and played back by a user of the other display control device;
an acquisition means for acquiring the requested display mode information;
a display playback means for controlling the display and playback of the plurality of moving images included in the set simultaneously on the screen in an arrangement according to the display mode information. A video distribution system including a video distribution device and a plurality of display control devices,
The video distribution device includes:
an acquisition means for acquiring a plurality of moving images generated by simultaneously capturing images using a plurality of imaging devices;
a distribution means for distributing the plurality of moving images as a set to the plurality of display control devices;
a receiving means for receiving, from a first display control device among the plurality of display control devices , display mode information indicating how the plurality of videos included in the set distributed to the first display control device are arranged and played back by a user of the first display control device;
a transmitting means for transmitting the display mode information received from the first display control device to the second display control device in response to a request from the second display control device, the second display control device being different from the first display control device, among the plurality of display control devices;
The display control device includes:
a display/playback means for controlling the plurality of moving images to be simultaneously displayed and played back on a screen;
The first display control device
a change means for changing the arrangement of the plurality of moving images included in the set in response to a user operation;
How the plurality of videos included in the set are arranged and played by the user
a generating means for generating display mode information indicating whether the
a transmitting means for transmitting the generated display mode information to the video distribution device;
The second display control device
receiving means for receiving the display mode information from the video distribution device;
The display playback means of the second display control device controls the display and playback of the plurality of moving images included in the set on a screen at the same time in an arrangement according to the display mode information.
A video distribution system characterized by: A video distribution method for distributing video to a plurality of display control devices, comprising:
an acquisition step of acquiring a plurality of moving images generated by simultaneously capturing images using a plurality of imaging devices;
a distribution step of distributing the plurality of moving images as a set to a plurality of display control devices;
a receiving step of receiving, from the first display control device, display mode information indicating how the plurality of videos included in the set distributed to the first display control device are arranged and played back by a user of the first display control device;
a transmitting step of transmitting the display mode information received from the first display control device to a second display control device different from the first display control device in response to a request from the second display control device,
A video distribution method characterized in that the second display control device simultaneously displays and plays the multiple videos included in the set on the screen in an arrangement according to the display mode information. A control method for a display control device, comprising:
a receiving step of receiving, as a set, a plurality of moving images generated by simultaneously capturing images using a plurality of imaging devices from a moving image distribution device;
a display/playback step of controlling the plurality of moving images to be simultaneously displayed and played on a screen;
a changing step of changing the arrangement of the plurality of moving images included in the set in response to a user operation;
a generating step of generating display mode information indicating how the plurality of moving images included in the set are arranged and played by the user ;
a transmitting step of transmitting the generated display mode information to the video distribution device. A control method for a display control device, comprising:
a receiving step of receiving, as a set, a plurality of moving images generated by simultaneously capturing images using a plurality of imaging devices from a moving image distribution device;
a display/playback step of controlling the plurality of moving images to be simultaneously displayed and played on a screen;
a request step of requesting, from the video distribution device, display mode information indicating how the plurality of videos included in the set distributed to the other display control device are arranged and played back by a user of the other display control device;
an acquisition step of acquiring the requested display mode information;
a display playback step of controlling the display control device so that the plurality of moving images included in the set are simultaneously displayed and played on the screen in an arrangement according to the display mode information. A program for causing a computer to execute each step of the video distribution method described in claim 11. A program for causing a computer to execute each step of the control method for a display control device described in claim 12. A program for causing a computer to execute each step of the control method for a display control device described in claim 13.