JP7761162B2 - Distribution control system, distribution control device, distribution control method, and program - Google Patents

Description

This disclosure relates to a distribution control system, a distribution control device, a distribution control method, and a program.

Six degrees of freedom (6DoF) stereoscopic video content, such as volumetric video and holograms, is well known. Delivering such content with high quality over a communications network requires not only advanced data compression technology and communications network/system load balancing technology, but also a mechanism for controlling the distribution of the content itself. In particular, a mechanism for dynamically controlling content distribution based on information such as the field of view of the client XR (VR/AR/MR/SR, etc.) device and the user's position in the virtual space is crucial.

Volumetric video is animation data composed of polygon meshes (hereinafter simply referred to as "meshes") and textures, and is displayed on a display by rendering it together with the virtual environment on the client side, allowing it to be viewed.

Technologies described in Non-Patent Documents 1 and 2 are known as volumetric video distribution technologies. Non-Patent Document 1 proposes a method in which volumetric video is rendered on the server side based on the user's head movements detected by the client AR/VR device, and then transmitted to the client as 2D data. Non-Patent Document 2 also proposes a method in which the volumetric video's Level Of Detail is dynamically changed depending on the bandwidth of the communication network, thereby reducing the amount of data required for playback.

Serhan Gul, Dimitri Podborski, Thomas Buchholz, Thomas Schierl and Cornelius Hellge, "Low-latency cloud-based volumetric video streaming using head motion prediction", NOSSDAV '20: Proceedings of the 30th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, June 2020, Pages 27-33 Holostream/Arctrus, Internet <URL: https://arcturus.studio/holostream>

However, volumetric video requires a large amount of data and requires a large communication network bandwidth for its distribution, so an efficient distribution method is required.

However, the method proposed in the above-mentioned non-patent document 1 requires rendering for each user to be performed on the server side, which places a heavy load on the server. Furthermore, as the number of users increases, the division of server resources may result in a degradation of the quality of the video viewed by each user. Furthermore, location information must be sent from the client to the server frequently and with low latency. For example, keeping the Motion to Photon delay, at which VR sickness begins to occur, to 20 ms or less places a heavy burden on both the communication network and the server.

On the other hand, the method proposed in the above-mentioned non-patent document 2 reduces the image quality and level of detail of the volumetric video being viewed, including the 3D data visible to the user (i.e., the 3D data that corresponds to the front of the user), when the available bandwidth of the communication network is narrow, resulting in a significant degradation in the quality of experience.

This disclosure has been made in consideration of the above points and provides technology that can reduce the amount of data required to deliver three-dimensional video content while maintaining the user's quality of experience.

A distribution control system according to one aspect of the present disclosure includes an arrangement unit that arranges multiple virtual viewpoints centered on an object represented by three-dimensional data constituting three-dimensional video content; a calculation unit that calculates, for each virtual viewpoint, a first virtual viewpoint obtained by rotating the virtual viewpoint by a predetermined angle in the positive direction around the object, and a second virtual viewpoint obtained by rotating the virtual viewpoint by the same angle in the negative direction around the object; a creation unit that creates one-plane three-dimensional data for each virtual viewpoint by reducing the amount of data for portions that cannot be seen from the first virtual viewpoint and the second virtual viewpoint; and a distribution unit that distributes one-plane three-dimensional data for one virtual viewpoint from the one-plane three-dimensional data for each virtual viewpoint to a user's terminal in accordance with the user's position and field of view in a virtual space in which the object is located.

Technology is provided that can reduce the amount of data required to deliver 3D video content while maintaining the user's quality of experience.

1 is a diagram illustrating an example of the overall configuration of a distribution control system according to an embodiment of the present invention. 10 is a flowchart illustrating an example of a one-plane 3D data creation process according to the present embodiment. FIG. 10 is a diagram illustrating an example of the arrangement of virtual viewpoints. FIG. 2 is a diagram showing the visible range of a virtual viewpoint _Vi . FIG. 10 is a diagram showing the visible range of a virtual viewpoint V _i1 . FIG. 10 is a diagram showing the visible range of a virtual viewpoint _Vi2 . FIG. 10 is a diagram showing the visible range and outside the visible range of a virtual viewpoint V _i from −π/N to +π/N. 10 is a flowchart illustrating an example of a correspondence table creation process according to the present embodiment. FIG. 10 is a diagram showing an example of a viewing angle range correspondence table. 10 is a flowchart illustrating an example of a distribution process according to the present embodiment. FIG. 2 illustrates an example of a hardware configuration of a computer.

An embodiment of the present invention will be described below. In the following embodiment, a distribution control system 1 will be described that targets volumetric video as an example of stereoscopic video content and can reduce the amount of data required for distribution while maintaining the user's quality of experience. Here, volumetric video refers to animation data composed of 3D data (also referred to as three-dimensional data or stereoscopic data) represented by meshes and textures. That is, for example, if the 3D data of a frame at time t is _dt , the volumetric video is expressed as { _dt |tε[ _ts , _te ]}, where _ts is the start time of the volumetric video and _te is the end time.

Note that the embodiments described below are not limited to volumetric video, but can also be applied to three-dimensional video content with six degrees of freedom, such as holograms.

<Overall configuration example>
First, the overall configuration of a distribution control system 1 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the overall configuration of the distribution control system 1 according to this embodiment.

As shown in FIG. 1, the distribution control system 1 according to this embodiment includes a distribution control server 10, a content server 20, a client 30, and a service site server 40. The distribution control server 10 and the client 30 are communicatively connected via a communication network 50 such as the Internet. Similarly, the client 30 and the service site server 40 are communicatively connected via the communication network 50.

Note that the distribution control server 10 and the content server 20 are assumed to exist within the same local network and to be connected so that they can communicate within that local network, but this is not limited to this and they may also be connected so that they can communicate via a communication network 50, for example.

The distribution control server 10 creates multiple single-panel 3D data from the 3D data that constitutes the given volumetric video, and stores these multiple single-panel 3D data on the content server 20. Here, single-panel 3D data refers to three-dimensional data of an object represented by the 3D data that constitutes the volumetric video when viewed from a single viewpoint, and is reduced in data volume compared to the original 3D data while maintaining the user's quality of experience. Note that the object refers to the subject of the volumetric video, and can be any object that can be represented by a mesh and texture, such as a person, animal, plant, structure, building, machine, celestial body, natural phenomenon, etc.

In addition, in response to a viewing request from a client 30, the distribution control server 10 determines appropriate single-sided 3D data based on the user's viewpoint, spatial position, field of view (line of sight direction and field of view range), etc., and distributes this single-sided 3D data to the client 30. Note that the user's viewpoint and spatial position refer to the user's position within the virtual space in which the object is located.

The content server 20 stores multiple single-sided 3D data. In addition, in response to a data request from the distribution control server 10, the content server 20 returns the single-sided 3D data corresponding to this data request to the distribution control server 10.

The client 30 is a device (e.g., an XR (VR/AR/MR/SR, etc.) device) used by a user to view the volumetric video, and renders the single-sided 3D data distributed from the distribution control server 10 and plays the volumetric video. Note that XR devices include HMDs (Head Mount Displays), as well as smartphones, tablet devices, and wearable devices equipped with application programs that function as XR devices.

When a user views a volumetric video, the following procedure is performed, for example. First, the user accesses the service site server 40 using the client 30 and obtains a list of content (volumetric videos) that the user can view. Next, the user selects the volumetric video they wish to view from this list and obtains a link to the selected volumetric video. Then, when the client 30 accesses the link, a viewing request is sent to the distribution control server 10, and playback of the volumetric video begins when full-screen 3D data is returned in response to this request.

In addition, the client 30 appropriately transmits information such as the user's viewpoint, spatial position, field of view, etc. (hereinafter, information representing the user's viewpoint or spatial position and field of view will also be referred to as "user viewpoint information") to the distribution control server 10. As a result, one-sided 3D data corresponding to the user's viewpoint, spatial position, field of view, etc. will be returned from the distribution control server 10 and played back on the client 30.

The service site server 40 presents a list of content (volumetric video) that the user can view and provides the client 30 with a link to content selected from this list.

The distribution control server 10 according to this embodiment includes a single-sided 3D data creation unit 101, a correspondence table creation unit 102, a distribution control unit 103, and a distribution unit 104. Each of these units is realized, for example, by one or more programs installed in the distribution control server 10 being executed by a processor such as a CPU (Central Processing Unit).

The distribution control server 10 according to this embodiment also has a correspondence table storage unit 105. The correspondence table storage unit 105 is realized by an auxiliary storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

The single-panel 3D data creation unit 101 creates multiple single-panel 3D data from the 3D data constituting the volumetric video provided to the distribution control server 10. More specifically, the single-panel 3D data creation unit 101 arranges multiple virtual viewpoints at equal (or unequal) intervals around the object represented by the 3D data constituting the volumetric video, and then, for each virtual viewpoint, performs geometry simplification (Level of Detail reduction processing) on meshes corresponding to portions of the object that cannot be seen when the object is viewed from the virtual viewpoint and a viewpoint rotated a certain angle around the virtual viewpoint, thereby creating single-panel 3D data. In this way, for each virtual viewpoint, geometry simplification, such as vertex deletion, is performed on meshes corresponding to portions of the object's mesh that cannot be seen from the virtual viewpoint and a viewpoint rotated a certain angle around the virtual viewpoint, thereby creating single-panel 3D data for each virtual viewpoint.

Then, the single-sided 3D data creation unit 101 stores the created multiple single-sided 3D data in the content server 20.

The correspondence table creation unit 102 creates a correspondence table (hereinafter also referred to as a "viewing angle range correspondence table") that associates each virtual viewpoint placed when creating multiple single-plane 3D data with the viewing angle range of that virtual viewpoint. The viewing angle range is the range within which, when viewing an object, the same quality of experience can be obtained as when viewing the object from that virtual viewpoint. In other words, it is the range within which the simplified geometric parts of the object cannot be seen when viewing the object from that virtual viewpoint.

When the distribution control unit 103 receives user viewpoint information from the client 30, it refers to the viewing angle range correspondence table and determines appropriate single-sided 3D data from this user viewpoint information. More specifically, when at least a portion of an object is captured within the field of view from the viewpoint or spatial position included in the user viewpoint information, the distribution control unit 103 identifies a virtual viewpoint corresponding to the viewing angle range that includes that viewpoint or spatial position, and determines the single-sided 3D data corresponding to the identified virtual viewpoint as appropriate.

The distribution unit 104 transmits the single-sided 3D data determined by the distribution control unit 103 to the client 30.

The correspondence table memory unit 105 stores the viewing angle range correspondence table created by the correspondence table creation unit 102.

The content server 20 according to this embodiment also has a single-sided 3D data storage unit 201. The single-sided 3D data storage unit 201 is realized, for example, by an auxiliary storage device such as an HDD or SSD.

The single-sided 3D data storage unit 201 stores each single-sided 3D data created by the single-sided 3D data creation unit 101.

Note that the configuration of the distribution control system 1 shown in Figure 1 is an example, and other configurations may be used. For example, the distribution control server 10 and the content server 20 may be configured as an integrated server.

<Details of various processes>
Below, we will explain each of the following processes: a single-panel 3D data creation process for creating multiple single-panel 3D data, a correspondence table creation process for creating a viewing angle range correspondence table, and a distribution process for distributing appropriate single-panel 3D data to client 30. Note that the single-panel 3D data creation process and the correspondence table creation process are pre-processing executed before the distribution process.

<<One-sided 3D data creation process>>
First, the single-screen 3D data creation process will be described with reference to Fig. 2. Fig. 2 is a flowchart showing an example of the single-screen 3D data creation process according to this embodiment. In the following, it is assumed that volumetric video has been provided to the distribution control server 10.

The single-screen 3D data creation unit 101 acquires one frame of 3D data from the 3D data that constitutes the volumetric video (step S101). In the following, it is assumed that the single-screen 3D data creation unit 101 acquires 3D data d _t for a frame at time t. Note that the initial value of t may be set to t = t _s .

Note that 3D data consists of a mesh and a texture, but since no special processing is performed on the texture below, in step S101 above, only the mesh of the 3D data may be obtained.

Next, the single-plane 3D data creation unit 101 arranges N virtual viewpoints around the object represented by the 3D data _dt acquired in step S102 (step S102). The line-of-sight direction of each virtual viewpoint is the object. For example, the single-plane 3D data creation unit 101 arranges N virtual viewpoints, with the object as the line-of-sight direction, at equal intervals (or may be unequal intervals) on the circumference of a circle of a predetermined radius centered on the object. N is a predetermined integer of 2 or greater.

An example of the arrangement of virtual viewpoints when N = 8 is shown in Fig. 3. In the example shown in Fig. 3, virtual viewpoints _V1 to _V8 , with the object O as the line of sight, are arranged counterclockwise at equal intervals (i.e., 2π/N intervals) on the circumference of a circle with radius R centered on the object O. Also, in the example shown in Fig. 3, the angle of virtual viewpoint _V1 is set as the reference angle with respect to the object O ( _θ1 = 0), and the angles _θ2 to _θ8 of virtual viewpoints _V2 to _V8 with respect to the object O are set as rotation angles from virtual viewpoint _V1 .

Note that placing virtual viewpoints on the circumference of a circle is one example and is not limited to this, and for example, virtual viewpoints may be placed on the sides or vertices of a polygon centered on the object. Also, in the example shown in Figure 3, for simplicity, virtual viewpoints _V1 to _V8 are placed on the xy plane of xyz space with the center of object O as the origin, but this is not limited to this, and for example, virtual viewpoints may be placed on a sphere (or polyhedron) centered at the origin.

In the following, we will assume that N virtual viewpoints are arranged counterclockwise at equal intervals on the circumference of a circle of a certain radius centered on the object.

Next, the single-sided 3D data creation unit 101 selects one virtual viewpoint from the N virtual viewpoints arranged in step S102 above (step S103).

Next, the single-plane 3D data creation unit 101 performs geometry simplification (Level Of Detail reduction processing) on meshes corresponding to portions of the object that cannot be seen when the object is viewed from the virtual viewpoint selected in step S103 and its range of -π/N to +π/N, thereby creating single-plane 3D data corresponding to the virtual viewpoint (step S104). Specifically, the single-plane 3D data creation unit 101 creates single-plane 3D data corresponding to the virtual viewpoint by the following steps 1 to 6. In the following, the virtual viewpoint selected in step S103 is defined as V _i , and the angle of virtual viewpoint V _i with respect to the object is defined as θ _i . Note that θ _i is a rotation angle from a reference angle (for example, the angle θ ₁ of virtual viewpoint V ₁ =0).

Step 1: The single-side 3D data creation unit 101 calculates the angular range of the visible portion of the object when the object is viewed from a virtual viewpoint _Vi (hereinafter also referred to as the visible range). The visible range may be, for example, a range of -π/2 to +π/2 from the angle of the viewpoint relative to the object. The visible range when the object O is viewed from a virtual viewpoint _Vi is shown in Figure 4. This indicates that when the object O is viewed from the virtual viewpoint _Vi , the portion of the object O that satisfies θi _-π /2 < θ < _θi +π/2 can be viewed from the virtual viewpoint _Vi . Note that θ represents a variable that takes an angle.

Step 2: The single-view 3D data creation unit 101 sets a virtual viewpoint V _i1 at a position obtained by rotating the virtual viewpoint V _i by +π/N around the object, and calculates the viewable range when the object is viewed from this virtual viewpoint V _i1 . The viewable range when the object O is viewed from the virtual viewpoint V _i1 is shown in FIG. 5. This indicates that when the object O is viewed from the virtual viewpoint V _i1 , the portion of the object O that satisfies θ _i + (1/N - 1/2)π < θ < θ _i + (1/N + 1/2)π can be viewed from the virtual viewpoint V _i1 . Note that the angle θ _i + π/N is an angle that equally divides the distance between the virtual viewpoint V _i and the virtual viewpoint V _i+1 (however, V ₁ when i = N), and can be considered the boundary between the virtual viewpoint V _i and the virtual viewpoint V _i+1 (however, V ₁ when i = N). Therefore, hereinafter, this angle is also referred to as the "positive boundary angle."

Step 3: The single-view 3D data creation unit 101 sets a virtual viewpoint V _i2 at a position obtained by rotating the virtual viewpoint V _i by -π/N around the object, and calculates the visible range when the object is viewed from this virtual viewpoint V _i2 . The visible range when the object O is viewed from the virtual viewpoint V _i2 is shown in FIG. 6. This indicates that when the object O is viewed from the virtual viewpoint V _i2 , the portion of the object O that satisfies θ _i -(1/N+1/2)π<θ<θ _i -(1/N-1/2)π can be viewed from the virtual viewpoint V _i2 . The angle θ _i -π/N is an angle that equally divides the distance between the virtual viewpoint V _i and the virtual viewpoint V _i-1 (however, V _N when i = 1), and can be considered the boundary between the virtual viewpoint V _i and the virtual viewpoint V _i-1 (however, V _N when i = 1). Therefore, this angle is also referred to as the "negative boundary angle" below.

Step 4: The single-sided 3D data creation unit 101 calculates the union of the visible range of virtual viewpoint V _i , the visible range of virtual viewpoint V _i1 , and the visible range of virtual viewpoint V _i2 as the visible range. In other words, the single-sided 3D data creation unit 101 determines the visible range to be θ _i - (1/N + 1/2)π < θ < θ _i + (1/N + 1/2)π.

Note that since the visible range coincides with the union of the visible range of the virtual viewpoint V _i1 and the visible range of the virtual viewpoint V _i2 , the visible range of the virtual viewpoint V _i is not necessary when calculating the visible range. Therefore, the above step 1 does not need to be performed. In this case, in the above step 4, the visible range can be calculated as the union of the visible range of the virtual viewpoint V _i1 and the visible range of the virtual viewpoint V _i2 .

Step 5: The single-plane 3D data creation unit 101 calculates the difference set between _θi -π<θ≦ _θi +π and the visible range as the out-of-view range. That is, the single-plane 3D data creation unit 101 calculates _θi -π<θ≦ _θi- (1/N+1/2)π and _θi +(1/N+1/2)π≦θ≦ _θi +π as the out-of-view range. The visible range and out-of-view range are shown in FIG. 7. The visible range represents the _angular range of the portion of the object O that is visible when the object O is viewed from a viewpoint within the range of _{θi -} π/N to _θi +π/N. On the other hand, the out-of-view range represents the angular range of the portion of the object O that is not visible from a viewpoint within the range of θi-π/N to θi+π/N.

Step 6: The single-plane 3D data creation unit 101 performs geometry simplification ₍ Level Of Detail reduction processing) on the mesh of the portion of the object that corresponds to outside the visible range. As a result, single-plane 3D data is created that provides a quality of experience equivalent to that of the original 3D data d t when the object is viewed from a viewpoint within the range of θ _{i -π/} N to θ i +π/N, and has a reduced data volume compared to the original 3D data d _t . Hereinafter, the single-plane 3D data corresponding to the virtual viewpoint _{V i of} the 3D data d _t at time t is referred to as d _ti .

Next, the single-sided 3D data creation unit 101 determines whether all N virtual viewpoints have been selected (step S105).

If it is determined in step S105 above that there is a virtual viewpoint that has not yet been selected, the single-sided 3D data creation unit 101 returns to step S103 above, selects one virtual viewpoint from the virtual viewpoints that have not yet been selected, and executes the processing from step S104 onwards.

On the other hand, if it is determined in step S105 above that all N virtual viewpoints have been selected, the one-sided 3D data creation unit 101 determines whether there is a next frame in the given volumetric video (step S106).

If it is determined in step S106 that there is a next frame, the single-plane 3D data creation unit 101 returns to step S101, acquires 3D data for the next frame, and executes the processes from step S102 onward. That is, in this case, the single-plane 3D data creation unit 101 sets t←t+1, returns to step S101, and acquires 3D data d _t for the next frame at time t.

On the other hand, if it is determined in step S106 that there is no next frame (i.e., time t=t _e ), the single-plane 3D data creation unit 101 stores each single-plane 3D data created in step S104 in the single-plane 3D data storage unit 201 of the content server 20 (step S107). As a result, the single-plane 3D data set {d _ti |t∈[t _s , t _e ], i∈[1, N]} is stored in the single-plane 3D data storage unit 201.

In this embodiment, the number N of virtual viewpoints is the same for all frames, but may be different for each frame. Also, in this embodiment, steps S102 to S105 are repeatedly executed for each frame, but for example, if the 3D data d t is the same between frames within a certain time span (including the case where the time span is t _e -t _s ), steps S102 to S105 may be executed only for the 3D _{data d t of} one frame included in that time span.

<Correspondence table creation process>
Next, the correspondence table creation process will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the correspondence table creation process according to this embodiment. Below, a case where a viewing angle range correspondence table relating to 3D data _dt (where tε[ _ts , _te ]) is created will be described. Note that d = _dt will be expressed below.

The correspondence table creation unit 102 arranges N virtual viewpoints around the object represented by the 3D data d (step S201), similar to step S102 in Figure 2. Note that the number of virtual viewpoints and the arrangement method (equally spaced or non-equally spaced, arranged on a circle or on a polygon, etc.) are the same as in step S102 in Figure 2.

Next, the correspondence table creation unit 102 selects one virtual viewpoint from the N virtual viewpoints arranged in step S201 above (step S202).

Next, the correspondence table creation unit 102 calculates the boundary angles (positive boundary angle and negative boundary angle) related to the virtual viewpoint selected in step S202 (step S203). That is, if the virtual viewpoint selected in step S202 is V _i and the angle of the virtual viewpoint V _i with respect to the object is θ _i , the correspondence table creation unit 102 calculates the positive boundary angle θ _i +π/N and the negative boundary angle θ _i -π/N.

Next, the correspondence table creation unit 102 determines whether all N virtual viewpoints have been selected (step S204).

If it is determined in step S204 above that there is a virtual viewpoint that has not yet been selected, the correspondence table creation unit 102 returns to step S202 above, selects one virtual viewpoint from the virtual viewpoints that have not yet been selected, and executes the processing from step S203 onwards.

On the other hand, if it is determined in step S204 that all N virtual viewpoints have been selected, the correspondence table creation unit 102 creates a viewing angle range correspondence table by calculating the viewing angle range of each virtual viewpoint from the boundary angle calculated in step S203, and stores the created viewing angle range correspondence table in the correspondence table storage unit 105 (step S205). The viewing angle range of virtual viewpoint V _i (where i∈[1,N]) is calculated to be an angle range equal to or greater than the boundary angle θ _i -π/N in the negative direction and less than the boundary angle θ _i +π/N in the positive direction, that is, θ _i -π/N≦θ<θ _i +π/N. In this way, the viewing angle range correspondence table is created by associating virtual viewpoint V _i with the viewing angle range θ _i -π/N≦θ<θ _i +π/N. An example of the viewing angle range correspondence table created in this manner is shown in FIG. 9.

In this embodiment, a common viewing angle range correspondence table is created for 3D data _dt at time t∈[ _ts , _te ]. However, for example, if the number of virtual viewpoints differs depending on the frame, a viewing angle range correspondence table may be created for each number of virtual viewpoints.

<Distribution process>
Next, the distribution process will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the distribution process according to this embodiment. The following steps S301 to S303 are executed each time user viewpoint information is received, and steps S304 to S305 are executed at each frame interval. However, in the following, it is assumed that at least a portion of the target object is included in the user's field of view.

The distribution control unit 103 calculates the user position relative to the object using the viewpoint or spatial position included in the user viewpoint information received from the client 30 (step S301). Here, the user position is the angle of the viewpoint or spatial position relative to the object. Note that the angle criteria are the same as those used when determining the angle of the virtual viewpoint relative to the object in step S203 of Figure 8.

Next, the distribution control unit 103 refers to the viewing angle range correspondence table stored in the correspondence table storage unit 105 and identifies a virtual viewpoint from the user position calculated in step S301 (step S302). That is, the distribution control unit 103 identifies, from among the virtual viewpoints, a virtual viewpoint corresponding to the viewing angle range that includes the user position. Specifically, when the user position calculated in step S301 is θ _U , the distribution control unit 103 identifies a virtual viewpoint V _i that satisfies θ _i - π/N ≦ θ _U < θ _i + π/N.

Next, the distribution control unit 103 determines the single-plane 3D data corresponding to the virtual viewpoint identified in step S302 as the distribution target (step S303). That is, for example, if the virtual viewpoint identified in step S302 is V _i , the distribution control unit 103 determines the single-plane 3D data {d _ti } as the distribution target.

The distribution unit 104 obtains the single-sided 3D data for the frame at the corresponding time from the single-sided 3D data to be distributed determined in step S303 above from the content server 20 (step S304).

Then, the distribution unit 104 distributes the single-sided 3D data acquired in step S304 to the client 30 (step S305). As a result, the single-sided 3D data is rendered on the client 30 side, and the object is displayed on the display.

<Hardware configuration>
Finally, the hardware configuration of the delivery control server 10 and the content server 20 according to this embodiment will be described. The delivery control server 10 and the content server 20 according to this embodiment are realized, for example, by the hardware configuration of a computer 500 shown in Fig. 11. The client 30 and the service site server 40 may also be realized by a similar hardware configuration.

The computer 500 shown in Figure 11 has an input device 501, a display device 502, an external I/F 503, a communication I/F 504, a processor 505, and a memory device 506. Each of these pieces of hardware is connected to each other so that they can communicate with each other via a bus 507.

The input device 501 is, for example, a keyboard, a mouse, a touch panel, etc. The display device 502 is, for example, a display, etc. Note that the computer 500 does not necessarily have to have at least one of the input device 501 and the display device 502.

The external I/F 503 is an interface with external devices such as a recording medium 503a. Examples of recording media 503a include a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

The communication I/F 504 is an interface for data communication with other devices, equipment, systems, etc. The processor 505 is, for example, a CPU or other computing device. The memory device 506 is, for example, a HDD, SSD, RAM (Random Access Memory), ROM (Read Only Memory), flash memory, or other storage device.

The distribution control server 10 and content server 20 according to this embodiment have the hardware configuration of a computer 500 shown in FIG. 11, and are therefore able to perform the various processes described above. Note that the hardware configuration of the computer 500 shown in FIG. 11 is merely an example, and the computer 500 may have other hardware configurations. For example, the computer 500 may have multiple processors 505 or multiple memory devices 506.

<Summary>
As described above, the distribution control system 1 according to this embodiment arranges multiple virtual viewpoints with respect to an object represented by 3D data constituting stereoscopic video content, and then, for each of these virtual viewpoints, geometrically simplifies the polygon mesh of the portion that cannot be seen from viewpoints within a range from the negative boundary angle to the positive boundary angle of the virtual viewpoint. Because the geometrically simplified portion cannot be seen from viewpoints within a range from the negative boundary angle to the positive boundary angle of each virtual viewpoint, it is possible to create single-plane 3D data that does not degrade the user's quality of experience and that has a reduced data volume compared to the original 3D data.

Therefore, by delivering appropriate single-sided 3D data according to the user's position and field of view at client 30, it is possible to reduce the amount of data required to deliver stereoscopic video content while maintaining the user's quality of experience. In addition, the rendering load on client 30 is also reduced, which makes it possible to reduce the processing load on client 30.

The present invention is not limited to the specifically disclosed embodiments above, and various modifications, alterations, and combinations with known technologies are possible without departing from the scope of the claims.

REFERENCE SIGNS LIST 1 Distribution control system 10 Distribution control server 20 Content server 30 Client 40 Service site server 50 Communication network 101 One-sided 3D data creation unit 102 Correspondence table creation unit 103 Distribution control unit 104 Distribution unit 105 Correspondence table storage unit 201 One-sided 3D data storage unit 500 Computer 501 Input device 502 Display device 503 External I/F
503a Recording medium 504 Communication I/F
505 Processor 506 Memory device 507 Bus

Claims (8)

a placement unit that places a plurality of virtual viewpoints centered on an object represented by three-dimensional data constituting three-dimensional video content;
a calculation unit that calculates, for each of the virtual viewpoints, a first virtual viewpoint obtained by rotating the virtual viewpoint by a predetermined angle in a positive direction around the object as a center, and a second virtual viewpoint obtained by rotating the virtual viewpoint by the predetermined angle in a negative direction around the object as a center;
a creating unit that creates, for each of the virtual viewpoints, one-plane three-dimensional data in which the amount of data of a portion that cannot be viewed from the first virtual viewpoint and the second virtual viewpoint is reduced;
a distribution unit that distributes one-plane three-dimensional data of one virtual viewpoint among the one-plane three-dimensional data for each virtual viewpoint to a terminal of the user in accordance with a position and field of view of the user in the virtual space in which the object is placed;
A distribution control system having: The placement unit
The distribution control system according to claim 1 , wherein the plurality of virtual viewpoints are arranged at equal or unequal intervals on circles or spheres, or on vertices or edges of a polygon or polyhedron, with the object at the center. The calculation unit
2. The distribution control system according to claim 1, wherein when the arrangement unit arranges N (where N is a predetermined integer of 2 or greater) virtual viewpoints at equal intervals on a circle, for each of the virtual viewpoints, a first virtual viewpoint obtained by rotating the virtual viewpoint by π/N in a positive direction and a second virtual viewpoint obtained by rotating the virtual viewpoint by π/N in a negative direction are calculated. The creation unit
calculating a range of ±π/2 of the angle of the first virtual viewpoint with respect to the object as a viewable range from the first virtual viewpoint;
calculating a range of ±π/2 of the angle of the second virtual viewpoint with respect to the object as a viewable range from the second virtual viewpoint;
The distribution control system according to claim 3, wherein a range excluding a range represented by the sum of the visible range at the first virtual viewpoint and the visible range at the second virtual viewpoint from a range of ±π of the angle of the virtual viewpoint relative to the object is calculated as the portion that cannot be seen from the first virtual viewpoint and the second virtual viewpoint. The distribution unit
5. The distribution control system according to claim 1, further comprising: identifying a virtual viewpoint where the angle of the position relative to the object is less than the angle of a first virtual viewpoint relative to the object and is greater than or equal to the angle of a second virtual viewpoint relative to the object; and distributing one-sided stereoscopic data corresponding to the identified virtual viewpoint to the user's terminal. a placement unit that places a plurality of virtual viewpoints centered on an object represented by three-dimensional data constituting three-dimensional video content;
a calculation unit that calculates, for each of the virtual viewpoints, a first virtual viewpoint obtained by rotating the virtual viewpoint by a predetermined angle in a positive direction around the object as a center, and a second virtual viewpoint obtained by rotating the virtual viewpoint by the predetermined angle in a negative direction around the object as a center;
a creating unit that creates, for each of the virtual viewpoints, one-plane three-dimensional data in which the amount of data of a portion that cannot be viewed from the first virtual viewpoint and the second virtual viewpoint is reduced;
a distribution unit that distributes one-plane three-dimensional data of one virtual viewpoint among the one-plane three-dimensional data for each virtual viewpoint to a terminal of the user in accordance with a position and field of view of the user in the virtual space in which the object is placed;
A distribution control device having the above configuration. an arrangement step of arranging a plurality of virtual viewpoints centered on an object represented by three-dimensional data constituting the three-dimensional video content;
a calculation step of calculating, for each of the virtual viewpoints, a first virtual viewpoint obtained by rotating the virtual viewpoint by a predetermined angle in a positive direction around the object as a center, and a second virtual viewpoint obtained by rotating the virtual viewpoint by the predetermined angle in a negative direction around the object as a center;
a creating step of creating, for each of the virtual viewpoints, one-plane three-dimensional data in which the amount of data of a portion that cannot be viewed from the first virtual viewpoint and the second virtual viewpoint is reduced;
a delivery procedure for delivering one of the one-plane three-dimensional data for each virtual viewpoint to a terminal of the user in accordance with a position and field of view of the user in the virtual space in which the object is placed;
The computer executes the delivery control method. an arrangement step of arranging a plurality of virtual viewpoints centered on an object represented by three-dimensional data constituting the three-dimensional video content;
a calculation step of calculating, for each of the virtual viewpoints, a first virtual viewpoint obtained by rotating the virtual viewpoint by a predetermined angle in a positive direction around the object as a center, and a second virtual viewpoint obtained by rotating the virtual viewpoint by the predetermined angle in a negative direction around the object as a center;
a creating step of creating, for each of the virtual viewpoints, one-plane three-dimensional data in which the amount of data of a portion that cannot be viewed from the first virtual viewpoint and the second virtual viewpoint is reduced;
a delivery procedure for delivering one of the one-plane three-dimensional data for each virtual viewpoint to a terminal of the user in accordance with a position and field of view of the user in the virtual space in which the object is placed;
A program that causes a computer to execute the following.