JP7685348B2 - Media Processing Device - Google Patents

Description

The present invention relates to a media processing device.

Conventionally, mechanisms for transmitting content such as 360° video and 3D objects have been proposed (for example, Non-Patent Document 1). Known mechanisms include 3DoF+ (Degree of Freedom), which involves viewpoint movement within the range in which the user moves their head while in a seated position, and 6DoF, which involves viewpoint movement within the range in which the user can move freely. In such mechanisms, the positional relationship between the 360° video and the 3D objects is indicated by a scene description.

3GPP TR 26.928 V16.1.0 December 2020

Against the above background, the inventors noticed that the user terminal that outputs the above content needs to generate content (360° video and 3D objects) based on viewpoint information (viewpoint position and line of sight direction), which increases the processing load on the user terminal.

The present invention has been made to solve the above-mentioned problems, and aims to provide a media processing device that makes it possible to reduce the processing load on a user terminal that generates specific content, including content with viewpoint freedom.

The disclosed media processing device includes a receiving unit that receives viewpoint information from a user terminal, a renderer that generates specific content including at least content with viewpoint freedom based on the viewpoint information, and a transmitting unit that transmits the specific content generated by the renderer to the user terminal.

The present invention provides a media processing device that can reduce the processing load on a user terminal that generates specific content, including content with viewpoint freedom.

FIG. 1 is a diagram showing a transmission system 10 according to an embodiment. FIG. 2 is a block diagram showing a media processing device 200 and a user terminal 300 according to an embodiment of the invention. FIG. 3 is a diagram for explaining the second content according to the embodiment. FIG. 4 is a diagram showing a method for viewing a specific content according to the embodiment. FIG. 5 is a diagram for explaining the first method according to the first modified example. FIG. 6 is a diagram for explaining the second method according to the first modified example.

Next, an embodiment of the present invention will be described. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, it should be noted that the drawings are schematic, and the ratios of the dimensions may differ from the actual ones.

Therefore, specific dimensions should be determined with reference to the following explanation. Of course, there are also parts in which the dimensional relationships and ratios differ between the drawings.

[Disclosure Summary]
The media processing device according to the disclosed summary includes a receiving unit that receives viewpoint information from a user terminal, a renderer that generates specific content including at least content having viewpoint freedom based on the viewpoint information, and a transmitting unit that transmits the specific content generated by the renderer to the user terminal.

In the summary of the disclosure, the media processing device generates specific content based on viewpoint information and then transmits the specific content to the user terminal. With this configuration, there is no need for the user terminal to generate specific content including content with viewpoint freedom, and the user terminal can present the specific content by simply providing viewpoint information to the media processing device. Therefore, although a delay occurs between the media processing device and the user terminal, the processing load on the user terminal can be reduced.

It should be noted that the specific content generated by the media processing device is generated based on viewpoint information, and the images contained in the specific content can be treated as 2D images on the user terminal side.

[Embodiment]
(Transmission System)
A transmission system according to an embodiment will be described below. Fig. 1 is a diagram showing a transmission system 10 according to an embodiment. As shown in Fig. 1, the digital wireless transmission system includes a transmitting device 100, a media processing device 200, and a user terminal 300.

In an embodiment, the transmitting device 100 transmits a first content that does not have freedom of viewpoint and a second content that has freedom of viewpoint to the media processing device 200. Furthermore, the transmitting device 100 transmits first control information associated with the first content and second control information associated with the second content to the media processing device 200.

The first content may include at least one of 2D video and audio. The first content and the first control information may be transmitted in a first method. The first method may be a method conforming to ISO/IEC 23008-1 (hereinafter, MMT (MPEG Media Transport)). In the following, a case will be illustrated in which the first method is MMTP (MMT Protocol) conforming to MMT. The first control information may be referred to as MMT-SI (Signaling Information).

The second content may include a 360° video and a 3D object. The second content and the second control information may be transmitted in a second manner. Alternatively, they may be transmitted in a protocol such as HTTP (Hyper Text Transfer Protocol). The second content may conform to 3DoF+ (Degree of Freedom), which involves viewpoint movement within the range in which a user moves their head while in a sitting position, or 6DoF, which involves viewpoint movement within the range in which a user can move freely. Since the second content has viewpoint freedom, it may include two or more 360° videos and two or more 3D objects at the same time (frame). The second control information may be referred to as a scene description.

Here, the second control information may be transmitted in the first method described above. That is, the second control information may be transmitted in the same first method (e.g., MMTP) as the first control information. Alternatively, the second control information may be transmitted in a protocol such as HTTP.

The transmission from the transmitting device 100 to the media processing device 200 is not particularly limited, but may be transmission using satellite broadcasting, transmission using the Internet network, or transmission using a mobile communication network.

Although not particularly limited, the transmission system may be a digital wireless transmission system. The digital wireless transmission system may be a system used for 4K and 8K satellite broadcasting.

The media processing device 200 generates specific content including at least the second content described above based on viewpoint information received from the user terminal 300, and transmits the generated specific content to the user terminal 300. Although not particularly limited, the transmission of the specific content may be transmission using the Internet network or transmission using a mobile communication network.

The user terminal 300 may be a user terminal such as a smartphone, a tablet terminal, or a head-mounted display. As shown in FIG. 1, two or more user terminals 300 may be provided as the user terminal 300. In other words, two or more user terminals 300 may request the media processing device 200 to generate specific content. Each user terminal 300 may transmit different viewpoint information to the media processing device 200.

(Media Processing Device and User Terminal)
A media processing device and a user terminal according to an embodiment of the present invention will now be described. Fig. 2 is a block diagram showing a media processing device 200 and a user terminal 300 according to an embodiment of the present invention.

First, the media processing device 200 has a reception unit 210, a renderer 220, and an encoding processing unit 230.

The reception unit 210 receives viewpoint information. In the embodiment, the reception unit 210 constitutes a receiving unit that receives viewpoint information from the user terminal 300. The viewpoint information includes an information element indicating the viewpoint position of the user of the user terminal 300 and an information element indicating the line of sight of the user of the user terminal 300.

The renderer 220 generates specific content including at least the second content based on the viewpoint information. Since the specific content is generated based on the viewpoint information, it may include one 360° video and one 3D object at the same time (frame). Below, an example is given of the specific content including the first content in addition to the second content.

As shown in FIG. 2, the renderer 220 generates a first content including 2D video and audio as part of a specific content based on the first control information (MMT-SI). Viewpoint information is not required in generating the first content.

Specifically, the renderer 220 acquires the 2D video, audio, and MMT-SI in the form of an MMTP packet in which the 2D video, audio, and MMT-SI are packetized.

For example, MMTP packets are stored in IP (Internet Protocol) packets. IP packets may be transmitted using UDP (User Datagram Protocol) or TCP (Transmission Control Protocol).

Here, the first content is processed in units (hereinafter, MPUs; Media Processing Units) that are separated by a certain time width. An MPU includes one or more access units. An access unit may also be treated as an MFU (Media Fragment Unit). An MFU related to 2D video may be referred to as a NAL (Network Abstraction Layer) unit, and an MFU related to audio may be referred to as an MHAS (MPEG-H 3D Audio Stream) packet.

The MMT-SI includes a PA (Package Access) message, and the PA message includes an MPT (MMT Package Table) that indicates a list of the first content. Furthermore, the MMT-SI includes an MPU timestamp descriptor that indicates the presentation time of the first content. The MPU timestamp descriptor may mean the presentation time of the MPU, i.e., the time of the access unit that is first presented in the MPU.

The MPU timestamp descriptor may be generated using UTC (Coordinated Universal Time) as the reference time. The reference time may be TAI (International Atomic Time) or may be a time provided by GPS (Global Positioning System). The reference time may be a time provided by an NTP (Network Time Protocol) server or a PTP (Precision Time Protocol) server.

Secondly, the renderer 220 generates second content including a 360° video and a 3D object as part of the specific content based on the second control information (scene description). The viewpoint information is used in generating the second content.

Specifically, the renderer 220 may acquire the scene description in the form of an MMTP packet in which the scene description is packetized. There are no particular limitations on the method of acquiring the 360° video and 3D objects.

The 360° video may be converted to 2D video by a projective transformation such as ERP (Equirectangular projection) or cube mapping. Metadata indicating the type of projective transformation applied to the 360° video may be added. The 3D object may be encoded in a mesh format. ISO/IEC 14496-16 "Animation framework extension (AFX)" may be used for encoding the mesh format. The 3D object may be encoded in a point cloud format. ISO/IEC 23090-5 "Video-based Point Cloud Compression" may be used for encoding the point cloud format.

Here, the second content is compiled into one file in units separated by a certain time width. The certain time width may be 500 ms. For example, if the frame rate is 60 fps (frames per second), one file contains 30 frames.

The scene description is generated for each file and includes information for each frame that identifies the 360° video and 3D objects. For example, the scene description includes an information element (object_name) indicating the name of the 3D object in the frame, an information element (frame_number) indicating the frame number, an information element (translation_object) indicating the position of the 3D object in the frame, an information element (rotation_object) indicating the rotation of the 3D object in the frame, an information element (scale_object) indicating the size of the 3D object in the frame, etc.

Third, the renderer 220 outputs the specific content including the first content and the second content to the encoding processing unit 230. The renderer 220 may output the presentation time of the specific content together with the specific content to the encoding processing unit 230.

Here, the presentation time of the specific content may be corrected based on the delay time between the media processing device 200 and the user terminal 300. Specifically, the renderer 220 may calculate the presentation time (T'=T+ΔT) of the specific content provided from the media processing device 200 to the user terminal 300 based on the presentation time (T) and delay time (ΔT) of the specific content provided from the transmitting device 100 to the media processing device 200. The delay time (ΔT) may be a value that is predetermined in the media processing device 200, or may be a different value for each user terminal 300.

Fourthly, the renderer 220 may constitute a transmission unit that transmits viewpoint information used to generate the specific content to the user terminal 300. The viewpoint information used to generate the specific content may be transmitted from the encoding processing unit 230 to the user terminal 300.

For example, the transmission method of the viewpoint information and the specific content may be MMTP or HTTP. When MMTP is used as the transmission method of the specific content, the viewpoint information may be stored as metadata in OMAF (Omnidirectional Media Format) defined in ISO/IEC 23090-2.

The encoding processing unit 230 encodes the specific content generated by the renderer 220. In an embodiment, the encoding processing unit 230 may be an example of a transmission unit that transmits the specific content to the user terminal 300.

Furthermore, the encoding processing unit 230 may encode the presentation time of the specific content. The encoding processing unit 230 may transmit an information element indicating the presentation time to the user terminal 300 together with the specific content.

Here, any compression encoding method can be used as the compression encoding method used by the encoding processing unit 230. For example, the compression encoding method may be HEVC (High Efficiency Video Coding) or VVC (Versatile Video Coding).

As described above, the second content included in the specific content is generated based on viewpoint information, so the image included in the specific content can be treated as a 2D image that does not have freedom of viewpoint.

For example, the transmission control method used to start and end viewing of specific content may include RTSP (Real Time Streaming Protocol). The transmission method may be MMTP or HTTP. When MMTP is used as the transmission method, the specific content may be stored in OMAF defined in ISO/IEC 23090-2.

As shown in FIG. 2, the user terminal 300 has a detection unit 310, a decoding processing unit 320, and a renderer 330.

The detection unit 310 detects the user's viewpoint position and line of sight direction. The detection unit 310 may include an acceleration sensor or a GPS (Global Positioning System) sensor. The detection unit 310 may include a user I/F (e.g., a touch sensor, a keyboard, a mouse, a controller, etc.) that is manually input by the user. The detection unit 310 may transmit viewpoint information (viewpoint position and line of sight direction) to the media processing device 200. The detection unit 310 may output the viewpoint information (viewport) to the renderer 330.

The decoding processing unit 320 decodes the specific content received from the media processing device 200. The decoding processing unit 320 may decode the presentation time received from the media processing device 200. The decoding processing unit 320 may output the specific content to the renderer 330, or may output the presentation time to the renderer 330.

The renderer 330 outputs the specific content decoded by the decoding processing unit 320. The renderer 330 may output the specific content based on the presentation time decoded by the decoding processing unit 320. For example, the renderer 330 may output the video content included in the specific content to a display and output the audio content included in the specific content to a speaker.

Here, the renderer 330 may generate specific content in which the viewpoint position and line of sight direction are corrected based on the difference between the viewpoint information received from the media processing device 200 and the viewpoint information input from the detection unit 310.

(Second Content)
The second content according to the embodiment will be described below. Here, the second content at t=0, t=1, and t=2 will be described. The time intervals between t=0, t=1, and t=2 are not particularly limited.

For example, as shown in FIG. 3, at t=0, a 360° video may be displayed without a 3D object being displayed. The 360° video may be considered to be a background video of the 3D object. At t=1, a 3D object may be displayed in a form superimposed on the 360° video. Furthermore, at t=1, the position and rotation of the 3D object superimposed on the 360° video may be changed.

The above-mentioned scene description includes information elements that indicate the position, rotation, and size of the 3D object for each of t=0, t=1, and t=2, and allows the 3D object to be appropriately superimposed on the 360° video.

(How to watch)
A viewing method according to the embodiment will be described below, in which viewing of a specific content including a first content and a second content will be exemplified.

As shown in FIG. 4, in step S11, the user terminal 300 sends an RTSP SETUP to the media processing device. The RTSP SETUP is a message indicating that viewing of a specific content is to begin.

Here, the RTSP SETUP includes the IP address of the user terminal 300, the listening port number, the content identification information (content ID), etc. The RTSP SETUP may include capability information of the user terminal 300 for viewing specific content. The capability information may include the frame rate, the display resolution, etc. The display resolution may include the field of view (FoV). The capability information may include information elements indicating the encoding method and compression method supported by the user terminal 300.

Here, a case is illustrated in which the capability information of the user terminal 300 is directly notified to the media processing device 200, but the embodiment is not limited to this. The capability information of the user terminal 300 may be notified to the transmitting device 100, and then notified from the transmitting device 100 to the media processing device 200.

In step S12, the media processing device 200 transmits a response to the RTSP SETUP. Here, an ACK is transmitted as a response indicating that the RTSP SETUP has been accepted.

In step S21, the user terminal 300 transmits the initial viewpoint information to the media processing device 200. The initial viewpoint information may be transmitted in the MMT-SI format.

In step S22, the media processing device 200 generates initial specific content based on the initial viewpoint information (rendering process). For example, the media processing device 200 generates second content to be included in the initial specific content based on the initial viewpoint information and the scene description.

Here, the media processing device 200 may generate the initial specific content using a range wider than the display resolution of the user terminal 300 as the viewport. For example, the range wider than the display resolution may be a range of the display resolution + 20% in the horizontal direction and the display resolution + 20% in the vertical direction.

The media processing device 200 applies a compression encoding method to the initial specific content. Although not particularly limited, the compression encoding method may be HEVC or VVC.

In step S23, the media processing device 200 transmits the initial specific content corresponding to the initial viewpoint information to the user terminal 300. The media processing device 200 transmits the presentation time of the initial specific content to the user terminal 300. As described above, the presentation time (T') provided to the user terminal 300 may be determined based on the delay time (ΔT).

When a different value is used for the delay time (ΔT) for each user terminal 300, it is possible to specify it on the media processing device 200 side by including the transmission time of the RTSP SETUP in the above-mentioned RTSP SETUP.

The user terminal 300 outputs the specific content based on the presentation time (T'). The user terminal 300 may generate the specific content with the viewpoint position and line of sight corrected based on the difference between the viewpoint information received from the media processing device 200 and the viewpoint information input from the detection unit 310.

In step S31, the user terminal 300 transmits viewpoint information to the media processing device 200. The viewpoint information may be transmitted in the form of MMT-SI. Here, the user terminal 300 may transmit the viewpoint information at a predetermined period (e.g., 500 ms), or may transmit the viewpoint information in response to a change in at least one of the viewpoint position and the line of sight direction.

In step S32, the media processing device 200 generates specific content based on the viewpoint information received in step S31 (rendering process).

In step S33, the media processing device 200 transmits to the user terminal 300 the specific content corresponding to the viewpoint information received in step S31.

The processing of steps S31 to S33 is similar to the processing of steps S21 to S23, except that the viewpoint information received in step S31 is used instead of the initial viewpoint information. Therefore, details of the processing of steps S31 to S33 are omitted. The processing of steps S31 to S33 may be repeated at a predetermined interval, or may be repeated every time the user's viewpoint position or line of sight changes.

In step S41, the user terminal 300 sends an RTSP TEARDOWN to the media processing device. The RTSP TEARDOWN is a message indicating that viewing of a specific content is to end.

In step S42, the media processing device 200 sends a response to the RTSP TEARDOWN. Here, an ACK is sent as the response indicating that the RTSP TEARDOWN has been accepted.

In FIG. 4, a case where steps S11 and S12 are executed based on RTSP is illustrated, but the embodiment is not limited to this. Steps S11 and S12 may be executed based on MMTP or HTTP.

Similarly, although the example shows a case where steps S41 and S42 are performed on an RTSP basis, the embodiment is not limited to this. Steps S41 and S42 may be performed on an MMTP basis or on an HTTP basis.

In FIG. 4, a case where steps S31 to S33 are executed based on MMTP is illustrated, but the embodiment is not limited to this. Steps S31 to S33 may be executed based on another method (e.g., HTTP).

Similarly, although the above example illustrates a case in which steps S41 to S43 are executed based on MMTP, the embodiment is not limited to this. Steps S41 to S43 may be executed based on other methods (e.g., HTTP).

(Action and Effects)
In the embodiment, media processing device 200 generates specific content based on viewpoint information and then transmits the specific content to user terminal 300. With this configuration, there is no need for user terminal 300 to generate specific content including second content with viewpoint freedom, and user terminal 300 can present the specific content by simply providing viewpoint information to media processing device 200. Therefore, although a delay occurs between media processing device 200 and user terminal 300, the processing load on user terminal 300 can be reduced.

[Modification 1]
Modification 1 of the embodiment will be described below. Differences from the embodiment will be mainly described below.

In the first modification, a method for synchronizing a first content with a second content when a specific content includes both a first content and a second content is described.

Note that in the following, synchronization means that the presentation times of the first content (e.g., MPU) and the second content (file) are properly aligned. Therefore, synchronization may include the presentation times of the 2D video and the 3D object being aligned, and may also include the presentation times of the audio and the 3D object being aligned. Similarly, synchronization may include the presentation times of the 2D video and the 360° video being aligned, and may also include the presentation times of the audio and the 360° video being aligned.

In the first method, a case will be described in which the media processing device 200 synchronizes the first content and the second content based on the first control information (MMT-SI). The media processing device 200 uses the MMT-SI as an entry point to check whether or not a scene description (second content) is present, and if a scene description is present, it uses the MPU timestamp descriptor to identify the presentation time of specific content including the first content and the second content.

Specifically, as shown in Figure 5, 2D video and audio are presented based on the MPU timestamp descriptor (simply "timestamp" in Figure 5), so that the 2D video and audio can be synchronized.

On the other hand, the presentation time of the first frame included in the scene description is identified by referencing the MPU timestamp descriptor included in MMT-SI. The presentation times of the second and subsequent frames included in the scene description can be identified by the frame number included in the scene description and the frame rate of the second content. For example, if the frame rate is 30 fps, the presentation time of the nth frame is identified by adding 1/30 x n to the time identified by the MPU timestamp descriptor. However, the frame number of the first frame included in the scene description is "0".

In the first method, a case is exemplified in which the scene description does not include the presentation time of the first frame included in the scene description, but the scene description may include the presentation time of the first frame included in the scene description.

In the second method, a case will be described in which the media processing device 200 synchronizes the first content and the second content based on the second control information (scene description). The media processing device 200 uses the scene description as an entry point to check whether or not MMT-SI (first content) is present, and if MMT-SI is present, identifies the presentation time of specific content including the first content and the second content based on the presentation time included in the scene description.

In such a case, the scene description includes absolute time information indicating the presentation time of the second content. The absolute time information may be the presentation time of the first frame included in the scene description.

For example, the absolute time information may be generated using UTC as the reference time. The reference time may be the TAI or may be the time provided by a GPS. The reference time may be the time provided by an NTP server or may be the time provided by a PTP server. Furthermore, the absolute time information may be generated based on the same reference time as the MPU timestamp descriptor.

Furthermore, the scene description includes reference information for identifying the first content. The reference information may be information for identifying an MPU that constitutes the first content. In other words, the reference information is information for treating the first content (MPU) as an object included in the scene description.

Specifically, as shown in FIG. 6, the presentation time of the first frame included in the scene description is specified by the absolute time information included in the scene description. The presentation times of the second and subsequent frames included in the scene description can be specified by the frame number included in the scene description and the frame rate of the second content. For example, if the frame rate is 30 fps, the presentation time of the nth frame is specified by adding 1/30×n to the time specified by the MPU timestamp descriptor. However, the frame number of the first frame included in the scene description is "0".

On the other hand, 2D video and audio are presented based on the MPU timestamp descriptor (simply "timestamp" in FIG. 6), so that the 2D video and audio can be synchronized. Here, because the above-mentioned reference information is included in the scene description, the media processing device 200 can check whether or not there is a first content that should be presented together with the second content, based on the reference information included in the scene description.

In the second method, synchronization between 2D video and audio is achieved based on the MPU timestamp descriptor included in MMT-SI, but in modification example 1, synchronization between 2D video and audio may also be achieved based on information elements included in the scene description (absolute time information and reference information). In such a case, at least the MPU timestamp descriptor included in MMT-SI may be omitted. Furthermore, MMT-SI itself may be omitted.

When the reference time of the MPU timestamp descriptor included in the MMT-SI (hereinafter, the first reference time) differs from the reference time of the absolute time information included in the scene description (the second reference time), at least one of the first control information (MMT-SI) and the second control information (scene description) may include conversion information between the first reference time and the second reference time. For example, the MMT-SI may include an MPU timestamp descriptor expressed in the second reference time (e.g., a reference time other than UTC) in addition to the MPU timestamp descriptor expressed in the first reference time (e.g., UTC). The scene description may include absolute time information expressed in the first reference time (e.g., UTC) in addition to the absolute time information expressed in the second reference time (e.g., a reference time other than UTC).

The MPU timestamp descriptor included in the MMT-SI may be referred to as the first absolute time information, and the absolute time information included in the scene description may be referred to as the second absolute time information.

[Other embodiments]
Although the present invention has been described by the above disclosure, the description and drawings forming a part of this disclosure should not be understood as limiting the present invention. From this disclosure, various alternative embodiments, examples and operating techniques will become apparent to those skilled in the art.

In the above disclosure, a case in which the specific content includes both the first content and the second content has been exemplified, but the above disclosure is not limited to this. The specific content may include at least the second content.

Although not specifically mentioned in the above disclosure, terms related to MMT may be interpreted based on the contents specified in ISO/IEC 23008-1, ARIB STD-B60, ARIB TR-B39, etc.

In the above disclosure, an MPU timestamp descriptor is exemplified as the first absolute time information included in MMT-SI. However, the above disclosure is not limited to this. The first absolute time information included in MMT-SI may be an MPU extended timestamp descriptor.

Although not specifically mentioned in the above disclosure, the media processing device 200 may request a portion of the second content from the transmitting device 100 as necessary. This configuration can save bandwidth associated with the transmission of the second content and suppress an increase in the processing load on the media processing device 200.

In the above disclosure, MMTP is exemplified as the transmission method of the first content. However, the above disclosure is not limited to this. The transmission method of the first content may be a method conforming to ISO/IEC 23009-1 (hereinafter, MPEG-DASH (Dynamic Adaptive Stream over HTTP)). In such a case, the first control information may be MPD (Media Presentation Description). That is, in the above disclosure, MMT-SI may be read as MPD.

Although not specifically mentioned in the above disclosure, a program may be provided that causes a computer to execute each process performed by the transmitting device 100, the media processing device 200, and the user terminal 300. The program may also be recorded on a computer-readable medium. Using the computer-readable medium, it is possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or DVD-ROM.

Alternatively, a chip may be provided that is configured with a memory that stores programs for executing the processes performed by the transmitting device 100, the media processing device 200, and the user terminal 300, and a processor that executes the programs stored in the memory.

10...transmission system, 100...transmission device, 200...media processing device, 210...reception unit, 220...renderer, 230...encoding processing unit, 300...user terminal, 310...detection unit, 320...decoding processing unit, 330...renderer

Claims (6)

A receiving unit that receives viewpoint information from a user terminal;
A renderer that generates specific content including a first content that does not have a degree of freedom of viewpoint and a second content that has a degree of freedom of viewpoint;
a transmission unit that transmits the specific content generated by the renderer to the user terminal ,
The renderer may:
generating the first content based on control information associated with the first content;
a media processing device that generates, based on control information accompanying the second content and the viewpoint information, the second content including an image that can be handled as a two-dimensional image that does not have viewpoint freedom. The media processing device according to claim 1, wherein the transmission unit transmits the viewpoint information used in generating the specific content. The media processing device according to claim 1, wherein the transmission unit transmits an information element indicating the presentation time of the specific content together with the specific content. The media processing device according to claim 3 , wherein the renderer identifies the presentation time of the specific content based on control information associated with the first content . The media processing device according to claim 3 , wherein the renderer identifies the presentation time of the specific content based on control information associated with the second content. The media processing device according to claim 3 , wherein the renderer identifies the presentation time of the specific content based on a delay time between the media processing device and the user terminal.

Images