JP7238066B2 - Transmission method, reception method, transmission device and reception device - Google Patents

Description

The present invention relates to a transmission method, a reception method, a transmission device and a reception device.

With the advancement of broadcasting and communication services, the introduction of ultra-high-definition video content such as 8K (7680 x 4320 pixels; also referred to as 8K4K below) and 4K (3840 x 2160 pixels; also referred to as 4K2K below) is being considered. It is The receiving device needs to decode and display the encoded data of the received ultra-high-definition moving image in real time. It is difficult to decode moving images in real time with one decoder. Therefore, a method of parallelizing the decoding process using a plurality of decoders to reduce the processing load per decoder and to achieve real-time processing is being studied.

Also, encoded data is multiplexed based on a multiplexing scheme such as MPEG-2 TS (Transport Stream) or MMT (MPEG Media Transport) before being transmitted. For example, Non-Patent Document 1 discloses a technique for transmitting encoded media data in units of packets according to MMT.

Information technology - High efficiency coding and media delivery in heterogeneous environment - Part 1: MPEG media transport (MMT), ISO/IEC DIS 23008-1

However, since the encoded data is multiplexed based on a multiplexing scheme such as MPEG-2 TS or MMT before being transmitted, the receiving device extracts the encoded video data from the multiplexed data prior to decoding. must be separated. The process of separating encoded data from multiplexed data is hereinafter referred to as demultiplexing.

When parallelizing the decoding process, the receiving device needs to distribute the encoded data to be decoded to each decoder. At this time, the receiving device needs to analyze the encoded data itself. In particular, content such as 8K has a very high bit rate, so the processing load for analysis is large. As a result, there is a problem that the demultiplexing process becomes a bottleneck and reproduction in real time may not be possible.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a transmission method or a reception method capable of reducing the amount of processing involved in generating decoding target data.

To achieve the above object, a transmission method according to an aspect of the present invention includes a dividing step of dividing a picture into a plurality of regions; a coding step of generating coded data corresponding to each of the plurality of regions; a packetization step of storing the generated plurality of coded data in a plurality of packets; each of the plurality of coded data is associated one-to-one with a basic data unit that is a unit of data stored in one or more packets, and the plurality of coded data is stored in the one or more packets, and the header information of each packet includes (1) the basic data unit includes only the packet, (2) the basic data unit includes a plurality of packets and the packet is the first packet of the basic data unit; (3) the basic data unit includes a plurality of packets, and the packet is a packet other than the first and last packets of the basic data unit; and (4) identification information indicating whether the basic data unit includes a plurality of packets and the packet is the last packet of the basic data unit, and the packetizing step Then, the control information used for the decoding unit in the picture is stored in one packet different from the plurality of packets in which the plurality of encoded data are stored, and the header information of the packet is stored in the basic data unit. Identification information indicating that only the packet is included, and the header information of the packet is an offset indicating a bit length from the beginning of the encoded data of the picture to the beginning of the encoded data included in the packet. Contains information.

Further, a reception method according to an aspect of the present invention is a reception method in a reception apparatus including a plurality of decoding units, wherein a plurality of regions obtained by dividing a picture can be independently decoded. a receiving step of receiving a plurality of packets obtained by packetizing a plurality of encoded data obtained by being encoded in the above manner; each of the plurality of encoded data is associated on a one-to-one basis with a basic data unit that is a unit of data stored in one or more packets; each of the data is stored in the one or more packets, and the header information of each of the packets is such that (1) the basic data unit includes only the packet, and (2) the basic data unit includes a plurality of packets. (3) the basic data unit includes a plurality of packets, and the packet is a packet other than the first and last packets of the basic data unit; and (4) the basic data unit includes a plurality of packets, and the packet is the last packet of the basic data unit. The control information used in the decoding unit is stored in one packet different from the plurality of packets in which the plurality of encoded data are stored, and the header information of the packet is stored in the basic data unit of the packet and the header information of the packet includes offset information indicating the bit length from the beginning of the encoded data of the picture to the beginning of the encoded data contained in the packet. include.

In addition, these general or specific aspects may be realized by a system, method, integrated circuit, computer program, or a recording medium such as a computer-readable CD-ROM. and any combination of recording media.

INDUSTRIAL APPLICABILITY The present invention can provide a transmission method or a reception method that can reduce the amount of processing involved in generating decoding target data.

FIG. 4 is a diagram showing an example of dividing a picture into slice segments; FIG. 4 is a diagram showing an example of a PES packet sequence in which picture data is stored; FIG. 4 is a diagram showing an example of dividing a picture according to the embodiment; FIG. 10 is a diagram showing an example of dividing a picture according to a comparative example of the embodiment; FIG. 4 is a diagram showing an example of data of an access unit according to the embodiment; FIG. 1 is a block diagram of a transmission device according to an embodiment; FIG. 1 is a block diagram of a receiving device according to an embodiment; FIG. FIG. 4 is a diagram showing an example of an MMT packet according to the embodiment; FIG. FIG. 10 is a diagram showing another example of an MMT packet according to the embodiment; FIG. It is a figure which shows an example of the data input into each decoding part which concerns on embodiment. FIG. 3 is a diagram showing an example of an MMT packet and header information according to an embodiment; FIG. FIG. 9 is a diagram showing another example of data input to each decoding unit according to the embodiment; FIG. 4 is a diagram showing an example of dividing a picture according to the embodiment; 4 is a flow chart of a transmission method according to an embodiment; 1 is a block diagram of a receiving device according to an embodiment; FIG. 4 is a flow chart of a receiving method according to an embodiment; FIG. 3 is a diagram showing an example of an MMT packet and header information according to an embodiment; FIG. FIG. 3 is a diagram showing an example of an MMT packet and header information according to an embodiment; FIG.

(Knowledge on which the present invention is based)
2. Description of the Related Art In recent years, the resolution of displays for TVs, smartphones, tablet terminals, and the like has been increasing. Especially in broadcasting in Japan, 8K4K (resolution is 8K×4K) service is scheduled in 2020. Since it is difficult to decode super-high-resolution video such as 8K4K in real time with a single decoder, a method of decoding in parallel using a plurality of decoders is being studied.

H.264 standardized by MPEG and ITU. 264 and H. In a video coding method such as H.265, a transmitting device divides a picture into a plurality of regions called slices or slice segments, and encodes each divided region so that it can be decoded independently. Thus, for example, H. In the case of H.265, a receiver that receives broadcast separates the data for each slice segment from the received data and outputs the data for each slice segment to separate decoders, thereby realizing parallel decoding processing. .

FIG. 1 is a diagram showing an example of dividing one picture into four slice segments in HEVC. For example, the receiving device has four decoders, each decoding one of the four slice segments.

In conventional broadcasting, a transmitting device stores one picture (access unit in the MPEG system standard) in one PES packet, and multiplexes the PES packet into a TS packet train. Therefore, the receiving device separates the payload of the PES packet, analyzes the data of the access unit stored in the payload, separates each slice segment, and decodes the data of each separated slice segment. I had to output to

However, the present inventor has found that there is a problem that it is difficult to perform this processing in real time because the amount of processing required to analyze access unit data and separate slice segments is large.

FIG. 2 is a diagram showing an example in which picture data divided into slice segments is stored in the payload of a PES packet.

As shown in FIG. 2, for example, data of multiple slice segments (slice segments 1 to 4) are stored in the payload of one PES packet. Also, the PES packets are multiplexed into the TS packet train.

A transmission method according to an aspect of the present invention includes a dividing step of dividing a picture into a plurality of regions; An encoding step of generating encoded data corresponding to each, a packetization step of storing the generated plurality of encoded data in a plurality of packets, and a transmission step of transmitting the plurality of packets, In the packetizing step, the plurality of encoded data are stored in the plurality of packets so that the encoded data corresponding to different areas are not stored in one packet.

According to this, since the coded data of each area is stored in different packets, the receiving device can read the data stored in the packet without analyzing the coded data stored in the payload of the packet. It is possible to determine which area the encoded data belongs to. As a result, the receiving device can perform processing for generating data to be decoded by each decoding unit with a small amount of processing. In this way, the amount of processing involved in generating decoding target data in the receiving device is reduced.

For example, in the packetizing step, control information commonly used for all decoding units in the picture may be stored in a packet different from the plurality of packets in which the plurality of encoded data are stored. .

According to this, the receiving device can determine the packet in which the control information is stored without analyzing the encoded data stored in the payload of the packet. As a result, the amount of processing involved in generating decoding target data in the receiving device can be reduced.

Further, a reception method according to an aspect of the present invention is a reception method in a reception apparatus including a plurality of decoding units, wherein a plurality of regions obtained by dividing a picture can be independently decoded. receive a plurality of packets obtained by packetizing a plurality of encoded data obtained by encoding in such a manner that the encoded data of different regions are not stored in one packet combining control information included in one of the plurality of packets and commonly used for all decoding units in the picture with each of the plurality of coded data of the plurality of regions; A combining step of generating a plurality of combined data by doing so, and a decoding step of decoding the multiple combined data in parallel by the plurality of decoding units.

According to this, since the coded data of each area is stored in different packets, the receiving device can read the data stored in the packet without analyzing the coded data stored in the payload of the packet. It is possible to determine which area the encoded data belongs to. As a result, the receiving device can perform processing for generating data to be decoded by each decoding unit with a small amount of processing. In this way, the amount of processing involved in generating decoding target data in the receiving device is reduced.

For example, the control information may be stored in packets different from the plurality of packets in which the plurality of encoded data are stored.

According to this, the receiving device can determine the packet in which the control information is stored without analyzing the encoded data stored in the payload of the packet. As a result, the amount of processing involved in generating decoding target data in the receiving device can be reduced.

For example, in the combining step, header information of the packet may be used to determine which of the plurality of areas the data stored in the packet is encoded data of.

According to this, the receiving device can use the header information of the packet to determine which area the data stored in the packet is encoded data.

For example, each of the plurality of encoded data is associated one-to-one with a basic data unit that is a unit of data stored in one or more packets, and each of the plurality of encoded data corresponds to the one The header information of each packet stored in the above packets includes: (1) the basic data unit includes only the packet; (2) the basic data unit includes a plurality of packets and the packet is the first packet of the basic data unit, (3) the basic data unit includes a plurality of packets, and the packet is a packet other than the first and last packets of the basic data unit, and (4 ) the basic data unit includes a plurality of packets, and the packet is the last packet of the basic data unit, and in the combining step, (1) the or (2) the identification information indicating that the basic data unit includes only the packet, or (2) the basic data unit includes a plurality of packets and the packet is the first packet of the basic data unit. may be determined to be the beginning of the encoded data in each area.

According to this, the receiving device can use the header information of the packet to determine which area the data stored in the packet is encoded data.

For example, the header information of the packet further includes offset information indicating a bit length from the top of the encoded data of the picture containing the plurality of encoded data to the top of the encoded data included in the packet. , in the combining step, (1) the basic data unit includes only the packet, or (2) the basic data unit includes a plurality of packets, and the packet is at the beginning of the basic data unit. The head of payload data contained in the packet having the header information containing the identification information indicating that it is a packet and the offset information indicating the non-zero bit length is encoded in each of the areas. It may be determined to be the beginning of the data.

According to this, the receiving device can use the header information of the packet to determine which area the data stored in the packet is encoded data.

For example, the receiving method further includes each of the plurality of combined data based on at least one of a resolution of the picture, a method of dividing the picture into the plurality of regions, and processing capabilities of the plurality of decoders. may include a determining step of determining the decoding unit that decodes the .

According to this, the receiving device can appropriately allocate the encoded data of each region to the plurality of decoding units.

Further, the transmission device according to an aspect of the present invention further includes a dividing unit that divides a picture into a plurality of regions, and a coding that allows each of the plurality of regions to be independently decoded, thereby providing the plurality of an encoding unit that generates encoded data corresponding to each region; a packetizing unit that stores the generated plurality of encoded data in a plurality of packets; and a transmitting unit that transmits the plurality of packets. The packetization unit stores the plurality of encoded data in the plurality of packets so that the encoded data corresponding to different regions are not stored in one packet.

According to this, since the coded data of each area is stored in different packets, the receiving device can read the data stored in the packet without analyzing the coded data stored in the payload of the packet. It is possible to determine which area the encoded data belongs to. As a result, the receiving device can perform processing for generating data to be decoded by each decoding unit with a small amount of processing. In this way, the amount of processing involved in generating decoding target data in the receiving device is reduced.

In addition, a receiving device according to an aspect of the present invention provides a plurality of encoded regions obtained by encoding a plurality of regions obtained by dividing a picture so that they can be decoded independently. a receiving unit for receiving a plurality of packets obtained by packetizing the data so that the encoded data of the different regions are not stored in one packet; a combining unit that generates a plurality of combined data by combining control information commonly used for all decoding units in the picture with each of the plurality of encoded data of the plurality of regions; and a plurality of decoding units that decode the plurality of combined data in parallel.

According to this, since the coded data of each area is stored in different packets, the receiving device can read the data stored in the packet without analyzing the coded data stored in the payload of the packet. It is possible to determine which area the encoded data belongs to. As a result, the receiving device can perform processing for generating data to be decoded by each decoding unit with a small amount of processing. In this way, the amount of processing involved in generating decoding target data in the receiving device is reduced.

In addition, these general or specific aspects may be realized by a system, method, integrated circuit, computer program, or a recording medium such as a computer-readable CD-ROM. and any combination of recording media.

Hereinafter, embodiments will be specifically described with reference to the drawings.

It should be noted that each of the embodiments described below is a specific example of the present invention. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are examples and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in independent claims representing the highest concept will be described as arbitrary constituent elements.

(Embodiment)
In the following description, H.264 is used as a video encoding method. 265 will be described as an example. This embodiment can also be applied when using other coding schemes such as H.264.

FIG. 3 is a diagram showing an example of dividing an access unit (picture) into division units according to the present embodiment. The access unit is H.264. A feature called tiles, introduced by H.265, bisects each horizontally and vertically into a total of four tiles. Also, slice segments and tiles are associated one-to-one.

The reason for such halving in the horizontal and vertical directions will be described. First, decoding generally requires a line memory for storing one horizontal line of data. However, when the resolution is super high such as 8K4K, the size of the line memory increases because the size in the horizontal direction increases. In implementation of the receiver, it is desirable to be able to reduce the size of the line memory. Vertical partitioning is required to reduce the size of the line memory. Vertical partitioning requires a data structure called a tile. For these reasons tiles are used.

On the other hand, since images generally have high correlation in the horizontal direction, the ability to refer to a wider range in the horizontal direction improves the coding efficiency. Therefore, from the viewpoint of coding efficiency, it is desirable to divide the access unit horizontally.

By dividing the access unit into two halves in the horizontal and vertical directions, these two characteristics are compatible, and both implementation and coding efficiency can be considered. If a single decoder can decode a 4K2K moving image in real time, the 8K4K image is divided into four equal parts, and each slice segment is divided into 4K2K, so that the receiving device can decode 8K4K images in real time.

Next, the reason why tiles obtained by dividing access units in the horizontal and vertical directions and slice segments are associated one-to-one will be described. H. In H.265, an access unit is composed of a plurality of units called NAL (Network Adaptation Layer) units.

The payload of the NAL unit includes an access unit delimiter that indicates the start position of an access unit, an SPS (Sequence Parameter Set) that is initialization information commonly used in decoding for each sequence, and an initialization information commonly used in a picture when decoding. PPS (Picture Parameter Set) which is encoding information, SEI (Supplemental Enhancement Information) which is unnecessary for the decoding process itself but is necessary for processing and displaying the decoding result, and encoded data of slice segments, etc. to store The NAL unit header contains type information to identify the data stored in the payload.

Here, the transmission device multiplexes the encoded data by a multiplexing format such as MPEG-2 TS, MMT (MPEG Media Transport), MPEG DASH (Dynamic Adaptive Streaming over HTTP), or RTP (Real-time Transport Protocol). When parsing, the base unit can be set to the NAL unit. In order to store one slice segment in one NAL unit, it is desirable to divide the access unit into slice segments when dividing the access unit into regions. For this reason, the transmitter associates tiles with slice segments on a one-to-one basis.

Note that, as shown in FIG. 4, the transmitting device can collectively set tiles 1 to 4 as one slice segment. However, in this case, all tiles are stored in one NAL unit, making it difficult for the receiving device to separate the tiles in the multiplexing layer.

Slice segments include independent slice segments that can be decoded independently and reference slice segments that refer to independent slice segments. Here, a case where independent slice segments are used will be described.

FIG. 5 is a diagram showing an example of access unit data divided so that the boundaries between tiles and slice segments are aligned as shown in FIG. The access unit data consists of the NAL unit storing the access unit delimiter placed at the beginning, the SPS, PPS, and SEI NAL units placed after that, and the tiles 1 to 4 placed after that. and data of the slice segment in which the data is stored. Access unit data may not include some or all of the NAL units of SPS, PPS, and SEI.

Next, the configuration of transmitting apparatus 100 according to this embodiment will be described. FIG. 6 is a block diagram showing a configuration example of transmitting apparatus 100 according to this embodiment. This transmitting apparatus 100 comprises an encoding section 101 , a multiplexing section 102 , a modulating section 103 and a transmitting section 104 .

The encoding unit 101 converts an input image into H.264, for example. The encoded data is generated by encoding according to H.265. Also, the encoding unit 101 divides an access unit into four slice segments (tiles) and encodes each slice segment, as shown in FIG. 3, for example.

Multiplexing section 102 multiplexes the encoded data generated by encoding section 101 . Modulation section 103 modulates the data obtained by multiplexing. The transmitting section 104 transmits the modulated data as a broadcast signal.

Next, the configuration of receiving apparatus 200 according to this embodiment will be described. FIG. 7 is a block diagram showing a configuration example of receiving apparatus 200 according to this embodiment. This receiver 200 comprises a tuner 201 , a demodulator 202 , a demultiplexer 203 , a plurality of decoders 204A to 204D, and a display 205 .

Tuner 201 receives a broadcast signal. Demodulator 202 demodulates the received broadcast signal. The demodulated data is input to demultiplexing section 203 .

Demultiplexing section 203 demultiplexes the demodulated data into division units, and outputs the data for each division unit to decoding sections 204A to 204D. Here, a division unit is a division area obtained by dividing an access unit. It is a slice segment in H.265. Also here, the 8K4K image is split into four 4K2K images. Therefore, there are four decoding units 204A-204D.

A plurality of decoding units 204A to 204D operate in synchronization with each other based on a predetermined reference clock. Each decoding unit decodes the encoded data in division units according to the DTS (Decoding Time Stamp) of the access unit, and outputs the decoding result to the display unit 205 .

The display unit 205 generates an 8K4K output image by integrating multiple decoding results output from the multiple decoding units 204A to 204D. The display unit 205 displays the generated output image according to the separately acquired PTS (Presentation Time Stamp) of the access unit. Note that when integrating the decoding results, the display unit 205 performs filter processing such as a deblocking filter in a boundary area between division units adjacent to each other, such as a boundary between tiles, so that the boundary becomes visually inconspicuous. good too.

In the above description, the transmission device 100 and the reception device 200 that transmit or receive broadcasts have been described as examples, but content may be transmitted and received via a communication network. When the receiving device 200 receives content via a communication network, the receiving device 200 separates multiplexed data from IP packets received via a network such as Ethernet.

In broadcasting, the transmission path delay from when a broadcast signal is transmitted until it reaches the receiver 200 is constant. On the other hand, in a communication network such as the Internet, due to the effects of congestion, the transmission path delay until data transmitted from a server reaches the receiving device 200 is not constant. Therefore, the receiving apparatus 200 often does not perform strictly synchronous reproduction based on a reference clock such as PCR in broadcast MPEG-2 TS. Therefore, the receiving device 200 may display the 8K4K output image on the display unit according to the PTS without strictly synchronizing the decoding units.

Also, due to congestion of the communication network, etc., there is a case where decoding processing for all division units has not been completed at the time indicated by the PTS of the access unit. In this case, the receiving device 200 skips the display of the access unit, or delays the display until the decoding of at least four division units is completed and the generation of the 8K4K image is completed.

Note that content may be transmitted and received using both broadcasting and communication. This method can also be applied when reproducing multiplexed data stored in a recording medium such as a hard disk or memory.

Next, a method of multiplexing access units divided into slice segments when MMT is used as the multiplexing method will be described.

FIG. 8 is a diagram showing an example of packetizing HEVC access unit data into MMT. Although SPS, PPS, SEI, etc. do not necessarily have to be included in the access unit, the case where they exist is exemplified here.

The NAL units, such as the access unit delimiter, SPS, PPS, and SEI, which are arranged before the head slice segment in the access unit, are collectively stored in the MMT packet #1. Subsequent slice segments are stored in separate MMT packets for each slice segment.

Note that, as shown in FIG. 9, the NAL unit arranged before the top slice segment in the access unit may be stored in the same MMT packet as the top slice segment.

Also, when NAL units such as End-of-Sequence or End-of-Bitstream, which indicate the end of a sequence or stream, are added after the final slice segment, these are included in the same MMT packet as the final slice segment. Stored. However, since NAL units such as End-of-Sequence or End-of-Bitstream are inserted at the end point of the decoding process, or at the connection point of two streams, the receiving device 200 inserts these NAL units. , can be easily obtained at the multiplexing layer. In this case, these NAL units may be stored in MMT packets separate from slice segments. This allows receiving apparatus 200 to easily separate these NAL units in the multiplexing layer.

Note that TS, DASH, RTP, or the like may be used as the multiplexing method. In these schemes as well, transmitting apparatus 100 stores different slice segments in different packets. This ensures that receiving apparatus 200 can separate slice segments in the multiplexing layer.

For example, when TS is used, encoded data is PES-packetized in units of slice segments. When RTP is used, encoded data is RTP-packetized in units of slice segments. Also in these cases, the NAL unit arranged before the slice segment and the slice segment may be packetized separately like MMT packet #1 shown in FIG.

When TS is used, transmitting apparatus 100 indicates the unit of data stored in a PES packet by using data alignment descriptors. Also, since DASH is a method of downloading MP4 format data units called segments using HTTP or the like, the transmitting apparatus 100 does not packetize encoded data for transmission. Therefore, transmitting apparatus 100 creates subsamples in units of slice segments and stores information indicating the storage positions of subsamples in the MP4 header so that receiving apparatus 200 can detect slice segments in the multiplexing layer of MP4. You may

MMT packetization of slice segments is described in detail below.

As shown in FIG. 8, by packetizing encoded data, data commonly referred to when decoding all slice segments in an access unit such as SPS and PPS is stored in MMT packet #1. In this case, receiving device 200 concatenates the payload data of MMT packet #1 and the data of each slice segment, and outputs the obtained data to the decoding unit. In this way, receiving apparatus 200 can easily generate input data to the decoding unit by concatenating payloads of a plurality of MMT packets.

FIG. 10 is a diagram showing an example in which input data to decoding units 204A to 204D are generated from the MMT packets shown in FIG. Demultiplexing section 203 concatenates the payload data of MMT packet #1 and MMT packet #2, thereby generating data necessary for decoding section 204A to decode slice segment 1. FIG. Demultiplexing section 203 similarly generates input data for decoding section 204B to decoding section 204D. That is, demultiplexing section 203 generates input data for decoding section 204B by concatenating the payload data of MMT packet #1 and MMT packet #3. Demultiplexing section 203 generates input data for decoding section 204C by concatenating the payload data of MMT packet #1 and MMT packet #4. Demultiplexing section 203 generates input data for decoding section 204D by concatenating the payload data of MMT packet #1 and MMT packet #5.

Note that the demultiplexing unit 203 removes NAL units that are not necessary for decoding processing, such as access unit delimiters and SEI, from the payload data of MMT packet #1, and removes only the NAL units of SPS and PPS that are necessary for decoding processing. may be separated and added to the slice segment data.

When the encoded data is packetized as shown in FIG. 9, demultiplexing section 203 outputs MMT packet #1 including the leading data of the access unit in the multiplexing layer to first decoding section 204A. . In addition, the demultiplexing unit 203 analyzes the MMT packet including the leading data of the access unit in the multiplexing layer, separates the SPS and PPS NAL units, and divides the separated SPS and PPS NAL units into the second and subsequent slices. Input data for each of the second and subsequent decoding units is generated by adding it to each of the segment data.

Furthermore, receiving device 200 uses the information included in the header of the MMT packet to determine the type of data stored in the MMT payload and the slice segment in the access unit when the slice segment is stored in the payload. It is desirable to be able to identify the index number. Here, the type of data includes pre-slice segment data (the NAL units arranged before the head slice segment in the access unit are collectively called in this way), and slice segment data. Either. When storing an MPU fragmented unit such as a slice segment in an MMT packet, a mode for storing an MFU (Media Fragment Unit) is used. When using this mode, transmitting apparatus 100 converts Data Unit, which is the basic unit of data in MFU, to samples (data units in MMT and corresponds to access units) or subsamples (samples). divided unit) can be set.

At this time, the header of the MMT packet includes a field called Fragmentation indicator and a field called Fragment counter.

Fragmentation indicator indicates whether the data stored in the payload of the MMT packet is a fragmented Data unit, and if the fragment is fragmented, the fragment is the first or last fragment in the Data unit, or , indicates whether the fragment is neither the first nor the last. In other words, the fragmentation indicator included in the header of a certain packet is (1) only the packet is included in the Data unit, which is a basic data unit, (2) the Data unit is divided and stored in a plurality of packets, and (3) the data unit is divided and stored in a plurality of packets, and the packet is a packet other than the first and last packets of the data unit; and (4) Identification information indicating whether a Data unit is divided and stored in a plurality of packets, or whether the packet is the last packet of the Data unit.

Fragment counter is an index number indicating which fragment in the data unit the data stored in the MMT packet corresponds to.

Therefore, transmitting apparatus 100 sets samples in MMT to Data unit, and sets pre-slice segment data and each slice segment in units of fragment of Data unit, respectively, so that receiving apparatus 200 can generate the header of the MMT packet. can be used to identify the type of data stored in the payload. That is, the demultiplexing unit 203 can refer to the header of the MMT packet to generate input data to each of the decoding units 204A to 204D.

FIG. 11 is a diagram showing an example in which the sample is set in the Data unit, and the pre-slice segment data and the slice segment are packetized as fragments of the Data unit.

The slice segment pre-data and the slice segment are divided into five fragments from fragment #1 to fragment #5. Each fragment is stored in a separate MMT packet. At this time, the values of the fragmentation indicator and the fragment counter included in the header of the MMT packet are as illustrated.

For example, the Fragment indicator is a binary 2-bit value. The fragment indicator of MMT packet #1, which is the head of the data unit, the fragment indicator of MMT packet #5, which is the last, and the fragment indicators of MMT packet #2 to MMT packet #4, which are packets in between, are different. set to the value Specifically, the Fragment indicator of MMT packet #1, which is the head of the Data unit, is set to 01, the Fragment indicator of MMT packet #5, which is the end, is set to 11, and from MMT packet #2, which is the packet between The Fragment indicator is set to 10 up to MMT packet #4. Note that the Fragment indicator is set to 00 when the Data unit contains only one MMT packet.

In addition, the fragment counter is 4, which is a value obtained by subtracting 1 from 5, which is the total number of fragments, in MMT packet #1, decreases by 1 in subsequent packets, and is 0 in the last MMT packet #5. be.

Therefore, receiving apparatus 200 can identify an MMT packet storing pre-slice segment data using either a Fragment indicator or a Fragment counter. Also, receiving device 200 can identify the MMT packet storing the N-th slice segment by referring to the Fragment counter.

The header of the MMT packet separately includes the sequence number within the MPU of the Movie Fragment to which the Data unit belongs, the sequence number of the MPU itself, and the sequence number within the Movie Fragment of the sample to which the Data unit belongs. The demultiplexing unit 203 can uniquely determine the sample to which the Data unit belongs by referring to these.

Furthermore, since the demultiplexing unit 203 can determine the index number of the fragment in the data unit from the fragment counter or the like, even when packet loss occurs, the slice segment stored in the fragment can be uniquely identified. For example, even if fragment #4 shown in FIG. 11 cannot be acquired due to packet loss, demultiplexing section 203 knows that the fragment received next to fragment #3 is fragment #5. can be correctly output to decoding section 204D instead of decoding section 204C.

Note that when a transmission line that guarantees that no packet loss occurs is used, the demultiplexing unit 203 refers to the header of the MMT packet to determine the type of data stored in the MMT packet, or the slice segment. It suffices to periodically process arriving packets without determining the index number of the . For example, when an access unit is transmitted by pre-slice data and a total of 5 MMT packets of 4 slice segments, after determining the pre-slice data of the access unit to start decoding, the receiving apparatus 200 , by sequentially processing the received MMT packets, the pre-slice data and the data of the four slice segments can be obtained in order.

Modifications of packetization will be described below.

The slice segment does not necessarily have to be obtained by dividing the plane of the access unit in both the horizontal direction and the vertical direction. As shown in FIG. 1, the slice segment may be obtained by dividing the access unit only in the horizontal direction. However, it may be divided only in the vertical direction.

Also, if the access unit is divided only horizontally, tiles need not be used.

Also, the number of in-plane divisions in the access unit is arbitrary and is not limited to four. However, the region sizes of slice segments and tiles conform to H.264. It must be at least the lower bound of an encoding standard such as H.265.

Transmitting apparatus 100 may store identification information indicating an intra-plane division method in an access unit in an MMT message, a TS descriptor, or the like. For example, information may be stored that indicates the number of divisions in the plane in the horizontal direction and the number of divisions in the vertical direction. Alternatively, identification information unique to the division method, such as halving in the horizontal direction and the vertical direction as shown in FIG. 3, or dividing into 4 equal parts in the horizontal direction as shown in FIG. may be assigned. For example, when the access unit is divided as shown in FIG. 3, the identification information indicates mode 1, and when the access unit is divided as shown in FIG. 1, the identification information indicates mode 1. .

In addition, the multiplexing layer may include information indicating restrictions on coding conditions related to the intra-plane division method. For example, information indicating that one slice segment consists of one tile may be used. Alternatively, reference blocks for motion compensation when decoding slice segments or tiles are limited to slice segments or tiles at the same position in the screen, or are limited to blocks within a predetermined range in adjacent slice segments. Information indicating such as that may be used.

Also, transmitting apparatus 100 may switch whether to divide an access unit into a plurality of slice segments according to the resolution of a moving image. For example, if the moving image to be processed has a resolution of 4K2K, the transmission device 100 does not perform intra-plane division, and if the moving image to be processed has a resolution of 8K4K, the access unit may be divided into four. good. By prescribing the division method in the case of 8K4K moving images, the receiving apparatus 200 obtains the resolution of the received moving image to determine whether or not to divide the screen and the division method, and decode. You can switch between actions.

In addition, receiving device 200 can detect the presence or absence of in-plane division by referring to the header of the MMT packet. For example, if the access unit is not fragmented, if the MMT Data unit is set to sample, the Data unit is not fragmented. Therefore, receiving apparatus 200 can determine that the access unit is not divided when the value of Fragment counter included in the header of the MMT packet is always zero. Alternatively, receiving device 200 may detect whether the value of the fragmentation indicator is always 01. Receiving apparatus 200 can determine that the access unit is not fragmented even when the value of the fragmentation indicator is always 01.

In addition, receiving apparatus 200 can cope with cases where the number of in-plane divisions in an access unit does not match the number of decoding units. For example, when receiving device 200 includes two decoding units 204A and 204B capable of decoding 8K2K encoded data in real time, demultiplexing unit 203 sends 8K4K encoded data to decoding unit 204A. outputs two of the four slice segments that make up the .

FIG. 12 is a diagram showing an operation example when MMT packetized data as shown in FIG. 8 is input to two decoding units 204A and 204B. Here, it is desirable that receiving apparatus 200 can integrate and output the decoding results of decoding sections 204A and 204B as they are. Therefore, demultiplexing section 203 selects slice segments to be output to decoding sections 204A and 204B so that the decoding results of decoding sections 204A and 204B are spatially continuous.

Also, the demultiplexing unit 203 may select a decoding unit to be used according to the resolution or frame rate of the encoded data of the moving image. For example, when receiving device 200 includes four 4K2K decoding units, if the resolution of the input image is 8K4K, receiving device 200 performs decoding processing using all four decoding units. Also, if the resolution of the input image is 4K2K, the reception device 200 performs decoding processing using only one decoding unit. Alternatively, the demultiplexing unit 203, even if the plane is divided into four, if 8K4K can be decoded in real time by a single decoding unit, all the division units are integrated into one decoding unit output to

Furthermore, receiving apparatus 200 may determine the decoding unit to be used in consideration of the frame rate. For example, if the receiving apparatus 200 includes two decoding units whose upper limit of the frame rate that can be decoded in real time is 60 fps when the resolution is 8K4K, there is a case where 8K4K encoded data of 120 fps is input. be. At this time, assuming that the plane is composed of four division units, slice segments 1 and 2 are input to the decoding unit 204A, and slice segments 3 and 4 are input as in the example of FIG. It is input to the decoding unit 204B. Since each of the decoding units 204A and 204B can decode in real time up to 120 fps for 8K2K (half the resolution of 8K4K), decoding processing is performed by these two decoding units 204A and 204B.

Also, even if the resolution and frame rate are the same, the profile or level in the encoding system, or the H.264 standard may be used. 264 or H.264. If the encoding method itself such as H.265 is different, the amount of processing will be different. Therefore, receiving apparatus 200 may select a decoding unit to be used based on these pieces of information. In addition, if the receiving apparatus 200 cannot decode all the encoded data received by broadcasting or communication, or if all the slice segments or tiles that make up the region selected by the user cannot be decoded, the decoding unit may automatically determine decodable slice segments or tiles within the processing range of . Alternatively, receiving device 200 may provide a user interface for the user to select a region to decode. At this time, receiving device 200 may display a warning message indicating that all regions cannot be decoded, or may display information indicating the number of decodable regions, slice segments, or tiles.

Moreover, the above method can also be applied when MMT packets storing slice segments of the same encoded data are transmitted and received using a plurality of transmission paths such as broadcasting and communication.

In addition, transmitting apparatus 100 may perform encoding such that the regions of each slice segment overlap in order to obscure boundaries between division units. In the example shown in FIG. 13, an 8K4K picture is divided into four slice segments 1-4. Each of slice segments 1-3 is, for example, 8K×1.1K, and slice segment 4 is 8K×1K. Adjacent slice segments also overlap each other. By doing this, the motion compensation at the time of encoding can be executed efficiently at the boundaries of the four divisions indicated by the dotted lines, so that the image quality of the boundary portions is improved. In this way, deterioration of image quality in the boundary portion is reduced.

In this case, the display unit 205 cuts out an 8K×1K area from an 8K×1.1K area, and integrates the obtained areas. In addition, transmitting apparatus 100 includes information indicating whether or not slice segments are encoded with overlap and the range of overlap in the multiplexing layer or encoded data, and separately transmits the information. good too.

A similar technique can be applied when tiles are used.

The operation flow of the transmission device 100 will be described below. FIG. 14 is a flow chart showing an operation example of the transmission device 100 .

First, the encoding unit 101 divides a picture (access unit) into a plurality of slice segments (tiles), which are a plurality of regions (S101). Next, the encoding unit 101 generates encoded data corresponding to each of the plurality of slice segments by encoding each of the plurality of slice segments so that they can be independently decoded (S102). Note that the encoding unit 101 may encode a plurality of slice segments by a single encoding unit, or may perform parallel processing by a plurality of encoding units.

Next, the multiplexing unit 102 multiplexes the multiple encoded data by storing the multiple encoded data generated by the encoding unit 101 in multiple MMT packets (S103). Specifically, as shown in FIGS. 8 and 9, multiplexing section 102 combines a plurality of encoded data into a plurality of slice segments so that encoded data corresponding to different slice segments are not stored in one MMT packet. Store in MMT packet. Further, as shown in FIG. 8, multiplexing section 102 sets control information commonly used for all decoding units in a picture to a plurality of MMT packets #2 to # in which a plurality of encoded data are stored. 5 is stored in MMT packet #1 different from 5. Here, the control information includes at least one of access unit delimiter, SPS, PPS and SEI.

Note that multiplexing section 102 may store the control information in the same MMT packet as one of a plurality of MMT packets in which a plurality of encoded data are stored. For example, as shown in FIG. 9, multiplexing section 102 stores control information in the first MMT packet (MMT packet #1 in FIG. 9) among multiple MMT packets storing multiple pieces of encoded data. You may

Finally, transmitting device 100 transmits a plurality of MMT packets. Specifically, the modulation section 103 modulates the data obtained by multiplexing, and the transmission section 104 transmits the modulated data (S104).

FIG. 15 is a block diagram showing a configuration example of receiving apparatus 200, and is a diagram showing in detail the configuration of demultiplexing section 203 shown in FIG. 7 and its subsequent stage. As shown in FIG. 15 , receiving device 200 further includes decoding instruction section 206 . Demultiplexing section 203 also includes type determining section 211 , control information acquiring section 212 , slice information acquiring section 213 , and decoded data generating section 214 .

The operation flow of the receiving device 200 will be described below. FIG. 16 is a flow chart showing an operation example of the receiving device 200 . Here, the operation for one access unit is shown. When the decryption processing of a plurality of access units is executed, the processing of this flowchart is repeated.

First, the receiving device 200 receives, for example, a plurality of packets (MMT packets) generated by the transmitting device 100 (S201).

Next, the type determination unit 211 acquires the type of encoded data stored in the received packet by analyzing the header of the received packet (S202).

Next, the type determination unit 211 determines whether the data stored in the received packet is pre-slice segment data or slice segment data based on the type of the acquired encoded data (S203). .

If the data stored in the received packet is pre-slice segment data (Yes in S203), the control information acquisition unit 212 acquires pre-slice segment data of the access unit to be processed from the payload of the received packet, The pre-segment data is stored in memory (S204).

On the other hand, if the data stored in the received packet is slice segment data (No in S203), receiving apparatus 200 uses the header information of the received packet to determine whether the data stored in the received packet is plural. It is determined which region of the region the coded data belongs to. Specifically, the slice information acquisition unit 213 acquires the index number Idx of the slice segment stored in the received packet by analyzing the header of the received packet (S205). Specifically, the index number Idx is the index number in the Movie Fragment of the access unit (sample in MMT).

Note that the process of step S205 may be collectively performed in step S202.

Next, the decoded data generation unit 214 determines a decoding unit that decodes the slice segment (S206). Specifically, the index number Idx and a plurality of decoding units are associated in advance, and the decoded data generation unit 214 causes the decoding unit corresponding to the index number Idx acquired in step S205 to decode the slice segment. Decide which decoder to use.

12, the decoded data generation unit 214 uses the resolution of the access unit (picture), the method of dividing the access unit into multiple slice segments (tiles), and the multiple A decoder for decoding the slice segment may be determined based on at least one of the processing capabilities of the decoder. For example, the decoded data generator 214 determines the access unit division method based on the identification information in the MMT message or the descriptor such as the TS section.

Next, the decoded data generation unit 214 generates control information commonly used for all decoding units in a picture included in any of the plurality of packets, and each of the plurality of encoded data of the plurality of slice segments. are combined to generate a plurality of input data (combined data) to be input to a plurality of decoding units. Specifically, the decoded data generation unit 214 acquires slice segment data from the payload of the received packet. The decoded data generation unit 214 combines the pre-slice segment data stored in the memory in step S204 and the acquired slice segment data to generate the input data to the decoding unit determined in step S206. (S207).

After step S204 or S207, if the data of the received packet is not the final data of the access unit (No in S208), the processing after step S201 is performed again. That is, the above processing is repeated until input data to the plurality of decoding units 204A to 204D corresponding to all slice segments included in the access unit are generated.

Note that the timing at which packets are received is not limited to the timing shown in FIG. 16, and a plurality of packets may be received in advance or sequentially and stored in a memory or the like.

On the other hand, if the data of the received packet is the final data of the access unit (Yes in S208), the decoding instruction unit 206 outputs the multiple pieces of input data generated in step S207 to the corresponding decoding units 204A-204D. (S209).

Next, the multiple decoding units 204A to 204D generate multiple decoded images by decoding multiple input data in parallel according to the DTS of the access unit (S210).

Finally, the display unit 205 generates a display image by combining the multiple decoded images generated by the multiple decoding units 204A to 204D, and displays the display image according to the PTS of the access unit (S211).

Receiving apparatus 200 acquires the DTS and PTS of the access unit by analyzing payload data of an MMT packet that stores MPU header information or Movie Fragment header information. Also, when TS is used as the multiplexing method, receiving device 200 acquires the DTS and PTS of the access unit from the header of the PES packet. Receiving device 200 acquires the DTS and PTS of the access unit from the header of the RTP packet when RTP is used as the multiplexing method.

Further, the display unit 205 may perform filter processing such as a deblocking filter at the boundary between adjacent division units when integrating the decoding results of the plurality of decoding units. Since filtering is not necessary when displaying the decoding results of a single decoding unit, the display unit 205 performs processing according to whether filtering is to be performed on the boundaries of the decoding results of a plurality of decoding units. You can switch. Whether or not filter processing is necessary may be defined in advance according to the presence or absence of division. Alternatively, information indicating whether filtering is required may be stored separately in the multiplexing layer. Information required for filtering, such as filter coefficients, may also be stored in SPS, PPS, SEI, or slice segments. The decoding units 204A to 204D or the demultiplexing unit 203 acquires the information by analyzing the SEI, and outputs the acquired information to the display unit 205. FIG. The display unit 205 performs filtering using these pieces of information. Note that when these pieces of information are stored in slice segments, it is desirable that the decoding units 204A to 204D acquire these pieces of information.

In the above description, an example in which two types of data are stored in a fragment, that is, pre-slice segment data and slice segment data, is shown, but the number of data types may be three or more. In this case, classification is performed according to the type in step S203.

Also, when the data size of the slice segment is large, transmitting apparatus 100 may fragment the slice segment and store the fragment in the MMT packet. That is, transmitting apparatus 100 may fragment the pre-slice segment data and the slice segment. In this case, setting the access unit equal to the data unit as in the example of packetization shown in FIG. 11 causes the following problem.

For example, when slice segment 1 is divided into three fragments, slice segment 1 is divided into three packets with Fragment counter values of 1 to 3 and transmitted. Also, after slice segment 2, the Fragment counter value is 4 or more, and the value of the Fragment counter cannot be associated with the data stored in the payload. Therefore, receiving device 200 cannot identify the packet that stores the head data of the slice segment from the information in the header of the MMT packet.

In such a case, receiving device 200 may analyze the payload data of the MMT packet to identify the start position of the slice segment. Here, H.I. 264 or H.264. As a format for storing NAL units in the multiplexing layer in H.265, a format called a byte stream format in which a start code consisting of a specific bit string is added immediately before the NAL unit header, and a field indicating the size of the NAL unit is added. There are two types: a format called NAL size format.

Byte stream formats are used in MPEG-2 systems, RTP, and the like. The NAL size format is used in MP4, DASH and MMT, etc. that use MP4.

When the byte stream format is used, receiving device 200 analyzes whether the head data of the packet matches the start code. If the head data of the packet matches the start code, receiving device 200 obtains the NAL unit type from the following NAL unit header, and determines whether the data included in the packet is slice segment data. can detect what

On the other hand, in the case of the NAL size format, receiving device 200 cannot detect the starting position of the NAL unit based on the bit string. Therefore, in order to acquire the starting position of the NAL unit, receiving apparatus 200 needs to shift the pointer by reading data by the size of the NAL unit in order from the leading NAL unit of the access unit.

However, in the header of the MPU or Movie Fragment in MMT, the size of the sub-sample unit is indicated, and if the sub-sample corresponds to the pre-slice data or slice segment, the receiving device 200 is based on the size information of the sub-sample The starting position of each NAL unit can be identified. Therefore, transmitting apparatus 100 may include information indicating whether sub-sample unit information exists in MPU or Movie Fragment in information such as MPT in MMT that receiving apparatus 200 acquires at the start of data reception. .

The MPU data is based on the MP4 format and expanded. In MP4, H. 264 or H.264. There are a mode in which parameter sets such as H.265 SPS and PPS can be stored as sample data and a mode in which they cannot be stored. Also, information for specifying this mode is indicated as the entry name of SampleEntry. If the storable mode is used and the parameter set is included in the sample, the receiving device 200 acquires the parameter set by the method described above.

On the other hand, if a non-storable mode is used, the parameter set is stored as Decoder Specific Information in the SampleEntry or stored using a stream for the parameter set. Here, since parameter set streams are not generally used, transmitting apparatus 100 preferably stores parameter sets in Decoder Specific Information. In this case, receiving device 200 analyzes SampleEntry transmitted as MPU metadata or Movie Fragment metadata in an MMT packet, and obtains a parameter set referred to by an access unit.

When the parameter set is stored as sample data, receiving apparatus 200 can obtain the parameter set necessary for decoding by referring only to the sample data without referring to SampleEntry. At this time, transmitting apparatus 100 does not need to store the parameter set in SampleEntry. By doing so, transmitting apparatus 100 can use the same SampleEntry in different MPUs, so that the processing load of transmitting apparatus 100 when generating MPUs can be reduced. Furthermore, there is the advantage that receiving apparatus 200 does not need to refer to the parameter set in SampleEntry.

Alternatively, transmitting device 100 may store one default parameter set in SampleEntry and store the parameter set referred to by the access unit in sample data. In conventional MP4, it was common to store parameter sets in SampleEntry. Therefore, if there is no parameter set in SampleEntry, there is a possibility that a receiving device will stop playing. This problem can be solved by using the above method.

Alternatively, transmitting device 100 may store the parameter set in the sample data only when a parameter set different from the default parameter set is used.

In both modes, it is possible to store the parameter set in SampleEntry, so transmitting apparatus 100 may always store the parameter set in VisualSampleEntry, and receiving apparatus 200 may always acquire the parameter set from VisualSampleEntry.

In the MMT standard, MP4 header information such as Moov and Moof is called MPU meta, but transmitting apparatus 100 does not necessarily need to transmit MPU meta. Further, the receiving device 200 determines whether SPS and PPS are stored in the sample data based on ARIB (Association of Radio Industries and Businesses) standard services, asset types, or whether or not MPU meta is transmitted. It is also possible to determine

FIG. 17 is a diagram showing an example in which the pre-slice segment data and each slice segment are set to different data units.

In the example shown in FIG. 17, the size of the data before the slice segment and the data size of slice segment 1 to slice segment 4 are Length#1 to Length#5, respectively. Field values of Fragmentation indicator, Fragment counter, and Offset included in the header of the MMT packet are shown in the figure.

Here, Offset indicates the bit length (offset) from the beginning of the coded data of the sample (access unit or picture) to which the payload data belongs to the first byte of the payload data (coded data) included in the MMT packet. This is offset information. The value of the Fragment counter is described as starting from a value obtained by subtracting 1 from the total number of fragments, but it may start from another value.

FIG. 18 is a diagram showing an example of fragmentation of Data units. In the example shown in FIG. 18, slice segment 1 is divided into three fragments, which are stored in MMT packet #2 to MMT packet #4, respectively. Also at this time, if the data size of each fragment is Length#2_1 to Length#2_3, the values of each field are as shown in the figure.

Thus, when a data unit such as a slice segment is set to Data unit, the beginning of the access unit and the beginning of the slice segment can be determined as follows based on the field values of the MMT packet header.

The beginning of the payload in a packet whose Offset value is 0 is the beginning of the access unit.

The head of the payload of a packet whose Offset value is different from 0 and whose Fragmentation indicator value is 00 or 01 is the head of the slice segment.

In addition, when data unit fragmentation does not occur and packet loss does not occur, receiving apparatus 200 stores in the MMT packet based on the number of slice segments acquired after detecting the beginning of the access unit. You can specify the index number of the slice segment.

Similarly, even when the Data unit of the pre-slice segment data is fragmented, receiving apparatus 200 can detect the beginning of the access unit and the slice segment.

Further, even when a packet loss occurs, or when the SPS, PPS, and SEI included in the pre-slice segment data are set to separate data units, the receiving apparatus 200 slices data based on the analysis result of the MMT header. By identifying the MMT packet storing the head data of the segment and then analyzing the header of the slice segment, the starting position of the slice segment or tile within the picture (access unit) can be identified. The amount of processing involved in parsing the slice header is small, and the processing load is not a problem.

In this way, each of a plurality of coded data of a plurality of slice segments is associated one-to-one with a basic data unit (data unit) that is a unit of data stored in one or more packets. Also, each of the multiple encoded data is stored in one or more MMT packets.

Header information of each MMT packet includes a Fragmentation indicator (identification information) and an Offset (offset information).

Receiving apparatus 200 determines that the beginning of payload data included in a packet having header information including a fragmentation indicator with a value of 00 or 01 is the beginning of encoded data of each slice segment. . Specifically, the beginning of the payload data contained in the packet having header information containing the Offset value not equal to 0 and the Fragmentation indicator whose value is 00 or 01 is the beginning of the encoded data of each slice segment. I judge.

Also, in the example of FIG. 17, the head of the data unit is either the head of the access unit or the head of the slice segment, and the value of the fragmentation indicator is 00 or 01. Furthermore, receiving apparatus 200 refers to the type of the NAL unit and determines whether the head of the Data Unit is an access unit delimiter or a slice segment. It is also possible to detect the beginning or the beginning of a slice segment.

In this way, transmitting apparatus 100 performs packetization so that the head of the NAL unit always starts from the head of the payload of the MMT packet, including the case where the pre-slice segment data is divided into a plurality of data units. Therefore, receiving apparatus 200 can detect the beginning of the access unit or slice segment by analyzing the fragmentation indicator and the NAL unit header. The NAL unit type is present in the first byte of the NAL unit header. Therefore, receiving device 200 can acquire the type of the NAL unit by additionally analyzing 1-byte data when analyzing the header portion of the MMT packet. In the case of audio, receiving apparatus 200 only needs to be able to detect the beginning of the access unit, and may determine based on whether the value of the fragmentation indicator is 00 or 01.

Further, as described above, when encoded data encoded so as to enable division decoding is stored in PES packets of MPEG-2 TS, transmitting apparatus 100 can use data alignment descriptors. be. An example of a method of storing encoded data in a PES packet will be described in detail below.

For example, in HEVC, the transmitting device 100 can indicate whether data stored in a PES packet is an access unit, a slice segment, or a tile by using a data alignment descriptor. Alignment types in HEVC are defined as follows.

Alignment type=8 indicates HEVC slice segments. Alignment type=9 indicates an HEVC slice segment or access unit. Alignment type=12 indicates HEVC slice segments or tiles.

Therefore, transmitting apparatus 100 can indicate that the data of the PES packet is either slice segment or pre-slice segment data by using type 9, for example. Since types indicating slices instead of slice segments are separately defined, transmitting apparatus 100 may use types indicating slices instead of slice segments.

Also, the DTS and PTS included in the header of the PES packet are set only in the PES packet including the leading data of the access unit. Therefore, if the type is 9 and the PES packet has a DTS or PTS field, the receiving device 200 assumes that the PES packet stores the entire access unit or the head division unit of the access unit. I can judge.

In addition, transmitting apparatus 100 uses a field such as transport_priority indicating the priority of a TS packet that stores a PES packet that includes the leading data of an access unit, so that receiving apparatus 200 can distinguish data included in the packet. good. Further, receiving device 200 may determine data included in the packet by analyzing whether the payload of the PES packet is an access unit delimiter. Also, the data_alignment_indicator in the PES packet header indicates whether data is stored in the PES packet according to these types. If this flag (data_alignment_indicator) is set to 1, the data stored in the PES packet is guaranteed to conform to the type indicated in the data alignment descriptor.

Also, transmitting apparatus 100 may use the data alignment descriptor only when performing PES packetization in units such as slice segments that can be divided and decoded. As a result, when the data alignment descriptor exists, receiving apparatus 200 can determine that the encoded data is PES-packetized in units that can be divided and decoded. It can be determined that the encrypted data is PES-packetized in access unit units. Note that the MPEG-2 TS standard defines that the unit of PES packetization is an access unit when the data_alignment_indicator is set to 1 and the data alignment descriptor does not exist.

If a data alignment descriptor is included in the PMT, receiving device 200 determines that PES packetization is performed in units that can be divided and decoded, and based on the packetized unit, input to each decoding unit data can be generated. In addition, if the receiving apparatus 200 determines that parallel decoding of encoded data is necessary based on program information or other descriptor information because the data alignment descriptor is not included in the PMT, , by analyzing the slice header of the slice segment, etc., to generate input data to each decoding unit. Also, when the encoded data can be decoded by a single decoding unit, receiving apparatus 200 decodes the data of the entire access unit by the decoding unit. Note that if the information indicating whether the encoded data is composed of a divisionally decodable unit such as a slice segment or a tile is separately indicated by a PMT descriptor or the like, receiving apparatus 200 uses the descriptor It may be determined whether the encoded data can be decoded in parallel based on the analysis result.

Also, since the DTS and PTS contained in the PES packet header are set only in the PES packet containing the top data of the access unit, when the access unit is divided into PES packets, the second and subsequent PES The packet does not contain information indicating the DTS and PTS of the access unit. Therefore, when decoding processing is performed in parallel, each of the decoding units 204A to 204D and the display unit 205 uses the DTS and PTS stored in the header of the PES packet containing the head data of the access unit.

Although the transmission apparatus, reception apparatus, transmission method, and reception method according to the embodiments have been described above, the present invention is not limited to these embodiments.

Further, each processing unit included in the transmitting device and the receiving device according to the above embodiments is typically implemented as an LSI, which is an integrated circuit. These may be made into one chip individually, or may be made into one chip so as to include part or all of them.

Further, circuit integration is not limited to LSIs, and may be realized by dedicated circuits or general-purpose processors. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure connections and settings of circuit cells inside the LSI may be used.

In each of the above embodiments, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may be realized by reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory by a program execution unit such as a CPU or processor.

In other words, the transmitter and receiver comprise processing circuitry and storage electrically coupled to the processing circuitry (accessible by the control circuitry). The processing circuitry includes at least one of dedicated hardware and program execution. Also, if the processing circuit includes a program execution unit, the storage device stores a software program executed by the program execution unit. The processing circuit uses the storage device to execute the transmission method or the reception method according to the above embodiments.

Furthermore, the present invention may be the above software program or a non-transitory computer-readable recording medium on which the above program is recorded. It goes without saying that the above program can be distributed via a transmission medium such as the Internet.

In addition, the numbers used above are all examples for specifically describing the present invention, and the present invention is not limited to the illustrated numbers.

Also, the division of functional blocks in the block diagram is an example, and a plurality of functional blocks can be realized as one functional block, one functional block can be divided into a plurality of functional blocks, and some functions can be moved to other functional blocks. may Moreover, single hardware or software may process the functions of a plurality of functional blocks having similar functions in parallel or in a time-sharing manner.

Also, the order in which the steps included in the above transmission method or reception method are executed is for illustrative purposes in order to specifically describe the present invention, and orders other than the above may be used. Also, some of the above steps may be executed concurrently (in parallel) with other steps.

The transmission device, reception device, transmission method, and reception method according to one or more aspects of the present invention have been described above based on the embodiments, but the present invention is not limited to these embodiments. do not have. As long as it does not depart from the spirit of the present invention, one or more modifications of the present embodiment that can be conceived by a person skilled in the art, or a form constructed by combining the components of different embodiments may be included within the scope of the embodiments.

INDUSTRIAL APPLICABILITY The present invention can be applied to devices or devices that perform media transport such as video data and audio data.

100 transmitter 101 encoder 102 multiplexer 103 modulator 104 transmitter 200 receiver 201 tuner 202 demodulator 203 demultiplexer 204A, 204B, 204C, 204D decoder 205 display 206 decoding commander 211 type determiner 212 control information acquisition unit 213 slice information acquisition unit 214 decoded data generation unit

Claims (4)

a segmentation step of segmenting the picture into a plurality of regions;
an encoding step of generating encoded data corresponding to each of the plurality of regions by encoding each of the plurality of regions so that they can be decoded independently;
a packetization step of storing the generated plurality of encoded data in a plurality of packets;
and a transmitting step of transmitting the plurality of packets;
each of the plurality of encoded data is associated one-to-one with a basic data unit that is a unit of data stored in one or more packets;
each of the plurality of encoded data is stored in the one or more packets;
The header information of each packet includes: (1) the basic data unit includes only the packet; (2) the basic data unit includes a plurality of packets, and the packet is at the beginning of the basic data unit (3) the basic data unit includes a plurality of packets, and the packet is a packet other than the first and last packets of the basic data unit; and (4) the basic data unit includes a plurality of packets including identification information indicating whether a packet is included and whether the packet is the last packet of the basic data unit;
In the packetizing step, control information used for decoding units in the picture is stored in one packet different from the plurality of packets in which the plurality of encoded data are stored, and the header information of the packet is the including identification information indicating that only the packet is included in the basic data unit,
the header information of the packet includes offset information indicating a bit length from the beginning of the encoded data of the picture to the beginning of the encoded data included in the packet;
Send method. A receiving method in a receiving device comprising a plurality of decoding units,
Multiple regions obtained by dividing a picture are encoded so that they can be decoded independently. a receiving step for receiving packets;
a decoding step in which the plurality of decoding units decode the plurality of encoded data in parallel;
each of the plurality of encoded data is associated one-to-one with a basic data unit that is a unit of data stored in one or more packets;
each of the plurality of encoded data is stored in the one or more packets;
The header information of each packet includes: (1) the basic data unit includes only the packet; (2) the basic data unit includes a plurality of packets, and the packet is at the beginning of the basic data unit (3) the basic data unit includes a plurality of packets, and the packet is a packet other than the first and last packets of the basic data unit; and (4) the basic data unit includes a plurality of packets including identification information indicating whether a packet is included and whether the packet is the last packet of the basic data unit;
The control information used for the decoding unit in the picture is stored in one packet different from the plurality of packets storing the plurality of encoded data, and the header information of the packet is stored in the basic data unit. including identification information indicating that only the packet is included,
the header information of the packet includes offset information indicating a bit length from the beginning of the encoded data of the picture to the beginning of the encoded data included in the packet;
receiving method. a dividing unit that divides a picture into a plurality of regions;
an encoding unit that generates encoded data corresponding to each of the plurality of regions by encoding each of the plurality of regions so that they can be decoded independently;
a packetizing unit that stores the generated plurality of encoded data in a plurality of packets;
A transmitting unit that transmits the plurality of packets,
each of the plurality of encoded data is associated one-to-one with a basic data unit that is a unit of data stored in one or more packets;
each of the plurality of encoded data is stored in the one or more packets;
The header information of each packet includes: (1) the basic data unit includes only the packet; (2) the basic data unit includes a plurality of packets, and the packet is at the beginning of the basic data unit (3) the basic data unit includes a plurality of packets, and the packet is a packet other than the first and last packets of the basic data unit; and (4) the basic data unit includes a plurality of packets including identification information indicating whether a packet is included and whether the packet is the last packet of the basic data unit;
The packetization unit stores control information used for decoding units in the picture in one packet different from the plurality of packets storing the plurality of encoded data, and the header information of the packet is the including identification information indicating that only the packet is included in the basic data unit,
the header information of the packet includes offset information indicating a bit length from the beginning of the encoded data of the picture to the beginning of the encoded data included in the packet;
transmitter. Multiple regions obtained by dividing a picture are encoded so that they can be decoded independently. a receiver for receiving packets;
A plurality of decoding units that decode the plurality of encoded data in parallel,
each of the plurality of encoded data is associated one-to-one with a basic data unit that is a unit of data stored in one or more packets;
each of the plurality of encoded data is stored in the one or more packets;
The header information of each packet includes: (1) the basic data unit includes only the packet; (2) the basic data unit includes a plurality of packets, and the packet is at the beginning of the basic data unit (3) the basic data unit includes a plurality of packets, and the packet is a packet other than the first and last packets of the basic data unit; and (4) the basic data unit includes a plurality of packets including identification information indicating whether a packet is included and whether the packet is the last packet of the basic data unit;
The control information used for the decoding unit in the picture is stored in one packet different from the plurality of packets storing the plurality of encoded data, and the header information of the packet is stored in the basic data unit. including identification information indicating that only the packet is included,
the header information of the packet includes offset information indicating a bit length from the beginning of the encoded data of the picture to the beginning of the encoded data included in the packet;
receiving device.

Images