JP5908055B2 - Implementation method of virtual reference decoder for scalable video coding - Google Patents

Description

The present invention relates to a virtual reference decoder for scalable video coding in a compressed video system.
This application claims the benefit of US Provisional Application No. 60/878729, filed Jan. 5, 2007, which is hereby incorporated by reference in its entirety.

  Virtual reference decoders are important in compressed video systems because they serve to enable the encoded bitstream to comply with certain standards. In an encoding standard such as H.264 / AVC, there are various interoperability points due to the features of scalable video encoding of the standard. The H.264 / AVC standard has rules (also called requirements, constraints, or behavioral specifications) that define HRD. HRD behavior is normative. AVC bitstreams must comply with HRD built according to the rules. SVC (Scalable Video Coding) is an extension to the AVC standard (Annex G). SVC bitstreams typically have a number of interoperability points due to the fact that the bitstream is at least partially scalable. Such a bitstream may be scalable in space, time and SNR, for example. A sub bitstream corresponding to the scalable aspect may be extracted from the bitstream. The previous HRD does not have sufficient rules to validate the bitstream, such as the SVC bitstream in AVC.

  This disclosure describes at least one implementation that provides a hypothetical reference decoder (HRD) for SVC. One such implementation proposes to modify the H.264 / AVC HRD for use with SVCs. The implementation defines HRD constraints for each interoperability point of SVC. In particular, one implementation is described, but other implementations are possible and devised by this disclosure. The first part of this disclosure describes each change in space, time and SNR scalability. The second part of this disclosure describes changes to the associated HRD parameters followed by the specification text.

  The H.264 / AVC standard has rules (also called requirements, constraints or behavior specifications) that define HRD. HRD behavior is normative. AVC bitstreams must comply with HRD built according to the rules. SVC (Scalable Video Coding) is an extension to the AVC standard (Annex G). This disclosure describes one or more implementations that provide HRD rules for SVCs. At least one implementation proposes the SVC-HRD rule as a change to the AVC-HRD rule. The user may build the SVC-HRD using the proposed SVC-HRD rules and test the bitstream for SVC compliance.

It is a table | surface which shows the sequence parameter set of an AVC series. It is a table | surface which shows the buffering period SEI message of AVC. It is a table | surface which shows the proposed HRD parameter. Fig. 6 is a table showing proposed VUI parameters. It is a table | surface which shows the proposed buffer_period SEI message. 6 is a table showing a proposed picture timing SEI message.

  An SVC bitstream typically has a number of interoperability points (also called operating points), at least in part due to the fact that the bitstream is scalable. Such a bitstream is scalable in space, time and SNR, for example. A sub bitstream corresponding to the scalable aspect is extracted from the bitstream. In one implementation, each interoperability point is checked by the HRD to ensure SVC compliance. The HRD may define HRD constraints for each checkpoint individually. Also, some checkpoints may obey one HRD constraint. In at least one implementation described in this disclosure, individual HRD constraints are proposed for each checkpoint. The use of individual HRD constraints may facilitate operation and may provide some similarities to the H.263 + approach.

  In this part, one specific implementation context should change from HRD in H.264 / AVC and what should be changed to meet the HRD in SVC from each of spatial, SNR and temporal scalability. Be analyzed. This provides a description from one conceptual point of view of many different possible types of scalability, and other conceptual points are possible.

1. Spatial SVC
1.1 HRD in VUI messages:
The VUI message is included in the SPS (see vui_parameters () in Table 1). For each spatial SVC layer, the picture size is different from the other layers, so the corresponding SPS is different from that of the other layers. Therefore, the VUI HRD in AVC can be applied directly to the VUI HRD in the spatial SVC without modification. For each checkpoint, as shown in Table 1, the correct VUI message can be obtained through the AVC HRD rule.

1.2 Buffering period SEI messages:
seq_parameter_set_id is in the buffering period SEI message. By indexing the seq_parameter_set_id in the SPS corresponding to each spatial SVC layer, the buffering period of each spatial SVC layer can be obtained. Therefore, for spatial SVC, the HRD buffering period SEI message in AVC can be applied directly for spatial SVC. For each checkpoint, the correct buffering period SEI message can be obtained without changing the current HRD, as shown in Table 2.

1.3 Picture Timing SEI message:
cpb_removal_delay and dpb_output_delay are the same for different layers in the same access unit (see Table 6). Therefore, no change is required for spatial scalability.

2. SNR Scalability 2.1 VUI messages:
For SNR SVC, the quality layer can be indicated by dependency_id or quality_level. Different quality layers / levels can share the same SPS, so the VUI message in AVC is modified to include HRD information for all quality layers / levels (see Table 3).

2.2 Buffering period SEI messages:
For SNR SVCs, different quality layers / levels can share the same SPS and therefore do not have a one-to-one mapping with seq_parameter_set_id present in the buffering period SEI message. The buffering period SEI message is modified to include HRD information for all quality layers / levels (Table 5).

2.3 Picture Timing SEI message:
cpb_removal_delay and dpb_output_delay are the same for different quality layers / levels in the same access unit. Therefore, no change is required for spatial scalability.

3. Time SVC
3.1 VUI message:
For temporal SVC, different temporal layers can share the same SPS, so the VUI message in AVC is modified to include HRD information for all temporal layers (see Table 3).

3.2 Buffering SEI messages:
For temporal SVC, different temporal layers can share the same SPS and therefore do not have a one-to-one mapping with seq_parameter_set_id present in the buffering period SEI message. The buffering period SEI message is modified to include HRD information for all time layers (see Table 5).

3.3 Picture Timing SEI message:
For temporal SVC, the frame rate is different for each temporal layer. This means that a single NAL unit with a given temporal_level may work for several frame rates, since a lower temporal layer can serve as a layer subordinate to an upper temporal layer Means. The picture timing SEI message is modified to include HRD information for all temporal layers (see Table 6).

  In the 3.4 VUI message, when timing_info_present_flag is true, consider changing num_units_in_tick, time_scale, and fixed_frame_rate_flag to reflect correct frame rate information (see Table 4).

  All three conceptual levels of (space, time and SNR) scalability are combined in the following changes to the AVC-HRD rules. Tables 3 to 6 are adopted from the AVC standard and relate to AVC-HRD. In addition to the AVC standard, tables are shown using italics. Other implementations may have deletions, but there is no deletion from the AVC standard table. A bold term is a syntax actually transmitted in the bit stream. As can be seen, each of Tables 3 through 6 shows that the AVC standard has been changed by introducing an “if-then” loop that tests the variable “profile_idc”. If “profile_idc” is equal to “SVC”, the if loop is executed one or more times to test one or more points. If “profile_idc” is not equal to “SVC”, “AVC” is assumed to be the relevant standard and “else” is used to test one point for AVC compliance (using existing AVC-HRD rules). "The loop is executed. In Table 3, the variables “dependency_id [i]”, “temporal_level [i]”, and “quality level [i]” provide various scalable options. Since these variables have a combined 8-bit length, there are up to 2 ** 8 checkpoints for the SVC bitstream. An implementation can loop from 0 to 255 using 8 bits. This is compared to one checkpoint for the AVC bitstream.

  1. In the VUI message, the HRD parameters are indicated for each subordinate layer, time layer and quality layer sharing the same SPS as shown in Table 3. When timing_infor_present_flag is true, num_units_in_tick, time_scale and fixed_frame_rate_flag are indicated for each time layer as shown in Table 4.

  2. In the buffering period SEI message, parameters related to the HRD are indicated for each subordinate layer, time layer and quality layer sharing the same sequence_parameter_set_id, as shown in Table 5.

  3. In the picture timing SEI message, parameters related to the HRD are indicated for each temporal layer as shown in Table 6.

  The operations of Tables 3-6 are outlined. Table 3 defines the bit rate and cpb (coded picture buffer) size for each checkpoint / layer. Table 4 defines the frame rate for each temporal layer. Table 5 defines the initial cpb delay and the initial dpb (decoded picture buffer) delay for each checkpoint / layer. Table 6 defines the cpb extraction delay and dpb output delay for each checkpoint / layer. For each checkpoint / layer, the previous parameters are used in the HRD rules, the same is done for AVC, and the bitstream is tested for compliance.

  A value obtained by adding 1 to num_layer_minus1 indicates the number of scalable layers or presentation points supported by the bitstream referring to the same seq_parameter_set_id in the SPS including this hrd_parameters ().

  dependency_id [i] indicates the dependency (CGS) layer of scalable layer i. This is equal to the dependency_id of the NAL unit in scalable layer i.

  temporal_level [i] indicates a temporal layer of scalable layer i. This is equal to the temporal_level of the NAL unit in scalable layer i.

  quality_level [i] indicates the quality layer of scalable layer i. This is equal to the quality_level of the NAL unit in scalable layer i.

cpb_cnt_minus1 [i], bit_rate_scale [i], cpb_size_scale [i], bit_rate_value_minus [i] [SchedSelldx], cpb_size_value_minus [i] [SchedSelldx], cbr_flag_i1 [SchedSelldx], initial_cpb_removed [i], time_offset_length [i] are respectively equal to their corresponding values of scalable layer i * ¹ . (* 1): In the definition, the bit rate includes bits of scalable layer i and its subordinate layers.
num_temporal_layer_minus1 plus 1 indicates the number of temporal layers supported by the bitstream. This is equal to the maximum temporal_level of NAL units in the bitstream.

  timing_info_present_flag [i], num_units_in_tick [i], time_scale [i], fixed_frame_rate_flag [i] are respectively equal to their corresponding values for the time layer i.

  num_layer_minus1 plus 1 indicates the number of scalable layers or presentation points supported by the bitstream that references the same seq_parameter_set_id in the buffer_period SEI message.

  dependency_id [i] indicates the dependency (CGS) layer of scalable layer i. This is equal to the dependency_id of the NAL unit in scalable layer i.

  temporal_level [i] indicates a temporal layer of scalable layer i. This is equal to the temporal_level of the NAL unit in the scalable layer i.

  quality_level [i] indicates the quality layer of scalable layer i. This is equal to the quality_level of the NAL unit in scalable layer i.

  initial_cpb_removal_delay [i] [SchedSelldx], initial_cpb_removal_delay_offset [i] [SchedSelldx] are respectively equal to their corresponding values for scalable layer i.

  num_temporal_layer_minus1 plus 1 indicates the number of temporal layers whose access unit depends on the NAL unit associated with this picture timing SEI message.

  temporal_layer [i] indicates the temporal level of temporal layer i.

  cpb_removal_delay [i] and dpb_output_delay [i] are respectively equal to their corresponding values for temporal layer i.

Various implementations are contemplated by this disclosure, and these implementations may include one or more features described in this disclosure. Such an implementation may be in the form of a program consisting of a method, apparatus or instructions, for example using hardware, software or a combination thereof. Some possible implementations are set forth in the following claims.
In addition to the above embodiment, the following supplementary notes are disclosed.
(Appendix 1)
A method for realizing a virtual reference decoder (HRD) for extending scalable video coding in a video compression environment.
(Appendix 2)
The HRD tests each interoperability point of the bitstream for compliance with scalable video coding;
The method according to appendix 1.
(Appendix 3)
The various interoperability points of the bitstream are indicated by one or more variables.
The method according to appendix 2.
(Appendix 4)
There are multiple points of interoperability in the bitstream, the multiple points of interoperability exhibit scalability with respect to each other;
The method according to appendix 2.
(Appendix 5)
The indicated scalability includes one or more of spatial scalability, temporal scalability, and SNR scalability.
The method according to appendix 4.
(Appendix 6)
The test includes examining a variable that indicates whether the bitstream is to be tested to comply with Advanced Video Coding (AVC) or Scalable Video Coding (SVC).
The method according to appendix 2.
(Appendix 7)
The variable is called "profile_idc"
The method according to appendix 6.
(Appendix 8)
When the variable indicates AVC, a rule based on AVC and HRD is realized,
When the variable indicates SVC, a rule based on SVC and HRD is realized.
The method according to appendix 6.
(Appendix 9)
If the variable indicates SVC, the implemented SVC and HRD rules include a loop that is executed multiple times;
Each execution of the loop tests one interoperability point,
The method according to appendix 8.
(Appendix 10)
Defined by a set of rules forming the HRD;
The method according to appendix 1.
(Appendix 11)
The set of rules is based on rules from AVC and HRD,
The method according to appendix 10.

Claims (11)

A virtual reference decoder implementation method for scalable video coding,
i as integer, determining the value of a variable included in the bit stream shown subordinate layer of scalable layer i, the temporal layer and quality layer,
And determining the virtual reference decoder parameters corresponding to the scalable layer i,
Virtual reference decoder parameters the determined includes at least one bit rate parameter, the bit rate parameter is determined based on the bit of the scalable layer i and its dependent layers,
The determined virtual reference decoder parameters further include a first coded picture buffer (CPB) extraction delay , a first CPB extraction delay offset, a CPB extraction delay, and a decoded picture buffer (DPB) output delay. Including
Determining the virtual reference decoder parameter comprises:
By using explicitly indicated parameters from the picture timing SEI message in the bitstream for each temporal layer, the CPB extraction delay and the DPB output delay are determined, and the picture timing SEI message Further indicate the number of temporal layers in the bitstream,
The extraction delay of the CPB and the output delay of the DPB are determined to be the same for scalable layers corresponding to different spatial layers or quality layers in each temporal layer.
Including the method. There are multiple points of interoperability in the bitstream, the multiple points of interoperability exhibit scalability with respect to each other;
  The method of claim 1. The indicated scalability includes one or more of spatial scalability, temporal scalability, and SNR scalability.
  The method of claim 2. Defined by a set of rules forming the virtual reference decoder;
  The method of claim 1. The extraction delay of the CPB and the output delay of the DPB remain the same for scalability layers corresponding to different spatial layers and quality layers in the respective temporal layers, and the values of the extraction delay of the CPB and the output delay of the DPB are: Different for each temporal layer,
  The method of claim 1. A scalable video encoding method comprising:
  determining a value of a variable indicating a subordinate layer, a temporal layer, and a quality layer of the scalable layer i, where i is an integer;
  Determining a virtual reference decoder parameter corresponding to the scalable layer i, wherein the determined virtual reference decoder parameter includes at least one bit rate parameter, the bit rate parameter including the scalable layer i and its The determined virtual reference decoder parameters are determined based on the bits of the dependent layer, and include the first coded picture buffer (CPB) extraction delay, the first CPB extraction delay offset, the CPB extraction delay, and the decoding Further comprising an output delay of a picture buffer (DPB);
  Generating a bitstream including the value of the determined variable and the determined virtual reference decoder parameter;
  Including
  The extraction delay of the CPB and the output delay of the DPB are explicitly indicated only in the picture timing SEI message in the bitstream for each temporal layer, and the picture timing SEI message indicates the number of temporal layers in the bitstream. Further,
  The CPB pull-out delay and the DPB output delay are determined to be the same for scalable layers corresponding to different spatial layers or quality layers in each temporal layer. There are multiple points of interoperability in the bitstream, the multiple points of interoperability exhibit scalability with respect to each other;
  The method of claim 6. The indicated scalability includes one or more of spatial scalability, temporal scalability, and SNR scalability.
  The method of claim 7. The extraction delay of the CPB and the output delay of the DPB remain the same for scalability layers corresponding to different spatial layers and quality layers in the respective temporal layers, and the values of the extraction delay of the CPB and the output delay of the DPB are: Different for each temporal layer,
  The method of claim 6. When executed by one or more processors, the one or more processors
  determining a value of a variable included in a bitstream indicating a subordinate layer, a temporal layer, and a quality layer of the scalable layer i, where i is an integer;
  Determining a virtual reference decoder parameter corresponding to the scalable layer i;
  Let it run
  The determined virtual reference decoder parameter includes at least one bit rate parameter, the bit rate parameter is determined based on the bits of the scalable layer i and its subordinate layers,
  The determined virtual reference decoder parameters further include a first coded picture buffer (CPB) extraction delay, a first CPB extraction delay offset, a CPB extraction delay, and a decoded picture buffer (DPB) output delay. Including
  Determining the virtual reference decoder parameter comprises:
  By using explicitly indicated parameters from the picture timing SEI message in the bitstream for each temporal layer, the CPB extraction delay and the DPB output delay are determined, and the picture timing SEI message Further indicate the number of temporal layers in the bitstream,
  The extraction delay of the CPB and the output delay of the DPB are determined to be the same for scalable layers corresponding to different spatial layers or quality layers in each temporal layer.
  Including the program. When executed by one or more processors, the one or more processors
  determining a value of a variable indicating a subordinate layer, a temporal layer, and a quality layer of the scalable layer i, where i is an integer;
  Determining a virtual reference decoder parameter corresponding to the scalable layer i, wherein the determined virtual reference decoder parameter includes at least one bit rate parameter, the bit rate parameter including the scalable layer i and its The determined virtual reference decoder parameters are determined based on the bits of the dependent layer, and include the first coded picture buffer (CPB) extraction delay, the first CPB extraction delay offset, the CPB extraction delay, and the decoding Further comprising an output delay of a picture buffer (DPB);
  Generating a bitstream including the value of the determined variable and the determined virtual reference decoder parameter;
  And execute
  The extraction delay of the CPB and the output delay of the DPB are explicitly indicated only in the picture timing SEI message in the bitstream for each temporal layer, and the picture timing SEI message indicates the number of temporal layers in the bitstream. Further,
  The CPB withdrawal delay and the DPB output delay are determined to be the same for scalable layers corresponding to different spatial layers or quality layers in each temporal layer.

Images