JP4928569B2 - Segment encoding method, segment encoding apparatus, segment encoding program and recording medium thereof, parallel encoding system, partial re-encoding system - Google Patents

Description

  The present invention encodes a segment obtained by cutting out a part of a sequence in a state where the buffer positions of the in point and the out point are determined, and finally combines the preceding and subsequent encoded streams to generate a single stream. Is related to the technology.

  FIG. 8 is a diagram illustrating a configuration example of a conventional encoding device that does not support segment encoding, and FIG. 9 is a process flowchart of the encoding device.

  A conventional general encoding method that does not support segment encoding will be described with reference to FIGS. In encoding an original image sequence, first, the encoding apparatus 200 stores an original image to be encoded in the original image storage unit 201 (step S201). Also, based on the target rate T, the initial buffer position Bstart, and the buffer size BS, the code amount control unit 202 determines an initial value of the quantization width (step S202). Thereafter, the processing of steps S204 to S207 is repeated for all frames (step S203).

  First, one frame is encoded by the encoding unit 203 based on the quantization width obtained by the code amount control unit 202, and the encoded information is sent to the time information adding unit 204 (step S204). The decoding time information is calculated by the code amount control unit 202 from the generated code amount of the encoding result and its history, the target rate T, the initial buffer position Bstart, and the buffer size BS, and transmitted to the time information adding unit 204 (step) S205). On the other hand, the code amount control unit 202 calculates the quantization width of the next frame from the generated code amount and its history, the target rate T, the initial buffer position Bstart, and the buffer size BS, and sends them to the encoding unit 203 (step S206). . The time information adding unit 204 adds the decoding time information from the code amount control unit 202 to the encoded information from the encoding unit 203 to generate a stream (step S207). The above operation is repeated for all frames (step S208).

  FIG. 10 shows a configuration example of the encoding unit 203 shown in FIG. The encoding unit 203 inputs an image signal of an original image to be encoded and a quantization width at the time of encoding. First, the subtractor 211 calculates the difference between the original image and the predicted image and generates a predicted residual image. The orthogonal transform unit 212 performs orthogonal transform such as DCT transform on the prediction residual image and outputs an orthogonal transform coefficient. The quantization unit 213 quantizes the orthogonal transform coefficient with the designated quantization width. The variable length encoding unit 214 converts the quantized orthogonal transform coefficient into a variable length code and outputs it as encoded information.

  On the other hand, the inverse quantization unit 215 inversely quantizes the quantized orthogonal transform coefficient. The inverse orthogonal transform unit 216 performs inverse orthogonal transform on the orthogonal transform coefficient after inverse quantization, and generates a prediction residual decoded image. The adder 217 generates a decoded image by adding the predicted residual decoded image and the predicted image, and stores the decoded image in the decoded image storage memory 218. The predicted image generation unit 219 uses the decoded image stored in the decoded image storage memory 218 as a reference image to generate a predicted image for obtaining the next predicted residual image using motion search or the like.

  The above encoding unit 203 is a general one, but other encoding methods can also be used.

  Next, segment encoding (segment encoding) targeted by the present invention will be described. FIG. 11 shows an example when segment encoding is required.

  Segment encoding is a technique for extracting and encoding a part of a sequence. For example, as shown in FIG. 11A, when an image sequence is divided into a plurality of parts (referred to as segments) in the time direction and encoded in parallel by a plurality of segment encoding devices 1001 to 1003, As shown in FIG. 11B, it is used when only a part of a stream that has already been encoded by editing work is re-encoded. The finally segment-encoded stream is combined with the preceding and succeeding streams to form one stream.

  The biggest problem when combining such segment-encoded streams with the preceding and succeeding streams is buffer mismatch. When encoding with CBR (constant bit rate), it is necessary to control the generated code amount while monitoring the buffer transition so that the decoder buffer does not fail. In the case of segment encoding, each stream can be encoded without using a buffer failure by using an existing encoder having the CBR mode. However, since the buffer positions of the preceding and succeeding streams are not checked, one stream is used. If it is connected, buffer corruption may occur.

  FIG. 12 shows an example in which a buffer failure occurs when streams are combined, and FIG. 13 shows an example in which a buffer failure does not occur.

  For example, as shown in FIG. 12A, when the end buffer position of a certain segment n is lower than the start buffer position of the next segment n + 1, these are combined, and as shown in FIG. 12B, the segment n + 1 May cause buffer underflow and buffer failure.

  On the contrary, as shown in FIG. 13A, when the end buffer position of a certain segment n is higher than the start buffer position of the next segment n + 1, when these are combined, the difference is expressed as invalid data (filler). By filling it, as shown in FIG. 13B, the streams can be combined without causing buffer failure.

  Therefore, in segment encoding, the buffer position at the end of the segment must be specified in addition to the buffer position at the start, and the buffer position at the end of segment encoding (end buffer position) must be greater than or equal to the specified value. is there.

  As a conventional technique for solving such a buffer failure problem, there is a “bitstream seamless connection encoding method and apparatus” described in Patent Document 1. Here, at the time of encoding each frame, the difference between the current buffer position and the estimated buffer position is monitored, and the allocated code amount is sequentially controlled so that the generated code amount does not exceed.

  FIG. 14 is a diagram showing a configuration example of a conventional segment encoding apparatus, and FIG. 15 is a process flowchart of the conventional segment encoding apparatus.

  When the segment encoding apparatus 1000 encodes a segment obtained by cutting out a part of the sequence of the original image, the segment encoding apparatus 1000 stores the original image to be encoded in the original image storage unit 1100 (step S301). Further, based on the target rate T, the initial buffer position Bstart, and the buffer size BS, the code amount control unit with termination processing 1101 determines the initial quantization width (step S302). Thereafter, the processing in steps S304 to S307 is repeated for all frames (step S303).

  First, based on the quantization width obtained by the code amount control unit 1101 with termination processing, the encoding unit 1102 encodes one frame while considering the termination buffer position Bend (step S304). The decoding time information is calculated by the code amount control unit with termination processing 1101 from the generated code amount of the encoding result and its history, the target rate T, the initial buffer position Bstart, and the buffer size BS, and transmitted to the time information adding unit 1103 (Step S305). On the other hand, the quantization width of the next frame is determined from the generated code amount and its history, the target rate T, the initial buffer position Bstart, and the buffer size BS so that the end is equal to or larger than the end buffer position Bend. The amount control unit 1101 calculates and sends the result to the encoding unit 1102 (step S306). The time information adding unit 1103 adds the decoding time information from the code amount control unit with termination process 1101 to the encoded information from the encoding unit 1102 to generate a stream (step S307). The above operation is repeated for all frames (step S308). Finally, a termination buffer position adjustment process for inserting invalid data is performed by the termination adjustment unit 1104 so that the buffer position becomes the termination buffer position Bend (step S309).

Japanese Patent No. 3963478

  When a segment obtained by cutting out a part of a sequence is encoded and finally combined with preceding and subsequent encoded streams to generate a single stream, the conventional segment encoding technique ensures that the amount of generated code is within the limit. Since strict monitoring is required, the error in the amount of generated code is reduced, but the image quality is likely to fluctuate.

  In addition, in the code amount control, when determining the quantization width of the next frame, it must always be calculated in consideration of the end buffer position Bend. In addition to lacking, there is a problem that costs such as image quality adjustment are required.

In order to solve the above-described problems, the present invention performs the following processing when encoding a segment.
1. Based on the original set rate, end buffer position, and segment length, a temporary target rate (temporary target code amount) is calculated.
2. The virtual buffer size is calculated from the original buffer size and the end buffer position.
3. The following process is repeated for all frames.

  3-1. Encoding is performed using the CBR (constant rate) mode of the existing encoder using the temporary target rate and the virtual buffer size.

  3-2. Based on the generated code amount, the decoding time is calculated from the original set rate and buffer size.

  As described above, it is possible to avoid a buffer failure when combining a plurality of streams without greatly changing the existing encoder. The reason is as follows.

  A sufficient condition that a buffer failure does not occur when a plurality of encoded streams are combined is that the buffer position at the end of encoding of a stream is higher than the start buffer position of the next stream. In the present invention, encoding is performed at a slightly lower set rate than the target rate using the CBR (constant rate) mode of an existing general encoder. For this stream, when the buffer transition is drawn with respect to the target rate, the generated code amount is slightly smaller than the target code amount, so that the buffer position gradually rises to the right. Therefore, by adding a mechanism to obtain the “slightly lower target rate” value for the existing encoder in advance, the buffer position at the end of encoding can be changed to the next stream without any special modification inside the encoder. You can build an encoding system that takes you higher than the starting buffer position.

  The decoding time is generally calculated using the buffer transition information obtained by the encoding control unit. However, in the present invention, the target rate given to the encoding control unit is lower than the original target rate. Therefore, the time information is shifted. Therefore, a mechanism for obtaining decoding time information at the original target rate is provided separately from the encoding control unit.

  As described above, in the present invention, since no particular effort is made inside the existing encoder, the construction of the system is easy and the versatility is high.

  Further, in the present invention, there is almost no need to worry about image quality fluctuations at the joints of segments when a plurality of streams are combined. When encoding while considering raising the buffer position at the end of the stream, the restriction on the amount of generated code becomes stricter in the latter half, but in the present invention, since encoding is performed at a constant rate with an existing encoder, the segment There is no need to think about measures for image quality fluctuations at the joints. For the “slightly lower target rate”, for example, when the initial buffer position is half the set rate and the segment length is 60 seconds, the target rate may be set to 0.5 / 60≈0.83% lower than the target rate. , The longer the segment length, the less influence on the image quality.

  According to the present invention, it is possible to perform encoding so that a buffer failure does not occur when a plurality of streams are combined. In addition, an existing encoder that does not support segment encoding can be used for segment encoding with almost no modification. Therefore, high versatility can be secured, and development and adjustment costs can be greatly reduced. In addition, there is an effect that there is almost no need to worry about image quality fluctuations at the segment joints.

It is a figure which shows the structural example of the parallel encoding system to which this invention is applied. It is a process flowchart of a parallel encoding system. It is a figure which shows the structural example of the partial re-encoding system which is another example of application of this invention. It is a process flowchart of a partial re-encoding system. It is a block diagram of the segment coding apparatus which concerns on embodiment of this invention. It is a process flowchart of a segment encoding apparatus. It is a figure explaining the example of the buffer transition in this Embodiment. It is a figure which shows the structural example of the conventional encoding apparatus which does not respond | correspond to segment encoding. It is a process flowchart of the encoding apparatus. It is a figure which shows the structural example of a general encoding part. It is a figure which shows the example when segment encoding is needed. It is a figure which shows the example which raise | generates a buffer failure, when a stream is combined. It is a figure which shows the example in which a buffer failure does not occur when a stream is combined. It is a figure which shows the structural example of the conventional segment encoding apparatus. It is a process flowchart of the conventional segment encoding apparatus.

  Embodiments of the present invention will be described below with reference to the drawings. Prior to describing details of the embodiment of the segment coding apparatus, an example of an application system of the present invention will be described.

  FIG. 1 is a diagram showing a configuration example of a parallel encoding system to which the present invention is applied.

  A parallel encoding system 100 shown in FIG. 1 is a system that automatically divides an input stream of an original image into a plurality of segments and encodes the segments in parallel. Therefore, the parallel encoding system 100 includes an original image storage unit 101 that stores an original image, a segment division unit 102 that divides an original image sequence into segments, and a plurality of segment encoding devices 103 that encode each segment in parallel. 104, 105,... Includes a stream combining unit 106 for combining encoded segment streams.

  The parallel encoding system 100 receives an initial buffer position Bstart, a target rate T, and a buffer size BS. As the initial buffer position of the first segment, the value of the initial buffer position Bstart given to the system is used. Further, the end buffer position Bend given to the last segment may be set to 0, for example.

  That is, an initial buffer position Bstart given to the system is input as an initial buffer position to the segment encoding device 103 that encodes the head segment. For example, the initial buffer position Bstart2 determined by the segment dividing unit 102 is input to the next segment encoding device 104, and the initial buffer position Bstart3 is input to the next segment encoding device 105. This initial buffer position Bstart2 is input as the end buffer position Bend to the first segment encoder 103, and the initial buffer position Bstart3 is input as the end buffer position Bend to the second segment encoder 104. . The third segment encoding device 105 receives the initial buffer position Bstart4 of the next segment as the end buffer position Bend.

  These initial buffer positions Bstart2, Bstart3, Bstart4,... May be obtained from the video feature quantity by the segment dividing unit 102, or the same fixed value may be used for all segments.

  FIG. 2 is a process flowchart of the parallel encoding system 100. The parallel encoding system 100 first stores the original image of the encoding target video in the original image storage unit 101 (step S101). Next, the segment division unit 102 divides the video stored in the original image storage unit 101 into n segments (step S102). If the number of segment encoders is m, n = m or n> m may be satisfied. In the case of n> m, the m + 1th and subsequent segments are similarly encoded in order from the head segment encoding device 103.

  Next, the segment division unit 102 determines the initial buffer positions Bstart2, Bstart3,... For each segment from the feature amount of each segment video (step S103). Note that the initial buffer position may be a fixed value determined in advance for all segments.

  Each of the segment encoders 103, 104, 105,... Encodes each divided segment according to a segment encoding method according to an embodiment of the present invention, which will be described in detail later (step S104). Finally, the stream combining unit 106 combines the encoded streams of the segments output from the segment encoding devices 103, 104, 105,... To generate one stream (step S105).

  FIG. 3 is a diagram showing a configuration example of a partial re-encoding system as another application example of the present invention.

  The partial re-encoding system 110 illustrated in FIG. 3 is a system that re-encodes only a section from a specified cut-out start time to a cut-out end time with respect to an encoded stream encoded based on an original image. The partial re-encoding system 110 includes an original image storage unit 111 that stores an original image, and a stream analysis that obtains buffer positions of a cutout start time and a cutout end time from an encoded stream as an initial buffer position Bstart and a terminal buffer position Bend, respectively. Segment segment 112, segment segment unit 113 that segments the segment image from segment start time to segment end time among segment images, and segment code that encodes segment segment video And a stream combining unit 115 that inserts a stream that is an encoding result of the segment encoding device 114 between a stream before the cutout start time and a stream after the cutout end time, and combines the streams. Is provided.

  FIG. 4 is a process flowchart of the partial re-encoding system 110. First, the partial re-encoding system 110 stores the original image of the encoding target video in the original image storage unit 111 (step S111). On the other hand, the stream analysis unit 112 analyzes the encoded stream that has already been encoded, and sets the buffer position at the extraction start time and the buffer position at the extraction end time from the encoded stream as the initial buffer position Bstart and the end buffer position Bend, respectively. (Step S112).

  The segment cutout unit 113 extracts the original image from the specified cutout start time to the cutout end time among the original images stored in the original image storage unit 111 as a segment video (step S113). The segment encoding device 114 encodes the extracted segment video according to the segment encoding method according to the embodiment of the present invention described in detail later (step S114). Finally, the stream combining unit 115 combines the stream that is the encoding result of the segment encoding device 114 and the stream before and after the cutout to generate one encoded stream (step S115).

  The present invention is applied to the segment encoding apparatus in the parallel encoding system 100 and the partial re-encoding system 110 as described above. Hereinafter, the embodiment will be described in detail.

  FIG. 5 is a configuration diagram of the segment coding apparatus according to the embodiment of the present invention. The segment encoding apparatus 1 includes an original image storage unit 10 that stores an original image to be encoded, a segment length detection unit 11 that detects a segment length to be encoded, and a temporary rate calculation unit 12 that calculates a temporary target rate. A virtual buffer size calculation unit 13 that calculates a virtual buffer size, a buffer transition calculation unit 14 that calculates a buffer transition and calculates a decoding time, a code amount control unit 15 that controls a code amount at the time of encoding, , An encoding unit 16 that encodes the original image, a time information adding unit 17 that adds decoding time information to the encoded information, and a termination adjusting unit 18 that adjusts the end of the stream.

  In the above configuration, the segment length detection unit 11, the provisional rate calculation unit 12, the virtual buffer size calculation unit 13, and the buffer transition calculation unit 14 are particularly unique to the present invention.

  The code amount control unit 15 may be the same as the conventional code amount control unit that does not support segment encoding. However, a terminal for outputting decoding time information is not necessary. Therefore, the code amount control unit 15 does not necessarily need to perform calculations related to the decoding time information.

The temporary rate calculation unit 12 calculates the temporary target rate Tt [bit / sec] from the target rate T [bit / sec], the end buffer position Bend [bits], and the segment length Ni [sec]. Here, the provisional target rate Tt is, for example, Tt = T−Bend / Ni
This guarantees that the buffer position at the end is higher than the end buffer position Bend. In anticipation of safety, this value can be multiplied by a safety factor or a smaller value with a margin subtracted.

The virtual buffer size calculation unit 13 calculates the virtual buffer size BSh from the buffer size BS at the time of sequence combination and the end buffer position Bend. Here, the virtual buffer size BSh is set to, for example, BSh = BS−Bend.
By doing so, it is guaranteed that the buffer size at the time of sequence combination is equal to or less than BS. In anticipation of safety, this value can be multiplied by a safety factor or a smaller value with a margin subtracted.

  The buffer transition calculation unit 14 may be the same as the “code amount control unit” that does not support segment encoding. However, a terminal for outputting the quantization width is not necessary. Therefore, it is not always necessary to perform processing related to the calculation of the quantization width.

  FIG. 6 is a process flowchart of the segment encoding apparatus 1 shown in FIG. Hereinafter, the process performed by the segment encoding apparatus 1 will be described with reference to FIG. First, an original image to be encoded is input and stored in the original image storage unit 10 (step S10). The segment length detection unit 11 detects the segment length Ni to be encoded (step S11).

  Next, the temporary target rate Tt is calculated by the temporary rate calculation unit 12 according to a predetermined calculation formula (step S12), and the virtual buffer size BSh is calculated by the virtual buffer size calculation unit 13 (step S13). The code amount control unit 15 determines the initial value of the quantization width based on the temporary target rate Tt, the initial buffer position Bstart, and the virtual buffer size BSh (step S14).

  Thereafter, the following steps S15 to S19 are repeated for all frames (step S15). The contents of the iteration process are as follows. First, one frame is encoded by the encoding unit 16 based on the obtained quantization width (step S16). The buffer transition calculation unit 14 calculates decoding time information from the generated code amount and its history in encoding, the target rate T, the initial buffer position Bstart, and the buffer size BS, and transmits the decoding time information to the time information addition unit 17 (step S1). S17).

  On the other hand, the code amount control unit 15 calculates the quantization width of the next frame from the generated code amount and its history, the temporary target rate Tt, the initial buffer position Bstart, and the virtual buffer size BSh, and sends it to the encoding unit 16. (Step S18). In the code amount control unit 15, the process of calculating the quantization width using the temporary target rate Tt, the virtual buffer size BSh, and the like is performed using the target rate T, the buffer size BS, and the like in the conventional code amount control. Since it is the same as the process of calculating the quantization width, further detailed explanation is omitted here. The time information adding unit 17 adds the decoding time information from the buffer transition calculating unit 14 to the encoded information from the encoding unit 16 to generate a stream (step S19).

  The above operation is repeated for all frames (step S20). Finally, the termination adjustment unit 18 performs termination buffer position adjustment processing for inserting invalid data so that the termination buffer position becomes Bend (step S21).

  FIG. 7 is a diagram for explaining an example of buffer transition in the present embodiment. The encoding condition of the original encoded stream is an encoding set A. That is, it is assumed that the encoding set A has a buffer size BS [bits], a target rate T [bits / sec], an initial buffer position Bstart [bits], and a segment length Ni [sec].

  In the present embodiment, as shown in FIG. 7A, the encoding of the segment is performed by setting the encoding set B for the encoding set A. As described above, when the segment encoding apparatus 1 encodes a segment with the setting of the encoding set B, encoding is performed with the virtual buffer size BSh that is obtained by subtracting the end buffer position Bend from the buffer size BS. The existence range of the graph when the graph of the buffer transition is drawn with the setting of the conversion set B is G1 shown in FIG.

  Similarly, when encoding is performed with the setting of encoding set B and a graph of buffer transition is drawn with the setting of encoding set A, the existence range of the graph becomes G2 shown in FIG. 7C. As is clear from this, in the present embodiment, the position of the buffer end is always above the end buffer position Bend. Therefore, by setting the initial buffer position of the next segment below the end buffer position Bend, it is possible to realize stream combination that does not cause buffer failure.

  The segment encoding process described above can be realized by a computer and a software program, and the program can be recorded on a computer-readable recording medium or provided through a network.

DESCRIPTION OF SYMBOLS 1 Segment encoding apparatus 10 Original image preservation | save part 11 Segment length detection part 12 Temporary rate calculation part 13 Virtual buffer size calculation part 14 Buffer transition calculation part 15 Code amount control part 16 Encoding part 17 Time information addition part 18 Termination adjustment part

Claims (8)

A segment encoding method for encoding a segment so that a buffer failure does not occur when an encoded stream of a segment obtained by cutting out a part of a sequence of an image is combined with a stream before and after the segment,
Calculating a provisional target rate less than the target rate from a predetermined target rate, a given end buffer position and a segment length;
Calculating a virtual buffer size smaller than the buffer size from a predetermined buffer size and the end buffer position;
Encoding each frame of the segment in CBR mode using the provisional target rate and the virtual buffer size;
A process of calculating decoding time information from the target rate and the buffer size based on the generated code amount of the encoding result;
A step of adding the decoding time information to the encoding result to generate an encoded stream of the segment. In the segment encoding method of Claim 1,
The segment encoding method, wherein the temporary target rate is set to a value equal to or less than a value obtained by subtracting a value obtained by dividing the end buffer position by a segment length from the predetermined target rate. In the segment encoding method according to claim 1 or 2,
The segment encoding method, wherein the virtual buffer size is set to a value equal to or smaller than a value obtained by subtracting the end buffer position from the predetermined buffer size. A segment encoding apparatus for encoding a segment so that a buffer failure does not occur when an encoded stream of a segment obtained by extracting a part of an image sequence is combined with a stream before and after the segment,
Means for calculating a provisional target rate less than the target rate from a predetermined target rate, a given end buffer position and a segment length;
Means for calculating a virtual buffer size smaller than the buffer size from a predetermined buffer size and the end buffer position;
Means for encoding each frame of the segment in CBR mode using the temporary target rate and the virtual buffer size;
Means for calculating decoding time information from the target rate and the buffer size based on the generated code amount of the encoding result;
Means for adding the decoding time information to the encoding result and generating an encoded stream of the segment.   A segment encoding program for causing a computer to execute the segment encoding method according to claim 1, 2 or 3.   A computer-readable recording medium on which a segment encoding program for causing a computer to execute the segment encoding method according to claim 1, 2 or 3 is recorded. A parallel encoding system that divides a sequence of images into multiple segments and encodes each segment in parallel,
A segment division unit for dividing a sequence of images to be encoded into a plurality of segments;
A plurality of segment encoding devices according to claim 4, each encoding each segment divided by said segment dividing unit;
A parallel encoding system, comprising: a stream combination unit that combines the stream of segments encoded by the plurality of segment encoding apparatuses. A partial re-encoding system for re-encoding a portion of an encoded stream,
A segment cutout unit for cutting out a segment of a video part to be re-encoded from the sequence of original images of the encoded stream;
A stream analysis unit that analyzes the encoded stream and obtains an initial buffer position and an end buffer position of a part of the encoded stream that is a target of re-encoding;
The segment encoding device according to claim 4, wherein the segment cut out by the segment cutout unit is encoded based on the initial buffer position and the end buffer position obtained by the stream analysis unit;
A partial re-encoding comprising: a stream combining unit that combines a stream of segments re-encoded by the segment encoding device and a stream of a part excluding the re-encoded part of the encoded stream. system.

Images