JP4127182B2 - Video time-axis hierarchical encoding method, encoding apparatus, decoding method, decoding apparatus, and computer program - Google Patents

Description

  The present invention relates to a moving picture time-axis hierarchical encoding method, encoding apparatus, decoding method and decoding apparatus, and a computer program, and in particular, encodes a moving picture thinned out in the time direction as a basic hierarchy, and the remaining moving pictures TECHNICAL FIELD The present invention relates to a video time-axis hierarchical encoding method and encoding apparatus for encoding video as an extension layer, a decoding method and decoding apparatus for decoding a hierarchically encoded video, and a computer program for hierarchical encoding and decoding .

  For the coding of moving images, there is hierarchical coding (Scalable Coding) having a double code string configuration, compared to simple coding of only one layer. Hierarchical coding can be decoded with a code sequence of only the base layer, and by decoding the code sequence of the extension layer and combining it with the decoding result of the base layer, it is possible to obtain a higher quality reproduced image Is.

  The hierarchization (division) direction includes a quantization direction (SNR), a spatial direction (Spatial), and a time axis direction (Temporal). In the case of the time axis hierarchy, for example, a 60 fps (field per second) interlaced scanned image is thinned out in field units to obtain and encode a 30 fps image, and the remaining fields that have not been encoded are encoded fields. Prediction is performed from the locally decoded image, and the prediction residual is encoded.

  Next, a conventional moving picture time axis hierarchical encoding apparatus will be described. FIG. 12 is a block diagram showing an example of a conventional moving picture time axis hierarchical encoding apparatus. In the figure, a moving image signal coming from the image input terminal 1 is given to the switch 51. The moving image signal is an interlaced scanning image of 60 fields per second (60 fps). The image setting device 7 determines the odd field and the even field of the interlaced video signal coming to the input terminal 1 and controls the switch 51. The switch 51 distributes the incoming video signal for each field, and supplies the even field to the encoder 52 and the odd field to the image delay unit 11.

  The encoder 52 encodes an image signal of an even field of 30 fps, and obtains a code string and a quantization result. The code string is supplied to the multiplexer 5. As a specific encoding method, intra-field encoding may be used in addition to inter-picture prediction encoding represented by MPEG.

  The local decoder 54 receives, as an input, a signal that has been subjected to at least quantization from the encoder 52, performs local decoding processing, and obtains a reproduced image. This reproduced image is supplied to the inter-picture predictor 53. The inter-picture predictor 53 forms a prediction signal for the odd field and supplies it to the subtractor 13.

  On the other hand, the image delay unit 12 delays the encoded field until the reference image of the encoded field is processed and arranged from the encoder 52 to the local decoder 54 and inter-picture prediction processing becomes possible. This is supplied to the subtracter 13. The subtracter 13 performs subtraction between the odd-field delayed image signal from the image delay unit 12 and the prediction signal given from the inter-picture predictor 53, thereby obtaining a prediction residual and supplying it to the DCT 14.

  The DCT 14 performs a DCT (Discrete Cosine Transform) conversion process on the prediction residual and gives the obtained coefficient to the quantizer 15. The quantizer 15 quantizes the input coefficient with a predetermined step width, and gives the coefficient that has become a fixed-length code to the variable-length encoder 16. The variable-length encoder 16 compresses the input fixed-length code (prediction residual) with the variable-length code, and supplies the resulting code string to the multiplexer 5, where the encoder 52 Is multiplexed with the code string from and output.

  Next, the configuration of the conventional moving picture time axis hierarchical code string 6 output from the multiplexer 5 in FIG. 12 will be described. The interlaced scanned image is divided into an even field and an odd field, one being a base layer code string and the other being an extension layer code string. FIG. 14A schematically shows a video time-axis hierarchical code string 6 including the basic hierarchical code string and the extended hierarchical code string. In FIG. 4A, “filed” is one field of an interlaced scanned image. The numbers are the image order of the encoded moving images. Since the enhancement layer is encoded by bi-directional prediction, it is necessary to transmit the base layer image that is later in time, and the time relationship is transmitted in reverse. In addition, when bi-directional prediction is performed in the base layer, the reverse is also performed in the base layer.

  Next, a conventional moving picture time axis hierarchical decoding apparatus will be described. FIG. 13 is a block diagram showing an example of a conventional moving picture time axis hierarchical decoding apparatus corresponding to the moving picture time axis hierarchical encoding apparatus of FIG. In FIG. 13, a moving picture time-axis hierarchical code string coming from a code input terminal 24 is separated into a base layer code string and an extended layer code string by a demultiplexer 25, and the base layer code string is decoded. The code string of the enhancement layer is supplied to the variable length decoder 30.

  The base layer code string is reverse-processed by the decoder 61 of the encoder 52 of FIG. 12 by the decoder 61 to obtain a reproduced image signal of 30 fps, and is supplied to the inter-picture predictor 62 and the image delay unit 23. This reproduced image signal is an even field of the interlaced scanned image signal. The inter-picture predictor 62 forms a prediction signal for the odd field from the reproduced image signal and supplies it to the adder 33.

  On the other hand, the code string of the enhancement layer is input to the inverse quantizer 31 after the variable length code of the prediction residual is returned to the fixed length code by the variable length decoder 30 and is inversely quantized according to the quantization parameter. Is done. The fixed-length code becomes a reproduction DCT coefficient value of the prediction residual by the inverse quantizer 31 and is input to the inverse DCT 32. The inverse DCT 32 converts a total of 64 coefficients, 8 in the vertical direction and 8 in the horizontal direction, into a decoded prediction residual signal and supplies it to the adder 33.

  The adder 33 adds the prediction signal given from the inter-picture predictor 62 to the decoded prediction residual signal from the inverse DCT 32 to obtain a decoded image signal. The 30 fps decoded image signal obtained in this way is an odd field of the interlaced scanned image signal and is supplied to the switch 27.

The switch 27 synchronizes the output reproduction image signal of the image delay unit 23 and the output decoded image signal of the adder 33 with the even / odd number of the field in synchronization with the hierarchy information of the code string output from the demultiplexer 25. Are selected and output from the reproduced image output terminal 28.
The output moving image signal is a 60 fps interlaced scanned image.

  As described above, in the conventional moving image temporal axis hierarchical coding, an interlaced scanned image is divided into an even field and an odd field, one being a base layer code string and the other being an extension layer. Here, in higher-layer encoding, since a field image contains many aliasing distortion components, a prediction error increases in motion compensated inter-picture prediction.

  On the other hand, in the inter-picture prediction in the lower hierarchy, since the field parity (even / odd) is different between the prediction reference picture and the encoded picture, appropriate inter-picture prediction is difficult. As a result, there is a problem in that the encoding efficiency is significantly reduced compared to the conventional encoding that is not hierarchized.

  The present invention has been made paying attention to the above points. An incoming interlaced scanning image signal is converted into a sequential scanning signal of the same frame rate and encoded as a basic layer, and a field at a different time from the sequential scanning signal. Video temporal axis hierarchical encoding method, encoding apparatus, and decoding that can obtain high encoding efficiency even for interlaced scanned image signals by performing inter-picture predictive encoding using a decoded signal of a progressive scanning signal as a reference image An object of the present invention is to provide an encoding method, a decoding device, and a computer program.

  In order to achieve the above object, a moving image time-axis hierarchical coding method according to a first aspect of the present invention is a first method for converting an input interlace scan image signal into a progressive scan image signal having the same frame rate as the input interlace scan image signal. A step, a second step of encoding a sequentially scanned image signal to obtain a first code string, and a local decoded image obtained by local decoding of the first code string as a reference image in an input interlaced scanned image signal A third step in which a field at a time position different from that of the sequentially scanned image signal is subjected to inter-picture prediction encoding to obtain a second code string, and the first code string and the second code string are multiplexed to obtain a moving picture time And a fourth step of outputting as an axis hierarchy encoded signal.

  In the present invention, the input interlaced scanning image signal is converted into a sequential scanning image signal of the same frame rate and encoded as a base layer to obtain a first code string, and a field at a time position different from that of the sequential scanning image signal is obtained. The second code string is obtained by performing inter-picture predictive coding using the locally decoded image obtained by local decoding of the first code string as a reference image, and the first code string and the second code string are multiplexed. And output as a video time-axis hierarchical encoded signal. Here, since the inter-picture prediction when obtaining the second code sequence is a prediction from a front-rear sequential scanning image with a very short time distance, extremely high coding efficiency is obtained, and the generated code amount is There are few.

  In order to achieve the above object, the moving picture temporal axis hierarchical decoding method according to the second aspect of the present invention converts a temporally hierarchized code string into a progressively scanned image signal having the same frame rate as the reproduced interlaced scanned image signal. A first step of separating a first code string encoded with respect to a second code string encoded for a field at a time position different from that of a sequentially scanned image signal in a reproduced interlaced scanned image signal; A second step of decoding the first code string to obtain a reproduction sequential scan image signal, and a first field that is one of an odd field or an even field of the reproduction interlaced scan image signal. The third step of converting the image signal into the second image sequence and the second code string are subjected to inter-picture predictive decoding using the reproduction sequential scanning image signal as a reference image, and the reproduction interlace A fourth step of obtaining a second field image signal different from the first field of the first scan image signal, and the first field image signal and the second field image signal are temporally switched and reproduced. And a fifth step of outputting as an interlaced scanning image signal.

  In the present invention, an image of a first field that is one of an odd field or an even field of a reproduced interlaced scanned image signal from a reproduced sequential scanned image signal obtained by decoding the first code string that is the base layer. Since the signal of the second field obtained by decoding the second code string that is the enhancement layer is obtained and the playback interlaced scanned image signal is obtained by temporally switching these signals, A reproduced interlaced scanned image signal with sufficient temporal resolution can be obtained as compared with decoding of only the base layer.

  In order to achieve the above object, the moving picture time-axis hierarchical coding apparatus according to the third aspect of the invention converts an input interlace scan image signal into a sequential scan image signal having the same frame rate as the input interlace scan image signal. A conversion means; a first code string generation means for obtaining a first code string by encoding a progressively scanned image signal; and a local decoded image obtained by local decoding of the first code string as an input interlace. A second code string generating means for obtaining a second code string by performing inter-picture predictive coding on a field at a time position different from that of the sequentially scanned image signal in the scanned image signal; a first code string and a second code string; And a multiplexing means for multiplexing and outputting as a moving picture time axis hierarchical encoded signal. In the present invention, the same moving picture time axis hierarchical coding signal can be obtained by the same action as the moving picture time axis hierarchical coding method of the first invention.

  In order to achieve the above object, the moving picture time-axis hierarchical coding apparatus of the fourth invention down-samples the sequentially scanned image signal obtained by the image conversion means of the third invention in the vertical direction, The scanning line down-sampling means for obtaining a progressive scanning image signal with a reduced number of scanning lines is further provided, and the first code string generating means encodes the sequential scanning image signal with a reduced number of scanning lines to obtain the number of scanning lines. The reduced first code string is generated, and the second code string generating means up-samples the local image signal obtained by locally decoding the first code string having the reduced number of scanning lines in the vertical direction. The scanning line down-sampling means includes scanning line up-sampling means using a signal having the same number of scanning lines as the sequential scanning image signal inputted to the scanning line down-sampling means as a reference image.

  In the present invention, since the progressive scan image signal is down-sampled in the vertical direction to encode the progressive scan image signal obtained by reducing the number of scanning lines, the first code string is generated. The generated code amount can be reduced as compared with the case where the first code string is generated from the progressively scanned image signal that is not down-sampled.

  In order to achieve the above object, a moving picture time-axis hierarchical decoding apparatus according to a fifth aspect of the present invention converts a time-series code sequence into a sequentially scanned image signal having the same frame rate as a reproduced interlaced scanned image signal. Multiplexing / separating means for separating the first code string encoded with respect to the second code string encoded with respect to the field at a time position different from the sequentially scanned image signal in the reproduced interlaced scanned image signal And a first decoding means for decoding the first code string to obtain a reproduction sequential scanning image signal, and a reproduction sequential scanning image signal that is one of an odd field or an even field of the reproduction interlaced scanning image signal. The image conversion means for converting the image signal of the field of the image and the second code string are subjected to inter-picture predictive decoding using the reproduction sequential scanning image signal as a reference image, and the reproduction interlace Second decoding means for obtaining an image signal of a second field different from the first field of the scanned image signal, and temporally switching between the image signal of the first field and the image signal of the second field And a multiplexing means for outputting as a reproduced interlaced scanning image signal. In the present invention, the same reproduced interlaced scanned image signal can be obtained by the same operation as the moving picture time axis hierarchical decoding method of the second invention.

  In order to achieve the above object, a moving picture time-axis hierarchical decoding apparatus according to a sixth aspect of the present invention uses a demultiplexing means according to the fifth aspect of the present invention as a reproduced interlaced scanned image from a time-axis hierarchical code string. A third code sequence encoded for a progressively scanned image signal downsampled in the vertical direction at the same frame rate as the signal, and a field at a time position different from the progressively scanned image signal in the reproduced interlaced scanned image signal Means for separating the encoded second code string, and the first decoding means for up-sampling the sequentially scanned image signal obtained by decoding the third code string in the vertical direction. The image converting means is a means for converting the sequentially scanned image signal upsampled in the vertical direction into an image signal of the first field.

  In the present invention, the decoding is performed on the third code string encoded on the sequentially scanned image signal down-sampled in the vertical direction at the same frame rate as the reproduced interlaced scanned image signal. The amount of decoding processing can be reduced as compared with the case of decoding sequentially scanned image signals that are not down-sampled in the vertical direction.

  In order to achieve the above object, a computer program according to a seventh aspect is characterized by causing a computer to execute each step of the moving picture time axis hierarchical encoding method according to the first aspect. In the present invention, the same moving picture time axis hierarchical coding signal can be obtained by the same action as the moving picture time axis hierarchical coding method of the first invention.

  Furthermore, in order to achieve the above object, a computer program according to an eighth invention is characterized by causing a computer to execute each step of the moving picture time axis hierarchical decoding method according to the second invention. In the present invention, the same reproduced interlaced scanned image signal can be obtained by the same operation as the moving picture time axis hierarchical decoding method of the second invention.

  The present invention has the following excellent features.

  (1) The interlaced scanning image signal is converted into a sequential scanning signal having the same frame rate and encoded, so that encoding is performed as a sequential scanning signal with the same number of scanning lines as the interlaced scanning image signal. The bit rate can be greatly reduced compared to encoding the scanned image signal as it is.

  (2) Inter-picture prediction when obtaining the second code string is based on prediction from a sequentially scanned image having a very short time distance, so that extremely high coding efficiency can be obtained, and the amount of generated code is small. Therefore, time-axis hierarchical coding can be performed with high coding efficiency on the interlaced scanned image signal. This is not only superior to the conventional time-axis hierarchical coding, but also can have higher coding efficiency than ordinary interlaced video coding without hierarchization.

  (3) The amount of coding processing according to the present invention increases as compared with conventional time-axis hierarchical coding, but is smaller than when all interlaced scanning image signals are converted into sequentially scanned image signals and coded. Can be processed.

  (4) By encoding the sequentially scanned image signal obtained by down-sampling in the vertical direction and reducing the number of scanning lines to generate the first code string, from the sequentially scanned image signal not down-sampled in the vertical direction The generated code amount can be reduced as compared with the case of generating the first code string. The progressive scanning image signal is converted from the interlaced scanning image signal, and the original interlaced scanning image signal suppresses the vertical frequency band component that reduces the number of scanning lines. Is slight.

  (5) Since the decoding is performed on the third code string encoded on the sequentially scanned image signal down-sampled in the vertical direction at the same frame rate as the reproduced interlaced scanned image signal, The amount of decoding processing can be reduced as compared with the case of decoding sequentially scanned image signals that are not down-sampled in the direction.

  Next, the best mode for carrying out the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a first embodiment of a moving picture time axis hierarchical encoding apparatus according to the present invention. In the figure, the same components as those in the conventional example of FIG. In the first embodiment shown in FIG. 1, a sequential scanning converter 2, a switch 10, and a field thinning device 11 are added as compared with the conventional example of FIG. 12, and an encoder 4 and a local decoder 9 are added. The processing operation of the inter-picture predictor 8 is different.

  In FIG. 1, a moving image signal of interlace scanning that comes from an image input terminal 1 is sequentially input to a scan converter 2, an image selector 7, and a switch 10. As the encoding process of the base layer, the sequential scanning converter 2 interpolates the scanning line thinned out by the interlaced scanning moving image signal from the image input terminal 1 from the scanning line adjacent in spatio-temporal, and scans it. A progressive (progressive) scanning signal of 60 frames per second in which the number of lines is doubled is obtained and supplied to the switch 3. The switch 3 thins out every other frame and obtains a sequential scanning signal of 30 frames per second.

  Note that the sequential scanning signal of 30 fps may be obtained directly from the input interlaced scanning image signal by simultaneously performing the operation of the sequential scanning converter 2 and the operation of the switch 3. However, in this case, an intermediate image between two odd and even fields is not obtained, but scanning line interpolation is performed with the existing scanning lines remaining, which corresponds to the same time as any field of the incoming interlaced scanning image signal. A progressive scan frame is formed.

  The encoder 4 encodes the 30 fps progressive scanning signal extracted from the switch 3, supplies the resulting code string to the multiplexer 5, and supplies the signal that has been quantized to the local decoder 9. . The encoding method of the encoder 4 is MPEG-2, MPEG-4, other inter-picture prediction encoding, intra-frame encoding, or the like.

  The encoding process in the local decoder 9 is performed for all frames of the 30 fps progressive scanning signal. In MPEG encoding, it is not always necessary to locally decode a bi-predictive frame (B-picture). However, in this embodiment, it is necessary to become a prediction reference image of an enhancement layer image, so that a reproduced image of 30 fps is obtained. . The reproduced image is supplied to the inter-image predictor 8 as a reference image. The inter-picture predictor 8 forms a prediction signal for each field of interlaced scanning sandwiched between frames of progressive scanning at 30 fps. The formed progressive scanning prediction signal is thinned out by the field thinning unit 11 to form a field for interlace scanning. The inter-image predictor 8 and the field decimation unit 11 may be integrated to obtain a predicted image of interlaced scanning directly from a reference image of sequential scanning.

  On the other hand, as an enhancement layer encoding process, the switch 10 selects a field having a time different from the frame selected by the switch 3 from the incoming interlaced moving image signal and supplies the selected field to the image delay unit 12. The image delay unit 12 delays the encoded field until the reference image of the encoded field is processed and arranged from the progressive scan conversion 2 to the local decoder 9, and inter-picture prediction processing becomes possible.

  The subtraction unit 13 subtracts the prediction signal supplied from the field decimation unit 11 from the image delay unit 12 and supplies it to the DCT 14 as a prediction residual. The DCT 14 performs DCT (Discrete Cosine Transform) conversion processing on the prediction residual from the subtractor 13 and supplies the obtained coefficient to the quantizer 15. The quantizer 15 quantizes the input coefficient with a predetermined step width, and supplies the coefficient that has become a fixed-length code to the variable-length encoder 16. The variable length encoder 16 compresses the fixed-length prediction residual with the variable length code and supplies the resulting code to the multiplexer 5. The multiplexer 5 inserts an index code in a timely manner into the code strings of the respective layers from the encoder 4 and the variable length encoder 16 and multiplexes them.

  As described above, according to the present embodiment, the moving image signal of the interlace scanning from the image input terminal 1 is converted into the sequential scanning signal of the same frame rate and encoded, thereby encoding the sequential scanning signal. Since the number of scanning lines has not increased, the bit rate can be greatly reduced as compared with the case where the moving image signal of the input interlace scanning is encoded as it is.

  Next, the scanning line relationship of the encoding process of this embodiment will be described with reference to FIG. FIG. 3A shows a scanning line of a moving image signal of 60 fps interlaced scanning coming from the image input terminal 1, but the scanning line positions are shifted between the even field and the odd field. On the other hand, the scanning line of the 30 fps progressive scanning signal encoded as the base layer has either an even field or an odd field of the moving image signal of the incoming interlace scanning as shown in FIG. All scan lines exist at the same time. In the case of such a scanning line structure, since there is no aliasing distortion and motion compensation can be performed appropriately, the prediction residual is very small.

  On the other hand, the image signal of the extension layer is a signal of one field whose time is different from the sequential scanning signal shown in FIG. 3B in the even field or the odd field of the moving image signal of the incoming interlace scanning. The line is shown in FIG. The number of encoded scanning lines of this signal is half that of the signals shown in FIGS. 3A and 3B, and the reference image is sequentially moved back and forth at 1/60 second intervals indicated by broken lines in the figure. Since it is a scanning signal, very good inter-picture prediction is possible, the prediction residual is very small, and the amount of generated code is small.

  Therefore, according to the present embodiment, the interlaced scanning video signal is superior to the conventional time-axis hierarchical coding and has a higher coding efficiency than the normal interlaced video coding without hierarchization. Axis hierarchical encoding is possible.

  Next, a second embodiment of the moving picture time axis hierarchical encoding apparatus according to the present invention will be described. FIG. 4 shows a block diagram of a second embodiment of the moving picture time axis hierarchical encoding apparatus according to the present invention. In the figure, the same components as those in FIG. The encoding device of the second embodiment shown in FIG. 4 is different from the encoding device of the first embodiment shown in FIG. 1 in that a scanning line down sampler 35 and a scanning line up sampler 38 are added. Yes. The processing operations of the encoder 36 and the local decoder 37 are different.

  Next, the operation of the second embodiment will be described. In FIG. 4, a moving image signal of 60 fps interlaced scanning coming from the image input terminal 1 is sequentially supplied to the scan converter 2, the image setting unit 7 and the switch 10. The operations of the progressive scan converter 2 and the switch 3 are the same as those in the first embodiment shown in FIG. 1, and a progressive scan signal of 30 fps is obtained.

  The obtained sequential scanning signal is down-sampled by the scanning line down sampler 35 so that the number of scanning lines is about 3/4 to 2/3. Down-sampling is performed while limiting the band with a vertical low-pass filter (V-LPF) so that there is no aliasing distortion in the number of scan lines after conversion. The number of scan lines after conversion obtained by the scan line down sampler 35 is 360 to 320 when the number of effective frame lines of the moving image signal of the incoming interlace scan is 480, and 810 when the number is 1080. This is from 720 to 720. In the interlaced scanning image signal, the region with the highest frame vertical frequency is suppressed in order to suppress flicker. Therefore, the image signal subjected to progressive scanning conversion has few components, and little image information is lost even if it is down-sampled. .

  The image signal downsampled in the vertical direction by the scanning line downsampler 35 is encoded by the encoder 36. The encoding method by the encoder 36 is the same as that of the encoder 4 in FIG. 1, but since the number of scanning lines is smaller than in the case of FIG. 1, all images are converted into sequential scanning and encoded. The amount of processing and the amount of generated codes are smaller than in the case of the code.

  The encoded signal output from the encoder 36 is subjected to local decoding for all 30 fps frames in the local decoder 37. This local decoding process also requires less processing because the number of scanning lines of the input signal is reduced. The local decoded signal output from the local decoder 37 is supplied to the scanning line upsampler 38, where it is returned to the original number of scanning lines and input to the inter-picture predictor 8. The operations of the inter-image predictor 8 and the field decimation unit 11 are the same as those in FIG.

  On the other hand, the moving image signal of the other field which is the enhancement layer supplied to the switch 10 is the same as that in FIG. 1 by the image delayer 12, the subtractor 13, the DCT 14, the quantizer 15, and the variable length encoder 16. Are processed into an enhancement layer code string, and then multiplexed by the multiplexer 5 with the encoded signal from the encoder 36. However, since the reference image for inter-picture prediction is different from the case of FIG. 1, the encoding result changes slightly.

  The scanning line upsampler 38 and the field decimation unit 11 may be integrated, and a field image signal to be given to the subtractor 13 from the output of the local decoder 37 may be directly created. When performing, since an image having twice the density is required vertically, the previous processing configuration is not wasteful.

  Next, each embodiment of the moving picture time axis hierarchical decoding apparatus of the present invention will be described. FIG. 2 shows a block diagram of the first embodiment of the moving picture time-axis hierarchical decoding apparatus of the present invention. The moving picture time axis hierarchical decoding apparatus of this embodiment is a decoding apparatus corresponding to the moving picture time axis hierarchical encoding apparatus of FIG. 1, and the same components as those of the conventional example of FIG. It is. The first embodiment of the decoding apparatus shown in FIG. 2 has field decipherers 22 and 29 added to the conventional decoding apparatus of FIG. Processing operation is different.

  In FIG. 2, a code string coming from a code input terminal 24 is supplied to a demultiplexer 25, where it is separated into a base layer code string and an extended layer code string, and the base layer code string is decoded. The code sequence of the enhancement layer is supplied to the variable length decoder 30.

  The decoder 21 performs reverse processing of the encoder 4 of FIG. 1 on the base layer code string to obtain a reconstructed image of 30 fps progressive scanning, and the reconstructed image is subjected to inter-field prediction with the field decimation unit 22. To the vessel 26. The field thinning device 22 deletes the interpolated scanning line, obtains a field image, and supplies it to the image delay device 23. The image delay unit 23 holds the reproduction field signal for several fields in order to synchronize with the fields of the extension layer.

  On the other hand, the code string of the enhancement layer is input to the inverse quantizer 31 after the variable length code of the prediction residual is returned to the fixed length code by the variable length decoder 30 and is inversely quantized according to the quantization parameter. Is done. The fixed-length code becomes a reproduction DCT coefficient value of the prediction residual by the inverse quantizer 31 and is input to the inverse DCT 32. The inverse DCT 32 converts a total of 64 coefficients, 8 in the vertical direction and 8 in the horizontal direction, into a decoded prediction residual signal and supplies it to the adder 33.

  The prediction signal given from the inter-picture predictor 26 is thinned out by the field thinning-out unit 29 and then added to the decoded prediction residual signal from the inverse DCT 32 in the adder 33 to become a decoded image signal. The 30 fps decoded image signal obtained in this way is the other field of the interlaced scanned image signal and is supplied to the switch 27.

  The switch 27 selects the output reproduction image signal of the image delay unit 23 and the output decoded image signal of the adder 33 in accordance with the field parity in synchronization with the hierarchy information of the code string output from the demultiplexer 25. Then, it is output from the reproduced image output terminal 28. The output moving image signal is a 60 fps interlaced scanned image.

  Next, a second embodiment of the moving picture time axis hierarchical decoding apparatus according to the present invention will be described. FIG. 5 shows a block diagram of a second embodiment of the moving picture time axis hierarchical decoding apparatus according to the present invention. The moving picture time axis hierarchical decoding apparatus of this embodiment is a decoding apparatus corresponding to the moving picture time axis hierarchical encoding apparatus shown in FIG. 4, and the same components as in FIG. The description is omitted. Compared with the decoding apparatus of the first embodiment shown in FIG. 2, the decoding apparatus of the second embodiment has a scanning line upsampler 42 added thereto. The operation is different.

  Next, the operation of this embodiment will be described. First, the code string input from the code input terminal 24 in FIG. 5 is separated into a base layer code string and an extension layer code string by the demultiplexer 25, and the base layer code string is supplied to the decoder 41. Then, the enhancement layer code string is supplied to the variable length decoder 30.

  The code stream of the base layer is decoded by the decoder 41 to be a reconstructed image signal of progressive scanning with the number of scanning lines reduced by 30 fps, and then supplied to the scanning line upsampler 42. The processing operation of the decoder 41 is the same as that of the decoder 21 of FIG. 2, but the amount of decoding processing is reduced because the number of scanning lines is reduced. The reproduced image signal from the decoder 41 is returned to the original number of scanning lines by the scanning line upsampler 42 and then supplied to the field thinning device 22 and the inter-picture predictor 26.

  The processing operation of the scanning line upsampler 42 is the same as the processing operation of the scanning line upsampler 38 of the encoding apparatus shown in FIG. Further, the operations of the image predictor 26, the field decimation unit 29, the field decimation unit 22, the image delay unit 23, the switch 27, and the image output terminal 28 shown in FIG. 5 are the same as those in FIG.

  On the other hand, the code string of the enhancement layer is generated by the circuit unit including the variable length decoder 30, the inverse quantizer 31, the inverse DCT 32, the adder 33, the inter-picture predictor 26, and the field thinning unit 29 shown in FIG. Decrypted by the same operation. The decoded image signal of 30 fps obtained by this decoding is the other field of the interlaced scanned image signal, and is input from the adder 33 to the switch 27.

  Next, the moving picture time axis hierarchical code string configuration in the present invention will be described. This code string configuration includes a base layer code string obtained by encoding a 30 fps progressive scan signal and an extended layer code string obtained by encoding one field of an interlaced scan image signal. Each is shown in FIG. In FIG. 5B, “Prog.” Is one frame of the progressive scanning signal, and “filed” is one field of the interlaced scanning image signal. The number is the image order of the encoded video signal.

  As shown in FIG. 14B, since the extended layer image (interlaced scanned image signal of one field) is encoded by bidirectional prediction, the base layer image (one frame of progressive scanning signal) that is temporally later is encoded. ) Must be transmitted first, and the time relationship is reversed. In addition, when bi-directional prediction is performed in the base layer, the reverse is also performed in the base layer. Further, a base layer code string for one frame that is sequential scanning and an extension layer code string that is one field of interlace scanning alternately exist.

  Note that the upper layer code sequence and the enhancement layer code sequence do not necessarily have to be alternated for each image, and may be packetized separately and multiplexed timely. Further, in the transmission system, as schematically shown in FIG. 14C, an upper layer code string may be transmitted first, and an extension layer code string may be sent later.

  Next, a moving image time axis hierarchical recording medium will be described. In this moving image time-axis hierarchical recording medium, a basic hierarchical code string obtained by encoding a 30 fps progressive scanning signal and an extended hierarchical code string obtained by encoding one field of an interlaced scanning image signal are recorded. Each code string is a separate packet, and index information indicating a hierarchy is given. It is desired that the recording form can reproduce only the information of the basic layer.

  In the reproduction of only the information on the base layer, only one field is used in the conventional example, so that the spatio-temporal resolution is inferior. However, since this recording medium can obtain a sequentially scanned image, the spatial resolution is good.

  Note that the present invention includes a computer program that realizes one or both of the constituent elements of the above-described moving picture time-axis hierarchical coding apparatus and moving picture time-axis hierarchical decoding apparatus by a computer. This computer program may be read into a computer from a recording medium, or may be taken in via a network.

  Next, a moving picture time axis hierarchical encoding operation by the computer program of the present invention will be described with reference to the flowchart of FIG. First, the interlaced scanning image signal input to the computer is converted into a sequential scanning image signal having the same frame rate as the image signal (step S1). The processing in step S1 corresponds to, for example, the operation of the sequential scanning converter 2, the switch 3, and the image setting unit 7 in the encoding apparatus of FIG.

  Subsequently, the sequential scanning image signal converted in step S1 is encoded to generate a first code string (step S2). At this time, the progressive scanning image signal is locally decoded, and a locally decoded image signal is also obtained. The processing in step S2 corresponds to the operation of the encoder 4 and the local decoder 9 in the encoding apparatus of FIG.

  Subsequently, in the interlaced scanned image signal input to the computer, a field at a time position different from the sequentially scanned image signal is subjected to inter-picture prediction coding using the locally decoded image signal as a reference image signal to generate a second code string. (Step S3). The processing of step S3 is performed by, for example, the inter-picture predictor 8, the switch 10, the field decimation unit 11, the image delay unit 12, the subtractor 13, the DCT 14, the quantizer 15 and the variable length coder in the encoding apparatus of FIG. 16 corresponds to the operation according to FIG.

  Then, the first code string is multiplexed as a base layer code string and the second code string is multiplexed as an extended layer code string, and a moving picture time axis layer encoded signal is output (step S4). The processing in step S4 corresponds to the operation of the multiplexer 5 in the encoding device of FIG. When the first code string is obtained as in the encoding device shown in FIG. 4, the first code string for the sequentially scanned image down-sampled in the vertical direction at the same frame rate as the input interlaced scanned image is obtained. You may make it obtain.

  Next, the moving picture time axis hierarchical decoding operation by the computer program of the present invention will be described with reference to the flowchart of FIG. First, the computer receives, as an input signal, a video time-axis hierarchical encoded signal multiplexed with the first code sequence as a base layer code sequence and the second code sequence as an extended layer code sequence, The code string is separated into the second code string (step S11). The process of step S11 corresponds to the operation of the demultiplexer 25 in the decoding device of FIG.

  Subsequently, the first code string is decoded to obtain a reproduction sequentially scanned image signal (step S12). The processing in step S12 corresponds to, for example, the operation by the decoder 21 in the decoding device in FIG. Then, the reproduction sequential scanning image signal is converted into an image signal of the first field which is one of the odd field and the even field of the reproduction interlace scanning image signal (step S13). The processing in step S13 corresponds to the operation of the circuit unit including the field thinning device 22 and the image delay device 23 in the decoding device of FIG.

  Further, inter-picture predictive decoding is performed on the second code string using the reproduced sequential scanning image signal as a reference image signal, thereby obtaining a second field image signal different from the first field of the reproduced interlaced scanning image (step S14). For example, in the decoding apparatus of FIG. 2, the processing in step S <b> 14 is a circuit unit including a variable length decoder 30, an inverse quantizer 31, an inverse DCT 32, an adder 33, an inter-picture predictor 26, and a field decimation unit 29. It corresponds to the operation by. Then, the first field image signal and the second field image signal are temporally switched to be output as a reproduction interlaced scan image signal (step S15). The processing in step S15 corresponds to the operation of the switch 27 in the decoding device of FIG.

  When the first code string is decoded as in the decoding apparatus shown in FIG. 5, the decoded first code string is down-sampled in the vertical direction at the same frame rate as the reproduced interlaced scanned image. A reproduced sequential scan image may be obtained.

  Note that the present invention is not limited to the above-described embodiment, and for example, it is possible to configure a transmitting apparatus and a receiving apparatus for moving image time-axis hierarchical encoded signals. FIG. 8 shows a block diagram of an embodiment of a transmitting apparatus for moving picture time-axis hierarchical encoded signals. In the figure, a reception interface (I / F) 71 receives and demodulates the computer program via a network and temporarily stores it in a program buffer 72.

  On the other hand, the computer 73 receives an interlaced moving image signal as an input, and performs an encoding operation similar to the encoding device described with reference to FIG. 1 or FIG. 4 based on the computer program from the program buffer 72. The encoded data obtained in this way is supplied to the transmission I / F 74. The transmission I / F 74 transmits the encoded data by the operation according to the flowchart shown in FIG.

  That is, the transmission I / F 74 first communicates with a partner terminal of the communication network according to a predetermined protocol, and determines whether or not transmission permission has been received from the partner terminal (step S21). When the transmission permission is received, the transmission I / F 74 converts the encoded data into a predetermined transmission format (step S22), and then transmits it to the communication network (step S23). On the other hand, if the transmission permission is not received in step S21, the transmission I / F 74 stops transmission of the encoded data (step S24).

  In the above-described embodiment, the transmission apparatus has not only encoded data but also a decoding computer program for causing a computer to perform the same operation as the moving picture time axis hierarchical signal decoding apparatus. You may make it transmit simultaneously.

  Next, the moving picture time axis hierarchized signal receiving apparatus of the present invention will be described. FIG. 10 shows a block diagram of an embodiment of a moving picture time-axis hierarchical signal receiving apparatus. Here, it is assumed that encoded data and a decoding computer program are transmitted from the network. In the figure, a reception interface (I / F) 81 is connected to a network (not shown) and performs an operation according to the flowchart shown in FIG. That is, it is determined whether or not reception of a signal received from the network is permitted by authentication or the like (step S31). If reception is permitted, the encoded data and the computer program for decoding from the network are received and demodulated. (Step S32).

  Subsequently, the reception I / F 81 deformats the received encoded data and decoding computer program (step S33) and records them in the memory (step S34). If it is determined in step S31 that reception is not permitted, reception is stopped (step S35).

  Of the encoded data and the decoding computer program stored in the memory of the reception I / F 81, the encoded data is supplied to the computer 82 shown in FIG. 10, and the decoding computer program is once taken into the program buffer 83. Thereafter, the data is supplied to the computer 82, and the same operation as that of the decoding apparatus shown in FIG. 2 or 5 is performed by software. As a result, the moving image signal of the original interlaced scanning is output from the computer 82.

  8 and 10, it has been described that the computer program is received via a network. However, it goes without saying that the computer program recorded on the recording medium may be reproduced and used.

It is a block diagram of 1st Embodiment of the moving image time-axis hierarchy encoding apparatus of this invention. It is a block diagram of 1st Embodiment of the moving image time-axis hierarchy decoding apparatus of this invention. It is a figure explaining the image structure of each part of FIG. It is a block diagram of 2nd Embodiment of the moving image time-axis hierarchy encoding apparatus of this invention. It is a block diagram of 2nd Embodiment of the moving image time-axis hierarchy decoding apparatus of this invention. It is a flowchart of one Embodiment of the moving image time-axis hierarchy encoding operation | movement by the computer program of this invention. It is a flowchart of one Embodiment of the moving image time-axis hierarchy decoding operation | movement by the computer program of this invention. It is a block diagram of one embodiment of a transmitting device of a moving picture time-axis hierarchical coding signal of the present invention. 9 is a flowchart for explaining an operation of an example of the transmission I / F in FIG. 8. It is a block diagram of one Embodiment of the transmission apparatus of the moving image time-axis hierarchy decoding signal of this invention. 11 is a flowchart for explaining the operation of an example of the reception I / F in FIG. 10. It is a block diagram of an example of the conventional moving image time-axis hierarchy encoding apparatus. It is a block diagram of an example of the conventional moving image time-axis hierarchy decoding apparatus. It is a figure which shows the code sequence structure of the past and this invention, respectively.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image input terminal 2 Sequential scan converter 3, 10, 27 Switch 4, 36 Encoder 5 Multiplexer 6 Code sequence output terminal 7 Image setting device 8, 26 Inter-picture predictor 9, 37 Local decoder 11, 22, 29 Field decimation device 12, 23 Image delay device 13 Subtractor 14 DCT
DESCRIPTION OF SYMBOLS 15 Quantizer 16 Variable length encoder 21, 41 Decoder 24 Code stream input terminal 25 Demultiplexer 28 Image output terminal 30 Variable length decoder 31 Inverse quantizer 32 Inverse DCT
33 Adder 35 Scan Line Down Sampler 38, 42 Scan Line Up Sampler 71, 81 Reception Interface (I / F)
72, 83 Program buffer 73, 82 Computer 74 Transmission interface (I / F)

Claims (8)

A first step of converting the input interlaced scanned image signal into a sequentially scanned image signal having the same frame rate as the input interlaced scanned image signal;
A second step of encoding the progressive scanning image signal to obtain a first code string;
A local decoded image obtained by local decoding of the first code sequence is used as a reference image, and a field at a time position different from the sequentially scanned image signal is subjected to inter-picture prediction encoding in the input interlaced scanned image signal. A third step of obtaining a code string of
And a fourth step of multiplexing the first code string and the second code string and outputting the multiplexed signal as a moving picture time axis hierarchical coded signal. A first code sequence encoded with respect to a progressive scan image signal having the same frame rate as that of the reproduced interlaced scanned image signal from the time sequence hierarchical code sequence, and the progressively scanned image signal in the reproduced interlaced scanned image signal Separating a second code string encoded for a field at a different time position with
A second step of decoding the first code string to obtain a reproduced sequential scan image signal;
A third step of converting the reproduction sequential scan image signal into an image signal of a first field which is one of an odd field or an even field of the reproduction interlaced scan image signal;
Inter-picture predictive decoding is performed on the second code string using the reproduced sequential scanning image signal as a reference image, and a second field image signal different from the first field of the reproduced interlaced scanning image signal is obtained. And the steps
A moving image time axis hierarchy comprising: a fifth step of switching the image signal of the first field and the image signal of the second field in time and outputting the image signal as the reproduction interlaced scan image signal. Decryption method. Image conversion means for converting the input interlaced scanning image signal into a sequential scanning image signal having the same frame rate as the input interlaced scanning image signal;
First code string generation means for encoding the progressive scanning image signal to obtain a first code string;
A local decoded image obtained by local decoding of the first code sequence is used as a reference image, and a field at a time position different from the sequentially scanned image signal is subjected to inter-picture prediction encoding in the input interlaced scanned image signal. Second code string generation means for obtaining a code string of
A video time-axis hierarchical encoding apparatus, comprising: multiplexing means for multiplexing the first code string and the second code string and outputting the result as a video time-axis hierarchical encoded signal.   The first code is further provided with scanning line down-sampling means for down-sampling the progressive scanning image signal obtained by the image converting means in the vertical direction to obtain a progressive scanning image signal with a reduced number of scanning lines. The column generation means encodes the progressive scanning image signal with the reduced number of scanning lines to generate a first code string with the reduced number of scanning lines, and the second code string generation means has the number of scanning lines. The local image signal obtained by local decoding of the first code sequence with reduced is up-sampled in the vertical direction so as to have the same number of scanning lines as the sequential scanning image signal input to the scanning line down-sampling means 4. The moving picture time-axis hierarchical coding apparatus according to claim 3, further comprising scanning line up-sampling means that uses a signal as the reference picture. A first code sequence encoded with respect to a progressive scan image signal having the same frame rate as that of the reproduced interlaced scanned image signal from the time sequence hierarchical code sequence, and the progressively scanned image signal in the reproduced interlaced scanned image signal Demultiplexing means for separating a second code string encoded for a field at a different time position from
First decoding means for decoding the first code string to obtain a reproduced progressively scanned image signal;
Image conversion means for converting the reproduction sequential scanning image signal into an image signal of a first field which is one of an odd field or an even field of the reproduction interlaced scanning image signal;
The second code string is subjected to inter-picture predictive decoding using the reproduced sequential scanning image signal as a reference image to obtain an image signal of a second field different from the first field of the reproduced interlaced scanning image signal. Decryption means of
Video time-axis hierarchical decoding, comprising: multiplexing means for temporally switching the image signal of the first field and the image signal of the second field and outputting as the reproduced interlaced scanned image signal Device.   The demultiplexing means encodes a third code encoded with respect to a sequentially scanned image signal that is down-sampled in the vertical direction at the same frame rate as the reproduced interlaced scanned image signal from a time-series hierarchical code string. Means for separating a sequence and a second code sequence encoded in a field at a time position different from that of the progressive scan image signal in the reproduced interlaced scan image signal, wherein the first decoding means is , Means for up-sampling the progressive scan image signal obtained by decoding the third code string in the vertical direction, and the image conversion means is configured to output the progressive scan image signal up-sampled in the vertical direction. 6. The moving picture time-axis hierarchical decoding apparatus according to claim 5, wherein the moving picture time-axis hierarchical decoding apparatus is means for converting the first field image signal into the image signal. On the computer,
A first step of converting the input interlaced scanned image signal into a sequentially scanned image signal having the same frame rate as the input interlaced scanned image signal;
A second step of encoding the progressive scanning image signal to obtain a first code string;
A local decoded image obtained by local decoding of the first code sequence is used as a reference image, and a field at a time position different from the sequentially scanned image signal is subjected to inter-picture prediction encoding in the input interlaced scanned image signal. A third step of obtaining a code string of
A computer program for executing the fourth step of multiplexing the first code string and the second code string and outputting them as a moving picture time-axis hierarchical encoded signal. On the computer,
A first code sequence encoded with respect to a progressive scan image signal having the same frame rate as that of the reproduced interlaced scanned image signal from the time sequence hierarchical code sequence, and the progressively scanned image signal in the reproduced interlaced scanned image signal Separating a second code string encoded for a field at a different time position with
A second step of decoding the first code string to obtain a reproduced sequential scan image signal;
A third step of converting the reproduction sequential scan image signal into an image signal of a first field which is one of an odd field or an even field of the reproduction interlaced scan image signal;
Inter-picture predictive decoding is performed on the second code string using the reproduced sequential scanning image signal as a reference image, and a second field image signal different from the first field of the reproduced interlaced scanning image signal is obtained. And the steps
A computer program for executing the fifth step of switching the image signal of the first field and the image signal of the second field temporally and outputting them as the reproduction interlaced scan image signal.

Images