JP3812407B2 - STREAM AREA EXTRACTION METHOD AND DEVICE, PROGRAM THEREOF, AND RECORDING MEDIUM CONTAINING THE PROGRAM - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a technique for extracting any one or a plurality of regions from a stream encoded by an MPEG encoding method and other digital encoding methods, and more particularly, a plurality of streams without completely decoding the stream. It relates to the technology to extract.
[0002]
[Prior art]
In Japanese Patent Laid-Open No. 2000-125263, each of a plurality of moving images is combined into one image, and this is encoded by one encoding device, whereby a plurality of moving images can be simultaneously processed by one encoding device. A method of recording or transmitting is shown. The encoding method is advantageous in terms of cost because only one encoder is required for encoding a plurality of moving images. However, when each image is extracted from the stream encoded by the encoding method, it is necessary to perform a process of completely decoding the stream and extracting an arbitrary image from the decoded composite image.
[0003]
[Problems to be solved by the invention]
When extracting each image synthesized from the stream encoded by the encoding device, a process of extracting an arbitrary image from the decoded synthesized image after decoding all images by a general decoding process Since it is necessary, there is a problem that a general decoding device cannot be used for decoding. In addition, there is a problem that wasteful processing increases because an image in an area that is not an extraction target is also subjected to decoding processing. Furthermore, since a plurality of contents combined into one image are recorded as one stream, there is a problem that separate management such as deletion or genre division for each content cannot be performed.
[0004]
Here, the stream encoded by the encoding device is once decoded into an image, an arbitrary area is extracted, and if only that area is re-encoded, it can be decoded by a general decoder and processing for decoding unnecessary areas is performed. There is no need to divide, and a stream for each content can be created by setting the number of areas to be extracted in the image by the number of contents and repeating the re-encoding process. However, in this case as well, there is a problem that a large amount of processing is required because it is necessary to perform decoding processing up to an image and re-encoding of an extraction region in the image. Further, when extracting a plurality of regions, the re-encoding method needs to repeat the extraction process as many times as the number of extraction regions with a single encoder, which requires much more labor and time. Further, when processing is performed using as many encoders as the number of areas to be extracted, only one extraction process is required, but the cost is increased and the apparatus is complicated.
[0005]
The present invention has been made in view of these problems, and a first object of the present invention is to combine a plurality of video contents into a single image, and from one or more streams obtained by encoding it. By extracting multiple areas and creating a new stream for each extracted area, this provides a stream arbitrary area extraction method that facilitates individual management such as deletion and genre division for each content, for example. is there.
[0006]
The second object of the present invention is to extract one or a plurality of regions from the stream and make them separate streams, so that even when decoded by a general decoder, the image of the region to be extracted is as intended. It is to provide a stream arbitrary area extraction method that can be decoded.
[0007]
A third object of the present invention is to provide an arbitrary stream that can quickly extract one or a plurality of areas from the stream and that can extract a plurality of areas in a single process regardless of the number of areas to be extracted. It is to provide a region extraction method.
[0008]
A fourth object of the present invention is to detect one or more areas from the stream and arrange them for each unit of data in the recording medium when detecting the target data when recording it on a recording apparatus. When a recording medium with a relatively long seek time until the target data is detected is recorded by recording so that the number of seek operations is reduced, the stream read processing from the recording device is performed for decoding the recorded stream It is an object of the present invention to provide a stream arbitrary area extraction method that can prevent a situation where the time is not in time and can further reduce the burden on the recording apparatus.
[0009]
According to a fifth object of the present invention, when a plurality of areas are extracted from the stream and recorded on a recording apparatus, the extracted streams are arranged adjacent to each other in a unit of data on the recording medium. By recording, the number of seeks and seek time can be reduced even when simultaneous decoding of multiple streams is performed, so it is possible to prevent situations where the read processing of each stream is not in time and further reduce the burden on the recording device A stream arbitrary area extraction method is provided.
[0010]
[Means for Solving the Problems]
In the stream arbitrary area extraction method of the present invention, the MB area determination unit that determines the area to which the MB data in the MPEG stream belongs, and the MB can be extracted as it is based on the determination result of the MB area determination unit, or re-encoding is necessary. An MB processing determining unit that determines whether or not the MB processing determining unit determines that re-encoding is necessary, and a re-encoding unit that performs re-encoding when the MB processing determining unit determines that re-encoding is necessary. Even when referring for motion compensation, motion compensation is performed for each MPEG stream after extraction by performing motion prediction from the same region of the reference image or by re-encoding as an intra-frame MB without motion compensation. Reference can be made from within the extracted region. Furthermore, a slice configuration unit that reconstructs slice data to fit the extraction area by the extracted MB data, and a header that modifies and duplicates the header information of the MPEG stream according to the MPEG standard according to the number and range of extraction areas By including a duplicating unit and a synthesizing unit (synthesizing unit A, synthesizing unit B) for synthesizing the configured slice data and the modified and duplicated header information for each region, any arbitrary MPEG stream can be obtained. A new stream can be extracted from one or a plurality of areas and recorded in a recording device. When streams are extracted and recorded from a plurality of areas, each stream is recorded as an independent stream conforming to the MPEG standard, so that each stream can be deleted or managed individually. Can achieve the purpose. Here, the extracted stream is a stream including only an image as an extraction area, and is a stream that complies with the MPEG standard as described above. Therefore, each extracted stream is decoded by a general-purpose decoder. However, the image of the region to be extracted can be decoded as intended, and the second object of the present invention can be achieved.
[0011]
In the arbitrary region extraction processing according to the configuration of the present invention, the MB data is mainly classified for each region to which the data belongs, and the re-encoding processing requiring a very large load is almost necessary for the boundary between the regions. Since only the nearby MB data is required, the processing can be performed at a very high speed as compared with a general method in which the entire region is completely decoded once and an arbitrary region is re-encoded. The MB data sorted for each region is combined with the header information duplicated by the number of extractions by the header duplication by the synthesis unit provided for the number of extraction regions, respectively, so that the streams for the number of extractions are configured at the same time. That is, regardless of the number of areas to be extracted, a plurality of areas can be extracted by a single process, and the third object of the present invention can be achieved.
[0012]
In the stream arbitrary area extraction method of the present invention, since the slice data configured by the slice configuration unit is output to the recording medium, the arrangement of the stream on the recording medium is almost the arrangement of each slice data. When a recording medium having a relatively long seek time until the target data is detected, such as an optical disk, is used, there may occur a situation where stream read cannot be performed in time for decoding the recorded stream. In order to prevent this, a memory A and a memory B that store the extracted streams in a certain unit of data, and a write control unit that reads out the stream from the units of data from the memory A and the memory B and outputs them to the recording device continuously. The stream arbitrary area extraction method is configured to be provided, and the extracted stream is output to the recording device so as to be continuously arranged in the data unit on the recording medium, so that the number of seeks and seek time of the recording device can be reduced. Since it is possible to prevent the read process from being in time for the stream decoding process and to reduce the load on the recording apparatus, the fourth object of the present invention can be achieved. Further, by outputting each of the plurality of streams extracted by the write control unit to the recording apparatus so as to be arranged on the recording medium adjacent to each data unit, even when simultaneous decoding processing of the plurality of streams is performed, the recording apparatus The number of seeks and seek time can be reduced. As a result, it is possible to prevent a situation in which read processing is not in time even when simultaneously decoding a plurality of streams, and the load on the recording apparatus can be reduced. Thus, the fifth object of the present invention can be achieved.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
An embodiment of a stream arbitrary area extraction method of the present invention will be described in detail with reference to the drawings.
[0014]
(First Embodiment of the Invention)
Stream of the present invention Arbitrary area extraction method The first embodiment has the configuration shown in FIG.
[0015]
In this embodiment, a region designation signal for designating the number of extractions and extraction regions is designated from the outside. As shown in FIG. 2, the number of region extractions is 2, and the horizontal position h of an image of 720 × 480 pixels is 0 to 351. 352 pixels up to 352 pixels from 368 to 719 pixels are defined as region B, and regions A and B are defined as extraction regions. However, the number of extraction and the extraction area are not limited to this. When the number of extractions and extraction areas are fixed, it is not necessary to specify an area designation signal from the outside. Further, it is also possible to input some parameters as an area designation signal without designating the number of direct areas or the extraction areas, and calculate the number of extractions or the extraction areas therefrom. Further, the number of extractions and information on the extraction area may be superimposed on the input stream, and the stream separation unit 101 may separate the information on the number of extractions and the area to be extracted, and the path for inputting the area designation signal may be omitted. . In any case, the extraction area designation of the area designation signal is designated with an accuracy of 16 × 16 pixels corresponding to a macroblock (hereinafter referred to as MB).
[0016]
In the first embodiment shown in FIG. 1, stream arbitrary area extraction according to the present invention is performed. Part 100 The stream separation unit 101, the MB data analysis unit 102, the MB region determination unit 103, the MB processing determination unit 104, the MB region change determination unit 105, the memory 106, the slice configuration unit 107, the header analysis unit 108, the header duplication unit 109, and The combining unit A 110, the combining unit B 111, the re-encoding unit 120, the selector A 130, and the selector B 131 are included.
[0017]
Here, the re-encoding unit 120 further includes an MB decoding unit 121, an MB encoding unit 122, and a reference image memory 123.
[0018]
As shown in FIG. 3, for example, a video stream obtained by encoding an image obtained by combining two images into one by MPEG encoding is input to the stream separation unit 101. The stream separation unit 101 analyzes the input MPEG video stream. Then, the header information such as sequence header, GOP (Group Of Pictures) header, and picture header defined by MPEG is separated from the MB data, and the header information is output to the header analysis unit 108. On the other hand, the MB data is sequentially output to the MB data analysis unit 102 in MB units. In the present embodiment, it is assumed that the images A and B are reduced by pixel thinning or the like and are arranged in the regions A and B in FIG.
[0019]
The MB data analysis unit 102 analyzes the MB data input from the stream separation unit 101, and extracts a macro block address (hereinafter referred to as MBA) and motion vector information therefrom. The MB data analysis unit 102 also calculates the position (MBx, MBy) in the image of the MB data from the MBA according to Equation 1. Here, H_MB_NUM in the expression 1 is the number of horizontal MBs obtained by dividing the number of horizontal pixels of the image input from the header analysis unit 108 by 16, and% is an operator for calculating the remainder of integer division. Represents an operator that calculates an integer division quotient.
(MBx, MBy) = ((MBA% H_MB_NUM) × 16, (MBA / H_MB_NUM) × 16) Equation 1
The MB data analysis unit 102 also stores the motion vector of the MB data input immediately before. If the motion vector of the MB data is a difference value of the previous MB data, the MB data analysis unit 102 stores the immediately preceding stored MB data. By adding a motion vector obtained by adding a motion vector of MB data to the calculated MB position (MBx, MBy), a position (RMBx, RMBy) referred to by the MB is calculated. If it is not the motion vector difference value of the MB data, the position referred to by the MB is calculated by adding the motion vector of the MB data to the calculated MB position. If the MB data is an intra MB that does not require motion compensation, the position referenced by the MB is set to the same value as the position of the MB, for example. When the MB has a plurality of motion vectors, reference positions for all motion vectors are calculated. The MB data analysis unit 102 outputs the position of the MB calculated in this way and the position referred to by the MB to the MB region determination unit 103, and outputs the MB data to the selector A 130 and the MB combination unit 121. . The MB data analysis unit 102 also outputs an intra MB determination signal for determining whether or not the MB data is an intra MB to the MB processing determination unit 104. The intra MB determination signal may be, for example, “Intra” for an intra MB and “NotIntra” for a non-intra MB.
[0020]
In the MB area determination unit 103, the extracted area information obtained from the area designation signal input from the outside, the MB position information (MBx, MBy) input from the MB data analysis unit 102, and the MB reference position information (referred to by the MB) ( An area to which the MB position and the MB reference position belong is determined from RMBx, RMBy), and an MB area determination signal corresponding to the area to which the MB belongs is sent to the MB area change determination unit 105, the MB encoding unit 122, and the selector B 131. Output to. The MB area determination signal includes information on whether or not the area to which the MB belongs is targeted for extraction. Here, when the region A and the region B are extracted regions from an image of 720 pixels × 480 pixels as shown in FIG. 2, the region to which the MB belongs can be determined as follows.
(A) MBx ≧ 0 and (MBx + 15) ≦ 359
The MB belongs to the area A.
(B) MBx ≧ 360 and (MBx + 15) ≦ 719
The MB belongs to region B.
(C) ( a )and( b Other than
The MB is between region A and region B.
[0021]
At this time, for example, in the case of (a), “A”, in the case of (b) “B”, and in the case of (c), “X” is an MB area as an area not to be extracted in this embodiment. The determination signal is output to the MB region change determination unit 105, the MB encoding unit 122, and the selector B 131.
[0022]
The MB region determination unit 103 further outputs the same region reference determination signal indicating the match or mismatch between the MB position and the region to which the MB reference position belongs to the MB processing determination unit 104. The same region reference determination signal also includes information on whether or not the region to which the MB belongs is targeted for extraction. Here, as in the MB area determination method, the MB and its reference position are determined. If the MB and the area referred to by the MB match, for example, “Same”, and if the MB does not match, “Diff”. Or when the MB is an area that is not subject to extraction, including the case of (c), “X” is output to the MB processing determination unit 104 as the same area reference determination signal.
[0023]
The MB processing determination unit 104 uses the intra MB determination signal input from the MB data analysis unit 102 and the same region reference determination signal input from the MB region determination unit 103 to process the MB in the processing flow shown in FIG. It is determined whether it is necessary to re-encode or whether it is an extraction target, and the processing determination signal obtained as a result is output to the MB decoding unit 121 and the selector A 130. In the present embodiment, the processing decision signal is “Orginal” when re-encoding is not necessary, “ReENC” when re-encoding is necessary, and “Dump” when it is not an extraction target.
[0024]
The flow until the processing determination signal shown in FIG. 4 is output will be described. First, in step A1, if the same region determination signal input from the MB region determination unit 103 is “X”, that is, if it is determined that the MB is not an extraction target region, the process proceeds to step A4 and “Dump” is output. . If not, the process proceeds to step A2 to check whether or not the intra MB determination signal input from the MB data analysis unit 102 is “Intra”. If the result of determination is “Intra”, that is, if it is determined that the MB is an intra MB, the process proceeds to step A5, where “Original” is output. If not, the process proceeds to step A3, and it is checked whether the same area determination signal is "Same". If the result of determination is “Same”, that is, if it is determined that the area to which the MB belongs and the area to which it refers, the process proceeds to step A6, where “Original” is also output. Otherwise, “ReENC” is output. The flow until outputting the processing determination signal does not necessarily follow the flow chart of FIG. 4 described above, and it is sufficient if a similar result can be obtained.
[0025]
The MB region change determination unit 105 compares the MB region determination signal input from the MB region determination unit 103 with the MB region determination signal input immediately before, and the region to which the MB belongs is a certain extraction target. When changing from an area to another extraction target area, or when changing from a certain extraction area to a non-extraction target area, an MB area change signal notifying this is output to the slice configuration unit 107. Specifically, when the MB area determination signal changes from “A” to “B” or “X”, or when it changes from “B” to “A” or “X”, the MB area change signal is Output.
[0026]
The MB decoding unit 121 determines that the MB data is an MB of an image that is referred to during motion compensation from another screen such as an I picture or a P picture according to the picture type information input from the header analysis unit 108, or determines MB processing. When the processing decision signal input from the unit 104 is “ReENC”, that is, when re-encoding is necessary, variable length decoding, inverse quantization and inverse DCT are performed on the input MB data, and further reference is made when necessary. Using the reference image in the image memory 123, motion compensation is performed and the image is decoded to a 16 × 16 image. When the processing determination signal is “ReENC”, the decoded 16 × 16 image is output to the MB encoding unit 122. If the MB data is an I picture or a P picture, the decoded 16 × 16 image is stored in the reference image memory 123 to serve as a reference image.
[0027]
When the MB processing determining unit 104 determines that the MB needs to be re-encoded, the MB encoding unit 122 performs a reference image on the MB data decoded into the 16 × 16 image input from the MB decoding unit 121. Motion prediction is performed from the same area as the MB in the reference image stored in the memory 123, and then re-encoded by DCT, quantization, and variable length encoding, and the re-encoded MB data is output to the selector B 131. To do. Here, it is possible to determine which area of the reference image is the same as the MB by using an area designation signal designated from the outside and area information to which the MB data input from the MB area judgment unit 103 belongs. . When it is desired to reduce the amount of processing, it may be encoded as an intra-frame MB without performing motion prediction at the time of re-encoding.
[0028]
The MB encoding unit 122 performs the re-encoding process when the MB process determining unit 104 determines that the MB needs to be re-encoded, but this is performed in a certain extraction region as shown in FIG. This is a case where the MB data to which it belongs refers to a different area of the reference image for motion compensation. Since the reference to the different area is almost in the vicinity of the boundary between the areas, re-encoding processing with a heavy load is not performed on most other MB data. Therefore, a high-speed arbitrary area extraction process can be performed.
[0029]
The reference image memory 123 stores MB data decoded by the MB decoding unit 121 into a 16 × 16 image. At this time, the MB data is arranged in the memory so that it can be referred to as a reference image. The reference image memory 123 stores reference screens for two screens, and each time a new reference image is stored, the reference images are deleted from the reference image memory 123 in the oldest order.
[0030]
The selector A 130 uses the processing determination signal input from the MB processing determination unit 104 to change the MB data output to the memory 106 to the MB data (FIG. 1 terminal a) input from the MB data analysis unit 102 or the MB encoding unit. The MB data input from 122 (terminal c in FIG. 1) or none of them is input (terminal b in FIG. 1), and when any MB data is selected, it is output to the memory 106. . As a specific example, when the processing decision signal is “Original”, the MB data not re-encoded input from the MB data analysis unit 102 is selected, and when it is “ReENC”, the MB encoding unit 122 inputs it. Re-encoded MB data is selected. Further, when the processing determination signal is “Dump”, MB data is not input and MB data is not output to the memory 106.
[0031]
The slice configuration unit 107 determines that the MB data of the MB belonging to the same region and in the same vertical position is stored in the memory 106 based on the MB region change signal input from the MB region change determination unit 105, and stores in the memory 106. The stored MB data is sequentially read out, and a slice header is generated using the information of the MB data, and one slice of data is configured by the generated header and the read MB data, and the configured slice data Is output to the selector B 131. If there is no MB data to be read to the memory 106 when the MB area change signal is input, the slice header is not generated and the slice data is not formed, and nothing is output to the selector B 131.
[0032]
When generating the slice header, the MB data stored in the memory 106 is analyzed. Specifically, the value of the quantization scale code of the MB data that is the head of the slice is used as the value of the quantization scale code in the slice header. Further, when all the MB data to be made into one slice are all intra MBs, the flag in the slice header indicating that is set to 1. In addition, when a slice is configured with MB data in the memory 106, processing is performed when it is necessary to change or delete the motion vector value in the MB data, the DC coefficient of the intra MB, and the quantization scale code. The MB data change or deletion process will be specifically described. The motion vector is not immediately after the MB region change signal input from the MB region conversion determination unit 105, that is, not the first MB of the slice, and the immediately preceding MB is not an intra MB, and the immediately preceding MB and the MBA of the MB are When the values are continuous, the motion vector value of the MB is changed to a difference value from the motion vector of the previous MB. The intra DC coefficient is such that when the MB is not the head of the slice, the previous MB and the MB are both intra MBs and the MBA is a continuous value, the intra DC coefficient value of the MB is the intra DC coefficient of the previous MB. Change to the difference value. The quantization scale code is deleted from the MB data when the MB has the same value as the quantization scale code of the MB at the head or immediately before the slice. As a result, as shown in FIG. 6, a set of MB data at the same horizontal position for each extraction region is set as one slice.
[0033]
In this embodiment, the slice configuration as shown in FIG. 6 is used, but the slice may have other configurations as long as it conforms to the MPEG standard.
[0034]
The header analysis unit 108 analyzes the header information input from the stream separation unit 101, and the picture type such as an I picture in which the image is encoded in the image or a P or B picture in which motion prediction is performed from another screen. Information is extracted and output to the MB decoding unit 121. On the other hand, the header information is output to the header duplication unit 109. Further, the number of horizontal pixels of the image is output to the MB data analysis unit 102.
The header duplicating unit 109 duplicates the header information input from the header analyzing unit 108 as many as the number of areas to be extracted, and at the same time, the screen pixel size and display size in the header information, a horizontal ratio of 1 pixel, a ratio of vertical size, etc. Change the parameter value that needs to be changed to extract the area. In this embodiment, the number of extraction is two, and the header information is duplicated into two. In addition, the pixel size of each extraction area is 352 × 480, the display size is 720 × 480, for example, and the ratio of the vertical and horizontal sizes of one pixel can be set to a value corresponding thereto. After duplicating the header, the header duplicating unit 109 outputs the header information about the area A to the synthesizing unit A 130 and the header information about the area B to the forming unit B 131.
[0035]
The selector B 131 distributes the output destination of the slice data input from the slice configuration unit 107 to the synthesis unit A 110 or the synthesis unit B 111 according to the MB region determination signal input from the MB region determination unit 103. Specifically, the slice data of region A is output to combining unit A 110 and the slice data of region B is output to combining unit B 111.
[0036]
The synthesizing unit A 110 synthesizes the header data input from the header duplicating unit 109 and the slice data input from the selector B 131 and outputs the combined MPEG stream to the recording apparatus 200 as the file A 201.
[0037]
Similar to the synthesizing unit 110, the synthesizing unit B 111 synthesizes the header data input from the header duplicating unit 109 and the slice data input from the selector B 131, and outputs the resulting MPEG stream to the recording apparatus 200 as the file B 202. To do.
[0038]
Here, the MPEG streams output from the synthesizing unit A 110 and the synthesizing unit B 111 are streams including only the images of the area A and the area B, respectively, and the first embodiment of the present invention described above is performed. Extraction processing of an arbitrary area of the stream is achieved by the configuration of the form. In addition, streams extracted from a plurality of areas in one process can be obtained simultaneously.
[0039]
Further, at that time, the extracted MPEG stream is modified so as to refer only to the respective areas to be extracted by the action of the MB processing determining unit 104 and the re-encoding unit 120. The header information is an appropriate parameter value corresponding to the extraction region due to the action, and the slice configuration corresponding to the extraction area is achieved due to the action of the slice configuration unit 107. Therefore, the extracted MPEG stream is a stream compliant with the MPEG standard, and when decoded by a general decoder, the image in the extraction area is decoded as intended.
[0040]
In the first embodiment, the MPEG streams output from the combining unit 110 and the combining unit 111 are output to the same storage device 200, but may be output to separate storage devices. Further, the output MPEG stream may be directly output to the decoder. Furthermore, the storage device or decoder that outputs the MPEG stream need not exist in the device, and may be arranged on the network.
[0041]
In the first embodiment, an example in which an arbitrary area is extracted from an MPEG stream has been described. However, in a similar method, an image is composed of a plurality of blocks such as MB or DCT blocks, and independent processing is performed for each block unit. It can be applied to an encoding method capable of When the encoding method is an encoding method that does not use the correlation between frames, the re-encoding unit 120 is not necessary, so that the stream arbitrary region extraction method can be implemented with a simpler configuration.
[0042]
(Second Embodiment of the Invention)
In the second embodiment of the present invention, in addition to the configuration of the embodiment of the present invention shown in FIG. 1 as shown in FIG. 7, a code amount control unit is provided in the subsequent stage of each of the combining unit A 110 and the combining unit B 111. 140 is further provided.
[0043]
The code amount control unit 140 includes a code rate conversion unit 141 and a code amount calculation unit 142 as shown in FIG.
[0044]
The code rate conversion unit 141 performs variable length decoding and inverse quantization on the input MPEG stream. After that, based on the code amount control signal input from the code amount calculation unit 142, a stream that is quantized with a new quantization step value and variable-length encoded and converted in code rate is output to the code amount calculation unit 142. Here, the code amount control signal is a signal representing an error in the actual code rate of the stream output from the code rate conversion unit 141 with respect to a target code rate designated from the outside. Therefore, when it is determined by the code amount control signal that the actual code rate is larger than the target code rate, the code rate can be reduced by setting the quantization step value to a large value, and the actual code rate is reduced. Can be controlled to be the target code rate.
[0045]
The code amount calculation unit 142 counts the code amount of the stream in which the code rate input from the code rate conversion unit 141 is converted, and for example, from the target code rate input from the outside for each picture and the count value of the code amount The calculated actual code rates are compared, and the error of the actual code rate with respect to the target code rate is output to the code rate conversion unit 141 as a code amount control signal. Also, the code amount calculation unit 142 outputs the input stream as it is. The comparison between the target code rate and the actual code rate is not limited to each picture, but may be made for each slice or other unit.
[0046]
As described above, in the second embodiment of the present invention, the code amount control unit 140 is provided in the subsequent stage of each of the combining unit A 110 and the combining unit B 111, and each code amount control unit 140 has a target for each extraction region. By inputting the code rate, the code rate of each stream extracted from each region can be arbitrarily set.
[0047]
The process performed by the code rate conversion unit 141 is a process with a much smaller load compared to the DCT or inverse DCT process, and can be further increased by processing the inverse quantization and the quantization together. . Therefore, according to the second embodiment, in addition to the function that enables the stream arbitrary area extraction method shown in the first embodiment while maintaining high speed, the code rate of each stream extracted from each area can be changed. An arbitrarily settable stream arbitrary area extraction method can be provided.
[0048]
In the second embodiment of the present invention, the code rate conversion unit 141 is arranged downstream of the combining unit A 110 and the combining unit B 111. However, the code rate conversion unit 141 is connected to the subsequent stage of the selector A 130, Alternatively, it may be arranged after the memory 106 or after the slice configuration unit 107. In this case, since it is necessary to input the code amount control signal for each stream extracted from the region A and the region B to the code rate conversion unit 141 and to determine the region to which the MB belongs, the MB region determination Although it is necessary to input an MB region determination signal from the unit 103, only one code rate conversion unit 141 is required regardless of the number of extraction regions, and the apparatus can be simplified.
[0049]
(Third embodiment of the invention)
The third embodiment of the present invention has a configuration shown in FIG. 9, further including a memory A 300, a memory B 400, and a write control unit 500 in addition to the stream arbitrary area extraction unit 100. The stream arbitrary area extraction unit 100 includes the stream arbitrary area extraction unit 100 in FIG. 1 which is the first embodiment of the present invention or the stream arbitrary area extraction unit 100 in FIG. 7 which is the second embodiment of the present invention. The recording apparatus 200 has the same function and can control the data writing position by a control signal from the write control unit 300.
[0050]
The memory A 300 and the memory B 400 record the two streams extracted from the area A and the area B by the stream arbitrary area extraction unit 100 input to each of them.
[0051]
The write control unit 500 uses a stream that is gathered to some extent as one frame or GOP as a data unit, and after the stream of the data unit is stored in the memory A 300 and the memory B 400, the memory A 300 and the memory B 400 respectively The data unit stream is read and output to the recording apparatus 200. At this time, a control signal for designating a recording position such that the read stream is arranged on the recording medium as physically as possible in units of the data is output to the recording apparatus 200. Further, when writing streams extracted from a plurality of areas, a control signal is output to the recording apparatus 200 so that the streams are arranged as close as possible to the recording medium. For example, when the recording medium used in the recording apparatus 200 is an optical disk, the physical arrangement of the streams extracted from the areas A and B is as shown in FIG. 10, and the streams extracted from the areas A and B are in units of the data. It is recorded continuously, and the respective streams are arranged alternately next to each other. The data unit may be a unit of one frame, a unit of GOP, a unit of more than that, or a unit of less than that. Further, the data unit may be the entire stream.
[0052]
At this time, when the recording apparatus 200 uses a ZCAV (Zone Constant Angular Velocity) method in which the optical disk is divided into several zones from the inner periphery to the outer periphery and the rotation speed is constant, the control signal is If the streams are arranged in the same zone as much as possible, it is possible to reduce the frequency with which the recording apparatus 200 changes the rotational speed, which takes time and load during the stream read processing, and thus the seek time and the load on the recording apparatus 200 can be reduced.
[0053]
When the recording apparatus 200 can accurately control the physical writing position by the control signal and the access speed such as seek time is sufficient, the memory A 300 and the memory B 400 can be omitted. . On the other hand, even when the recording device 200 cannot control the physical writing position by the control signal, the memory A 300 and the memory B 400 have the data unit to be written as a recordable capacity, and the stream is continuous in the data unit. By outputting to the recording apparatus 200, recording can be performed as physically continuous as possible in most recording apparatuses.
[0054]
With the third embodiment of the present invention having the above-described configuration, when a stream extracted from one or a plurality of areas from a stream is recorded on a recording device, each data unit of the stream is collected. It can be continuously arranged on the recording medium. As a result, when the stream is read from the recording apparatus 200 and decoded, the number of seeks and the seek time can be minimized, and a situation where the read process of the stream from the recording apparatus 200 is not in time during the decoding process can be prevented. It is possible to provide a stream arbitrary area extraction method that can reduce the burden on the stream. Also, with the configuration of the third embodiment of the present invention, each stream extracted from a plurality of areas can be arranged adjacent to the recording medium. As a result, the number of seeks and the seek time can be minimized even when performing simultaneous decoding processing of a plurality of streams, so that a situation in which read processing of the stream from the recording device 200 is not in time during simultaneous decoding processing of a plurality of streams can be prevented. It is possible to provide a stream arbitrary area extraction method that can reduce the burden.
[0055]
Stream Any area extracting method according to the present invention, when carried out as the above embodiment, a respective block of the stream any area extraction method can be realized by a circuit having an equivalent function. Further Similarly, each block of the stream any area extraction method can be implemented by executing a program for performing this processing equivalent on computer. At this time, the computer includes a recording medium for storing the program. This recording medium may be a semiconductor memory, a magnetic disk, an optical disk or other recording medium. Furthermore, since the processing in the re-encoding unit 120 in the stream arbitrary area extraction method of the present invention is relatively large in addition to processing, if it is not possible to achieve the required processing speed by software, those portions are realized by a circuit. It doesn't matter. Thus, in the practice of the present invention, the circuit part of the block, that is configured by hardware, may be the other portions as software configurable.
[0056]
Further, as the memory A 300 and the memory B 400, a semiconductor memory, a magnetic disk drive, can be carried out by a high-speed recording apparatus in time enough for buffering the stream to other.
[0057]
The recording device 200 can be implemented by a semiconductor memory or a device capable of writing / reading data to / from a magnetic disk, optical disk, or other storage medium. In practicing the third embodiment of the present invention, it is desirable that the recording apparatus 200 can control the physical recording position on the recording medium by a control signal, but this is not necessarily the case.
[0058]
【The invention's effect】
As described above, the present invention has the following effects.
[0059]
The first effect of the present invention extracts an image portion of the one or more optional areas from the MPEG stream as a separate stream is to be recorded as a separate file. Thus if the recording management of each content is facilitated as extracted separate file each content it was synthesized, for example a plurality of video contents into one image from the stream obtained by coding.
[0060]
The second effect of the present invention is that when decoding each stream, it is not necessary to use an MPEG decoder considering that the stream arbitrary area extraction method of the present invention is used, and a general MPEG decoder is used. Thus, an image as an extraction region can be decoded from each stream. This is because when MB data belonging to an extraction area is extracted, if the MB referred to by the MB belongs to a different area, the MB data decoded once and re-encoded by predicting motion from the same area is extracted. This is realized by duplicating the number of areas and reconstructing a stream compliant with the MPEG standard with MPEG header information changed according to the extraction area.
[0061]
The third effect of the present invention is that a much faster extraction process can be performed as compared with a general method in which a stream is once decoded into an image and then an extraction region is cut out and re-encoded. This is a method of performing re-encoding only when MB data to be extracted refers to a different region for compensation of a reference image for motion compensation, and in this case, MB data to be re-encoded is almost each region. This is because only the MB data in the vicinity of the boundary is not subjected to re-encoding processing with a very heavy load on most MB data.
[0062]
The fourth effect of the present invention is that, even when there are a plurality of extraction areas, stream extraction of all areas can be performed by a single process. This is because MB data belonging to each extraction area is extracted in a single process, and is combined with the header information copied by the number of extraction areas and slice data composed of MB data belonging to each area. This is because the stream of each area is created sequentially.
[0063]
The fifth effect of the present invention is that the extracted stream can be set to an arbitrary code rate. This changes the number of quantization steps of the stream according to the difference between the target code rate specified for each extracted stream and the actual code rate of the actual stream, and controls the actual code rate to be the target rate. Because.
[0064]
The sixth effect of the present invention is that the extracted stream is recorded on a recording medium so as to be continuously arranged in a certain unit of data, and the data seek frequency and seek time of the recording apparatus are reduced, so that the stream is decoded at the time of stream decoding. It is possible to prevent a situation in which the read process is not in time, and to reduce the load on the recording apparatus.
[0065]
The seventh effect of the present invention is that when a plurality of streams are extracted and recorded on a recording medium, the streams are recorded so as to be arranged adjacent to each other in a unit of data. When performing the decoding process, it is possible to prevent a situation in which the respective stream read processes are not in time, and to reduce the load on the recording apparatus.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a first exemplary embodiment of the present invention.
FIG. 2 is a diagram showing an extraction area in the first embodiment of the present invention.
FIG. 3 is a diagram showing an example of an image to be input in the first embodiment of the present invention.
FIG. 4 is a flowchart showing an MB process determination procedure in an MB process determination unit.
FIG. 5 is a diagram illustrating different area reference in a reference image;
FIG. 6 is a diagram illustrating a slice configuration example in a slice configuration unit.
FIG. 7 is a block diagram showing a configuration of a second exemplary embodiment of the present invention.
FIG. 8 is a block diagram illustrating a code amount control unit.
FIG. 9 is a block diagram showing a configuration of a third exemplary embodiment of the present invention.
FIG. 10 is a diagram illustrating a stream arrangement on a recording medium.
[Explanation of symbols]
100 stream arbitrary area extraction unit
101 Stream separator
102 MB data analysis department
103 MB area determination unit
104 MB processing decision unit
105 MB region change determination unit
106 MB data storage memory
107 Slice component
108 Header analyzer
109 Header replication part
110 Synthesizer A
111 Synthesis Unit B
120 Re-encoding unit
121 MB decoding unit
122 MB encoding unit
123 Reference image memory
130 Selector A
131 Selector B
140 Code amount control unit
141 Code rate conversion unit
142 Code amount calculator
200 Recording device
201 File A (stream extracted from area A)
202 File B (stream extracted from area B)
300 memory A
400 memory B
500 Light controller

Claims (7)

In a stream arbitrary area extraction method for extracting an area to which any one or a plurality of MBs (macroblocks) belong from a stream encoded by an MPEG (Motion Picture Expert Group) encoding system,
When extracting one or a plurality of areas in an image from a stream encoded by the MPEG encoding method, set from the outside, or set in advance by a predetermined method or determined internally, It is determined whether the MB belongs to the set image area, and when it is determined that the MB in the stream belongs to the set image area, the MB is included in the stream. Extract one or more areas in the set image by extracting the data part of the MB ,
When a region to which the MB belongs and a region to which the MB refers for motion compensation are different, motion prediction is newly performed from the same region as the region to which the MB belongs, and encoding using the correlation between frames or fields is performed. A stream arbitrary area extraction method comprising: means . In a stream arbitrary area extraction method for extracting an area to which any one or a plurality of MBs (macroblocks) belong from a stream encoded by an MPEG (Motion Picture Expert Group) encoding system,
When extracting one or a plurality of areas in an image from a stream encoded by the MPEG encoding method, set from the outside, or set in advance by a predetermined method or determined internally, It is determined whether the MB belongs to the set image area, and when it is determined that the MB in the stream belongs to the set image area, the MB is included in the stream. Extract one or more areas in the set image by extracting the data part of the MB ,
A stream arbitrary region extraction method comprising means for performing intraframe or field encoding when a region to which the MB belongs and a region to which the MB refers for motion compensation are different . In a stream arbitrary area extraction method for extracting an area to which any one or a plurality of MBs belong from a stream encoded by the MPEG encoding system,
It is determined whether or not the MB belongs to one of the image areas of the synthesized image from a stream obtained by combining an image obtained by synthesizing a plurality of images into one image using the MPEG encoding method. To extract all or part of the synthesized image by extracting the data portion of the MB ,
When a region to which the MB belongs and a region to which the MB refers for motion compensation are different, motion prediction is newly performed from the same region as the region to which the MB belongs, and encoding using the correlation between frames or fields is performed. A stream arbitrary area extraction method comprising: means . In a stream arbitrary area extraction method for extracting an area to which any one or a plurality of MBs belong from a stream encoded by the MPEG encoding system,
It is determined whether or not the MB belongs to one of the image areas of the synthesized image from a stream obtained by combining an image obtained by synthesizing a plurality of images into one image using the MPEG encoding method. To extract all or part of the synthesized image by extracting the data portion of the MB ,
A stream arbitrary region extraction method comprising means for performing intraframe or field encoding when a region to which the MB belongs and a region to which the MB refers for motion compensation are different . A stream arbitrary region extraction device that performs processing equivalent to the stream arbitrary region extraction method according to any one of claims 1 to 4 . The program for making a computer perform the process of the stream arbitrary area | region extraction system of any one of Claims 1-4 . The recording medium which recorded the program for making a computer perform the process of the stream arbitrary area | region extraction system of any one of Claims 1-4 .

Images