JP4369318B2 - ENCODING MODE DETERMINING DEVICE, IMAGE ENCODING DEVICE, ENCODING MODE DETERMINING METHOD, AND ENCODING MODE DETERMINING PROGRAM - Google Patents

Description

  The present invention relates to an encoding mode determination device, an image encoding device, an encoding mode determination method, and an encoding mode determination program.

  MPEG-4 is attracting attention as a key technology in the multimedia Internet era. MPEG-4 is characterized by improved coding efficiency and the like compared to MPEG-1 / 2 in order to deal with application areas such as mobile communication and the Internet (see Non-Patent Document 1, for example).

In MPEG-4, a method called AVC is formulated as a new high-efficiency encoding method. AVC is ISO MPEG-4 Part 10 Advanced Video Coding or ITU-T H.264. It is an encoding method called H.264.
This method enables motion estimation and DCT even with an image block of 4 × 4 pixels, and attempts to improve coding efficiency by selecting an image for motion estimation from a plurality of pictures. Since AVC is a high-performance encoding method that incorporates techniques used in conventional encoding methods, optimal use adapted to application areas has become a problem.

  For example, in MPEG-4 before AVC development, the number of combinations of encoding mode candidates (partition size, prediction direction, direct mode, etc.) in units of macroblocks is relatively small, and these candidates are covered during encoding. Even when searching for the optimum coding mode, the processing load on the encoder was not large.

  On the other hand, in AVC, as shown in FIG. 25, a 16 × 16 pixel (hereinafter referred to as 16 × 16) macroblock is converted into a 16 × 16, 16 × 8, 8 × 16, and 8 × 8 macroblock partition (hereinafter referred to as “16 × 16”). Can be divided into small blocks). A small block of 8 × 8 pixels can be divided into 8 × 8, 8 × 4, 4 × 8, and 4 × 4 sub-macroblock partitions.

  Hereinafter, one small block divided into 16 × 16 is divided into a small block Sb1, two small blocks divided into 16 × 8 into small blocks Sb2 and Sb3, and two small blocks divided into 8 × 16 into two small blocks. The small blocks Sb4 and Sb5 and the four small blocks divided into 8 × 8 are referred to as small blocks Sb6 to Sb9.

  In AVC, as shown in FIG. 26, motion estimation can be performed for each of the small blocks Sb1 to Sb9 with reference to a reference picture. The same applies to each sub macroblock partition. Further, in AVC, as shown in FIG. 27, forward prediction (refer to FIG. 27A) referring to a reference picture temporally preceding the coded picture, and temporally subsequent to the coded picture. Inter prediction such as backward prediction referring to the reference picture (see FIG. 27B) or bidirectional prediction referring to the bidirectional reference picture with respect to the encoded picture (see FIG. 27C) is performed. It is possible.

<Processing by conventional encoder>
The processing of the conventional encoder covering these coding modes will be described with reference to FIGS.
In a conventional encoder, motion estimation is performed for all small blocks obtained by dividing an image block by a plurality of division method candidates. Further, a reference picture for each small block and an image block dividing method are selected, and encoding using the selected dividing method is performed.

  Here, an amount of coding cost is used when selecting a reference picture for each small block and selecting an image block division method. The coding cost is an amount represented by the sum of the image quality degradation degree (absolute difference sum between a small block and a predicted image) and the code amount of motion information (such as a motion vector or a difference motion vector), and is an image block unit. The smaller the encoding cost is, the better the encoding efficiency of the image block is. In addition to the sum of absolute differences, a sum of squares of differences, an absolute value sum of errors after difference Hadamard transform, or DCT transform may be used.

  FIG. 28 is a block diagram showing a process flow of motion estimation for each of the small blocks. For each of the small blocks of M × N ((M, N) = (16,16), (16,8), (8,16), (8,8)) obtained by dividing the 16 × 16 image block) Thus, the process of FIG. 28 is performed. The motion estimation processing flow shown in FIG. 28 includes a full-pel prediction step S300, a sub-pel prediction step S301, and a reference direction selection step S302 for a small block.

  The full-pel prediction step S300 performs motion estimation with integer pixel accuracy by forward prediction and backward prediction for M × N small blocks (steps S305 and S306). Specifically, motion estimation within a determined search range (for example, ± 32) is performed with integer pixel accuracy. That is, motion vectors (hereinafter referred to as MV) 0f and MV0b that minimize the coding cost are detected within the search range.

  The sub-pel prediction step S301 performs motion estimation with non-integer pixel accuracy by forward prediction, backward prediction, and bidirectional prediction for M × N small blocks (steps S307 to S309). In AVC inter prediction, motion estimation can be performed with non-integer pixel accuracy such as 1/2 pixel accuracy or 1/4 pixel accuracy. Therefore, a reference picture with non-integer pixel precision is generated using a filter, and motion estimation is performed on the generated reference picture.

  In the forward prediction step S307, MV2f is detected by a two-step motion vector search. Specifically, the encoding cost is selected from 9 points including ½ pixel (or ¼ pixel) in the vicinity of 8 surroundings and MV0f in the center with MV0f detected in the full-pel prediction step S300 as the center. The minimum MV1f (not shown) is determined. Further, MV2f that minimizes the coding cost is obtained from 9 points including ½ pixel (or ¼ pixel) in the vicinity of the surrounding 8 and MV1f at the center with MV1f as the center. In the full-pel prediction, it is written that motion estimation with integer pixel accuracy is performed, but the mode selection method of the present invention can be applied even when thinned pixels, for example, one pixel is thinned horizontally.

Also in the backward prediction step S308, MV2b is detected from MV0b detected in the full-pel prediction step S300, as in the forward prediction step S307.
The bidirectional prediction step S309 has a large amount of processing because it refers to two reference pictures. Therefore, prediction using MV2f and MV2b detected in forward prediction step S307 and backward prediction step S308 is performed. Specifically, an average of the reference areas on the reference picture indicated by MV2f and MV2b is used as the predicted image.

Also, in the forward prediction step S307, the backward prediction step S308, and the bidirectional prediction step S309, the respective encoding costs C0, C1, and C2 are derived.
The reference direction selection step S302 selects the direction having the minimum encoding cost among the encoding costs C0 to C2 as the reference direction of the small block and outputs the minimum encoding cost.

  FIG. 29 is a block diagram illustrating a process flow of motion estimation for an image block. The motion estimation processing flow for the image block of FIG. 29 is obtained by dividing M × N ((M, N) = (16, 16), (16, 8) by dividing a 16 × 16 image block by four types of division method candidates. ), (8, 16), (8, 8)) motion estimation step S315 for performing motion estimation for each of the small blocks, and the coding cost of the image block is divided based on the result of motion estimation for each small block. An encoding cost conversion step S316 derived for each method candidate, and a division method selection step S317 for selecting the best division method from the image block coding costs derived for each division method candidate are provided.

  The motion estimation step S315 includes small block motion estimation steps S320 to S323 corresponding to the process flow of motion estimation for the small block described with reference to FIG. Here, in FIG. 29, the processing blocks of the small block motion estimation steps S321 to S323 are connected by a plurality of arrows. For example, in the 16 × 8 small block motion estimation step S321, each processing block is connected by two arrows. This indicates that each process is performed on two small blocks Sb2 and Sb3 that divide a 16 × 16 image block into 16 × 8. Similarly, in the 8 × 16 small block motion estimation step S322, each processing block is connected by two arrows, and in the 8 × 8 small block motion estimation step S323, each processing block is Connected by four arrows. Since the contents of the processing of each processing block are the same as those described with reference to FIG. 28, description thereof is omitted here.

  The encoding cost conversion step S316 includes MB cost conversion steps S325 to S328. In MB cost conversion steps S325 to S328, the encoding costs for the small blocks output in the small block motion estimation steps S320 to 323 are totaled, and the encoding cost of the image block is derived for each division method candidate.

In the division method selection step S317, among the encoding costs for each of the division method candidates derived by the MB cost conversion steps S325 to S328, the division method candidate indicating the minimum encoding cost is selected as the division method to be applied to the image block.
In addition, as shown in FIG. 30, AVC introduces the concept of an image block pair 73 composed of two image blocks 71 and 72, and adaptively performs field prediction and frame prediction using the image block pair 73 as a unit. It is possible to switch. For example, in the case of field prediction, motion estimation is performed for each of the field structure blocks 75 and 76. In the case of frame prediction, motion estimation is performed for each of the frame structure blocks 77 and 78.

  In addition, the coding mode of the image block pair 73 includes two types, that is, two types (field / frame) of the coded picture structure and a coding prediction scheme (intra / inter prediction), for a total of four types. Conventionally, since all these combinations are considered, there is a problem that the amount of processing is large. In particular, the processing load for intra prediction was large.

  Here, conventional coding mode determination will be described. In Codec before AVC, there is no concept of MBpair (large block), and there is field / frame as a type of MB (medium block). And it was common to cover four types of intra / inter and field / frame. As shown in FIG. 31, it consists of a motion estimation step S81 and a picture structure and coding prediction method determination step S82. The estimation step S81 includes first to sixth estimation steps S811 to S816. The first estimation step S811 performs inter prediction on the frame structure block. In the second estimation step S812, intra prediction is performed on the frame structure block. In the third estimation step S813, inter prediction is performed on the field structure top MB. In the fourth estimation step S814, inter prediction is performed on the bottom field of the field structure. The coding cost derived by the third estimation step S813 and the coding cost derived by the fourth estimation step S814 are added together to obtain the coding cost derived by inter prediction on the field structure block. . In the fifth estimation step S815, intra prediction is performed on the top field of the field structure. In the sixth estimation step S816, intra prediction is performed on the bottom field of the field structure. The coding cost derived by the fifth estimation step S815 and the coding cost derived by the sixth estimation step S816 are added together to obtain the coding cost derived by intra prediction on the field structure block. .

The picture structure and coding prediction method determination step S82 selects the smallest of the four types of coding costs.
The above is the prior art, but when such an idea is simply applied to AVC, a process as shown in FIG. 32 is assumed. In FIG. 32, the entire process includes a motion estimation step S81 ′, a coding prediction method determination step S83, and an MB pair picture structure determination step S82 ′.

  The motion estimation step S81 'includes first to eighth estimation steps S811' to S818 '. The first estimation step S811 'performs inter prediction on the frame structure top MB77, and the second estimation step S812' performs intra prediction on the frame structure top MB77. The third estimation step S813 'performs inter prediction on the frame structure bottom MB78, and the fourth estimation step S814' performs intra prediction on the frame structure bottom MB78. The fifth estimation step S815 'performs inter prediction on the field structure top MB75, and the sixth estimation step S816' performs intra prediction on the field structure top MB75. The seventh estimation step S817 'performs inter prediction on the field structure bottom MB76, and the eighth estimation step S818' performs intra prediction on the field structure bottom MB76.

  The encoding prediction method determination step S83 includes first to fourth prediction method determination steps S831 to S834. The first prediction method determination step S831 compares the coding costs of the first estimation step S811 'and the second estimation step S812', and selects intra / inter for the frame structure top MB77. The second prediction method determination step S832 compares the coding costs of the third prediction step S813 'and the fourth prediction step S814', and selects intra / inter for the frame structure bottom MB78. The coding costs of the top MB 77 and the bottom MB 78 of the frame structure for which intra / inter is selected are added up, and the coding cost of the frame structure block pair 77 and 78 is obtained. The third prediction method determination step S833 selects the intra / inter for the field structure top MB75 by comparing the coding costs of the fifth estimation step S815 'and the sixth estimation step S816'. The fourth prediction method determination step S834 compares the encoding costs of the seventh estimation step S817 'and the eighth estimation step S818', and selects intra / inter for the field structure bottom MB76. The coding costs of the top MB 75 and the bottom MB 76 of the field structure for which intra / inter is selected are summed, and the coding cost of the field structure block pair 75 and 76 is obtained.

The picture structure determination step S82 ′ compares the coding cost of the frame structure block pair 77 and 78 with the coding cost of the field structure block pair 75 and 76, and determines the field / frame of the image block pair 73 (71, 72). decide.
In the above processing, since the cost calculation for each field / frame is performed in both intra prediction and inter prediction, the compression rate is improved even in the case of an image in which the compression rate is improved by only one of inter prediction and intra prediction. Therefore, the coded picture structure and the coded prediction method can be determined so as to be the best. However, on the other hand, since the number of intra predictions is large, the processing amount becomes enormous.
Edited by Junichi Miki, “All about MPEG-4”, first edition, Industrial Research Co., Ltd., September 30, 1998, p. 37-58

As described above, in AVC, there are an enormous number of encoding mode candidates for each macroblock (pair), and when searching for an encoding mode with high encoding efficiency covering all candidates, the burden of the processing amount of the encoder is reduced. Becomes larger.
Therefore, the present invention provides an encoding mode determination device, an image encoding device, an encoding mode determination method, and an encoding mode determination program that enable selection of an appropriate encoding mode with a smaller amount of processing. Let it be an issue.

  The encoding mode determining apparatus according to claim 1 is an apparatus that determines at least one encoding mode of an image block from a plurality of candidates, and includes a simple motion estimation unit, an encoding mode selection unit, A motion estimation unit and an encoding mode determination unit are provided. The simple motion estimation unit derives the encoding cost of each encoding mode based on simple motion estimation for a small block that is a partition of an image block obtained by each encoding mode. The coding mode selection unit selects a part of the coding modes from the plurality of coding modes based on the coding cost derived by the simple motion estimation unit. The complex motion estimation unit derives the coding cost of each coding mode based on the complex motion estimation for the small block obtained by at least some of the coding modes. The encoding mode determination unit determines the encoding mode of the image block based on the encoding cost derived by the complex motion estimation unit.

  Here, complex motion estimation refers to motion estimation that is more complicated than simple motion estimation (the same applies hereinafter). For example, complex motion estimation is motion estimation with a more detailed accuracy (eg, non-integer pixel accuracy such as ½ pixel accuracy, ¼ pixel accuracy, etc.), non-integer than simple motion estimation with integer pixel accuracy For example, motion estimation with more detailed accuracy with respect to simple motion estimation of pixels, motion estimation with reference to a more detailed image with respect to simple motion estimation with reference to a reduced image (image with thinned pixel information), and the like.

  The encoding cost is represented, for example, by the sum of the image quality degradation level (absolute difference sum between the small block and the reference picture in motion estimation) and the code amount of motion information (such as a motion vector or a difference motion vector). The coding mode is, for example, a small block division method, a picture reference direction in motion estimation of a small block, a coded picture structure of a small block, or the like.

  In this apparatus, the encoding mode selection unit narrows down encoding modes from the encoding cost obtained by the simple motion estimation unit. Further, the complex motion estimator performs complex motion estimation on the narrow blocks of the encoded mode. Here, the complicated motion estimation has a larger processing amount than the simple motion estimation because, for example, it is necessary to apply a filter. However, in this apparatus, all small motion estimations are performed when determining the coding mode. There is no need to perform complex motion estimation for blocks. For this reason, the number of times of complicated motion estimation can be reduced, and the processing amount for determining the coding mode can be reduced. In addition, since a complicated motion estimation is performed for a necessary small block, it is possible to determine an encoding mode with appropriate encoding efficiency.

  In the encoding mode determination apparatus according to claim 2, in the first aspect, when the simple motion estimation unit derives the encoding cost of each encoding mode, a plurality of simple motion estimation units are provided for each small block obtained by each encoding mode. To calculate a coding cost by performing simple motion estimation in the picture reference direction, and then select a picture reference direction with a low coding cost for each small block, and then select all the small blocks related to the selected picture reference direction. Are summed for each division method candidate to derive the encoding mode coding cost for each division method candidate.

In this apparatus, since the simple motion estimation unit selects a picture reference direction with a low encoding cost for each small block, the combination of small blocks with the lowest encoding cost in the encoding mode for each division method candidate is selected. It becomes possible.
According to a third aspect of the present invention, there is provided a coding mode determining apparatus according to the first aspect, wherein the simple motion estimation unit includes a plurality of simple motion estimation units for each small block obtained by each coding mode when deriving the coding cost of each coding mode. The encoding cost is calculated by performing simple motion estimation in the picture reference direction of each picture, and then the coding cost for each picture reference direction of the small block is converted into image block units, and each picture reference direction of each division candidate is calculated. The encoding cost of each encoding mode is derived.

In this apparatus, since the simple motion estimation unit derives the encoding mode by converting the encoding cost for each picture reference direction of the small block into the image block unit, the encoding mode of different picture reference directions in one small block is also included. This is a target of the encoding mode selection unit.
The encoding mode determination apparatus according to claim 4, wherein the simple motion estimation in the plurality of picture reference directions of the simple motion estimation unit according to claim 2 or 3 is performed by forward prediction with reference to a temporally forward picture. And backward prediction that refers to the backward picture in time. That is, this device does not perform bi-directional prediction. Note that forward prediction and backward prediction each include multiple predictions that refer to multiple pictures in the same direction (hereinafter the same).

In this apparatus, the simple motion estimation unit performs only forward prediction and backward prediction. Since bidirectional prediction is not performed, the amount of processing can be reduced, and the processing time for simple motion estimation can be shortened.
The encoding mode determination apparatus according to claim 5, wherein the simple motion estimation in the plurality of picture reference directions of the simple motion estimation unit according to claim 2 or 3 is performed by forward prediction with reference to a temporally forward picture. And backward prediction that refers to temporally backward pictures and bidirectional prediction that refers to temporally bidirectional pictures.

Since this apparatus performs bi-directional prediction, it is possible to improve the accuracy of simple motion estimation. For this reason, it becomes possible to select a more appropriate encoding mode.
The coding mode determination apparatus according to claim 6, wherein the simple motion estimation in the plurality of picture reference directions of the simple motion estimation unit according to claim 2 or 3 is performed by forward prediction with reference to a temporally forward picture. And backward prediction with reference to the backward picture in time. The simple motion estimator derives the coding cost when bi-directional prediction with reference to bi-directional pictures is performed based on forward prediction and backward prediction. For example, when the encoding cost of forward prediction and the encoding cost of backward prediction are close to each other, the encoding cost of bi-directional prediction is slightly smaller than the value of the smaller encoding cost. It is estimated that it is a value.

  In this apparatus, since the prediction result of bidirectional prediction is estimated, the simple motion estimation unit does not need to perform bidirectional prediction, and the processing amount can be reduced. In addition, by reflecting the prediction result on the coding cost by the simple motion estimation unit, it is possible to easily obtain the same effect as when bi-directional prediction is performed. For this reason, encoding efficiency can be easily improved.

  The coding mode determining apparatus according to claim 7, wherein the complex motion estimator is a picture for complex motion estimation based on simple motion estimation in the simple motion estimator. Determine the reference direction. The complex motion estimation unit performs motion estimation with reference to the determined picture in the reference direction. That is, even when forward prediction or backward prediction can be performed, it is not necessary to always perform motion estimation in all directions.

In this apparatus, it is possible to execute complicated motion estimation with reference to a necessary reference direction. For this reason, the amount of processing of complicated motion estimation can be reduced, and the processing time of complicated motion estimation can be shortened.
The coding mode determining apparatus according to claim 8, wherein the complex motion estimation unit according to claim 7 is a coding result of forward prediction and backward prediction as a result of simple motion estimation for a small block in the simple motion estimation unit. If both are substantially the same, both are selected, and if they are different, only the one with the lower coding cost is selected.

  In this apparatus, when the encoding costs of forward prediction and backward prediction are substantially the same, both can be selected and further bi-directional prediction can be performed. If they are different, only the one with the lower coding cost is selected. This is because when one coding cost is large, it cannot be expected that the coding cost is reduced by bidirectional prediction.

  The coding mode determining apparatus according to claim 9, wherein the complex motion estimator is configured to perform partial coding based on simple motion estimation for a small block in the simple motion estimator. At least some coding modes are further selected from the modes. The complex motion estimation unit selects at least some of the coding modes from among some of the coding modes based on simple motion estimation for the small block.

  In this apparatus, it is not necessary to perform complicated motion estimation for all of some of the encoding modes selected from different encoding modes, and the processing amount can be reduced. It is also possible to select at least some of the coding modes among some of the coding modes so as to keep the processing amount constant.

In the coding mode determination device according to claim 10, in claim 9, the complex motion estimation unit selects each coding mode in ascending order of coding cost, and the coding cost of the selected coding mode is determined. The selection is aborted immediately before the sum exceeds the processing allowance.
In this apparatus, the complex motion estimation unit may not select all of the encoding modes selected by the encoding mode selection unit, but even in that case, an encoding mode with a low encoding cost is selected. Less is.

In the encoding mode determination apparatus according to claim 11, in any one of claims 1 to 10, the simple motion estimation unit or the complex motion estimation unit may perform simple motion estimation or a motion estimation process so that a processing amount of the motion estimation process is maintained substantially constant. Change the motion estimation method in complex motion estimation.
For example, the encoding mode determination apparatus changes the motion estimation method according to the image attribute of an image configured by image blocks. Here, the image attributes include, for example, the image size, the image encoding method (picture type [I picture, P picture, B picture], etc.), the image format (scanning method [progressive, interlaced], color difference). Format) and the amount of motion of the image.

The motion estimation method includes, for example, the number and direction of referenced pictures, a variation in partition size for performing motion estimation, a motion search range, and the like.
In this apparatus, processing with an appropriate processing amount is realized, and the mobility of the apparatus is improved. As an additional effect, more appropriate motion estimation can be performed.

The encoding mode determining apparatus according to claim 12, wherein the simple motion estimation is integer pixel precision motion estimation, and the complex motion estimation is non-integer pixel precision motion estimation. .
In this apparatus, the encoding mode selection unit narrows down encoding modes from the encoding cost obtained by the simple motion estimation unit. Further, the complex motion estimator performs motion estimation with non-integer pixel accuracy for the small blocks in the narrowed encoding mode. Here, the motion estimation with non-integer pixel accuracy needs to apply a filter, and the processing amount is larger than the motion estimation with integer pixel accuracy. However, in this apparatus, when determining the coding mode, There is no need to perform motion estimation with non-integer pixel accuracy. For this reason, the number of times of motion estimation with non-integer pixel precision can be reduced, and the processing amount for determining the coding mode can be reduced. In addition, since motion estimation with non-integer pixel accuracy is performed for the necessary small blocks, it is possible to determine an encoding mode with appropriate encoding efficiency.

An integrated circuit according to a thirteenth aspect includes the coding mode determining apparatus according to any one of the first to twelfth aspects.
In this integrated circuit, it is possible to obtain the same effect as that of the coding mode determining apparatus according to any one of claims 1 to 12.

An image coding apparatus according to a fourteenth aspect includes the coding mode determining apparatus according to any one of the first to twelfth aspects, and a coding apparatus. The encoding device encodes the image block based on the encoding mode of the image block determined by the encoding mode determination device.
In this image encoding apparatus, it is not necessary to perform complicated motion estimation for all partitions when determining the encoding mode. For this reason, the number of times of complicated motion estimation can be reduced, and the processing amount for determining the coding mode can be reduced. In addition, since complicated motion estimation is performed on the necessary partitions, it is possible to determine an encoding mode with appropriate encoding efficiency and perform encoding.

An integrated circuit according to a fifteenth aspect includes the image encoding device according to the fourteenth aspect.
In this integrated circuit, the same effect as that of the image encoding device according to claim 14 can be obtained.
The coding mode determination device according to claim 16, wherein the coding mode determination device determines a coding mode of an image block, and includes an inter prediction unit, a coded picture structure determination unit, an intra prediction unit, and a coding prediction method determination. Department. The inter prediction unit performs inter prediction on each block of the field structure block and the frame structure block of the image block, and derives an encoding cost. The encoded picture structure determining unit determines the encoded picture structure of the image block based on the encoding cost by the inter prediction unit. The intra prediction unit performs intra prediction on each block having the determined encoded picture structure, and derives an encoding cost. The encoding prediction method determination unit determines the encoding prediction method for each block of the image block having the determined encoded picture structure by comparing the encoding cost based on inter prediction and the encoding cost based on intra prediction. Here, the encoded picture structure is a picture structure when an image block is encoded, and means a field structure or a frame structure. The encoding prediction method means inter prediction or intra prediction when an image block is encoded.

  Here, the field structure block includes, for example, a block constituted by a set of odd lines of an image block and a block constituted by a set of even lines (hereinafter the same). The frame structure block is composed of, for example, blocks that sequentially include image block lines (hereinafter the same).

  In this apparatus, since the intra prediction unit performs intra prediction only for each block of the encoded picture structure determined by the encoded picture structure determination unit, the intra prediction unit performs intra prediction for all of the field structure block and the frame structure block. There is no need to do it. Since the number of intra predictions with a high processing load can be reduced in this way, the processing load for determining the coding prediction method for an image block can be reduced.

  The coding mode determination apparatus according to claim 17, wherein the inter prediction unit derives the coding cost of the frame structure block by summing up the coding costs of the blocks of the frame structure block, and calculates the field structure. The coding cost of the field structure block is derived by summing up the coding cost of each block of the block.

In this apparatus, the inter prediction unit derives the coding cost for each picture structure by deriving and summing the coding costs of each block for each picture structure.
In an encoding mode determining apparatus according to an eighteenth aspect, in the seventeenth aspect, the intra prediction unit performs intra prediction on each block having the determined encoded picture structure to derive an encoding cost. The coding prediction method determination unit compares the coding cost derived by the inter prediction unit and the coding cost derived by the intra prediction unit for each block having the determined coded picture structure, and The encoding prediction method is determined.

  In this apparatus, since the intra prediction unit performs intra prediction for each block having the determined coded picture structure to derive the coding cost, it is necessary to perform intra prediction for all of the field structure block and the frame structure block. Absent. Since the number of intra predictions with a high processing load can be reduced in this way, the processing load for determining the coding prediction method of the image block can be reduced, and further the processing amount of the entire encoding device can be reduced. It becomes.

In the encoding mode determining apparatus according to a nineteenth aspect, in any one of the sixteenth to eighteenth aspects, the image block is a block pair composed of two square blocks.
In this apparatus, the block pair is composed of two square blocks, and each of the field structure block and the frame structure block can be processed as a square block.

An integrated circuit according to a twentieth aspect includes the coding mode determining apparatus according to any one of the twelfth to nineteenth aspects.
In this integrated circuit, it is possible to obtain the same effect as that of the coding mode determining apparatus according to any one of claims 16 to 19.

An image encoding device according to claim 21 is based on the encoding mode determining device according to any of claims 16 to 19 and an image block encoding mode determined by the encoding mode determining device. And a coding device that performs coding of the above.
In this apparatus, since the intra prediction unit performs intra prediction only for the block having the picture structure determined by the picture structure determining unit, the intra prediction unit does not need to perform intra prediction for all of the field structure block and the frame structure block. Since the number of intra predictions with a high processing load can be reduced in this way, the processing load for determining the coding prediction method for an image block can be reduced.

An integrated circuit according to a twenty-second aspect includes the image encoding device according to the twenty-first aspect.
In this integrated circuit, the same effect as that of the image encoding device according to claim 21 can be obtained.
A coding mode determination apparatus according to a twenty-third aspect is an apparatus for determining a coding mode of an image block, and includes a simple motion estimation unit and a coded picture structure determination unit. The simple motion estimation unit derives an encoding cost by simple motion estimation for each of the field structure block and the frame structure block of the image block. The encoded picture structure determining unit determines the encoded picture structure by comparing the encoding costs of the field structure block and the frame structure block of the image block based on the encoding cost of the simple motion estimation unit.

In this apparatus, an encoding mode (specifically, an encoded picture structure) of an image block is determined based on simple motion estimation. For this reason, it is possible to reduce the processing amount for determining the encoding mode.
In the coding mode determination apparatus according to Claim 24, in Claim 23, the simple motion estimation unit performs simple inter prediction and simple intra prediction on each block, and then performs simple inter prediction encoding. Compare the cost and the coding cost of simple intra prediction, select either simple inter prediction or simple intra prediction for each block, and then add the block coding cost for each picture structure to the frame structure Deriving coding costs for blocks and field structure blocks.

  In this apparatus, since the simple motion estimation unit derives the coding cost of the frame structure block and the field structure block using inter prediction and intra prediction, an image block whose compression rate is improved by either inter prediction or intra prediction is determined. Even in this case, it is possible to determine a coded picture structure that provides the best compression rate.

In the coding mode determining apparatus according to a twenty-fifth aspect, in the twenty-fourth aspect, the simple inter prediction is inter prediction with integer pixel accuracy.
In this apparatus, the simple motion estimation unit can perform inter prediction with integer pixel accuracy and simple intra prediction.

According to a twenty-sixth aspect of the present invention, in any one of the twenty-third to twenty-fifth aspects, the image block is a block pair composed of two square blocks.
In this apparatus, the block pair is composed of two square blocks, and each of the field structure block and the frame structure block can be processed as a square block.

An integrated circuit according to a twenty-seventh aspect includes the coding mode determining device according to any one of the twenty-third to twenty-sixth aspects.
With this integrated circuit, an effect similar to that of the coding mode determining apparatus according to any one of claims 23 to 26 can be obtained.

  An image encoding device according to a twenty-eighth aspect is complex with respect to the coding mode determining device according to any one of the twenty-third to twenty-sixth aspects and an image block having a coded picture structure determined by the coding mode determining device. A complex motion estimation unit that performs accurate motion estimation and an encoding unit that encodes an image block based on a prediction result by the complex motion estimation unit.

  In this apparatus, since the image block is encoded by complicated motion estimation, the compression efficiency is improved. In addition, here, since the complicated motion estimation is performed only on the image block having the coded picture structure determined by the coding mode determining apparatus, the number of times of motion estimation more complicated than before can be reduced.

In an image coding apparatus according to a twenty-ninth aspect, in the twenty-eighth aspect, the complex prediction unit performs complex inter prediction or complex intra prediction on each block having the determined coded picture structure.
In this apparatus, the compression efficiency can be improved even for an image block whose compression rate is improved by either inter prediction or intra prediction.

In an image encoding device according to a thirty-third aspect, in the twenty-ninth aspect, the complicated inter prediction is inter prediction with non-integer pixel accuracy.
In this apparatus, it is possible to perform complex inter prediction using inter prediction with non-integer pixel accuracy.

An integrated circuit according to a thirty-first aspect includes the image encoding device according to any one of the twenty-eighth to thirty-third aspects.
With this integrated circuit, it is possible to obtain the same effect as that of the image encoding device according to any one of claims 28 to 30.

  The encoding mode determination method according to claim 32, wherein the encoding mode determination method determines an encoding mode of an image block from at least one of a plurality of candidates, and includes a simple motion estimation step and an encoding mode selection. A step, a complex motion estimation step, and a coding mode determination step. The simple motion estimation step derives the coding cost of each coding mode based on simple motion estimation for a small block which is a partition of an image block obtained by each coding mode. The encoding mode selection step selects some encoding modes from the plurality of encoding modes based on the encoding cost derived by the simple motion estimation step. The complex motion estimation step derives the coding cost of each coding mode based on the complex motion estimation for the small block obtained by at least some of the coding modes. The encoding mode determination step determines the encoding mode of the image block based on the encoding cost derived by the complex motion estimation step.

  In this method, the encoding mode selection step narrows down the encoding mode from the encoding cost obtained by the simple motion estimation step. Further, the complicated motion estimation step performs a complicated motion estimation on the narrow blocks of the narrowed coding mode. Here, the complicated motion estimation has a larger processing amount than the simple motion estimation because, for example, it is necessary to apply a filter. However, in this method, all of the small motion estimations are performed when determining the coding mode. There is no need to perform complex motion estimation for blocks. For this reason, the number of times of complicated motion estimation can be reduced, and the processing amount for determining the coding mode can be reduced. In addition, since a complicated motion estimation is performed for a necessary small block, it is possible to determine an encoding mode with appropriate encoding efficiency.

  The encoding mode determination method according to claim 33, wherein the encoding mode determination method determines an encoding mode of an image block, and includes an inter prediction step, an encoded picture structure determination step, an intra prediction step, A prediction method determining step. In the inter prediction step, inter prediction is performed on each of the field structure block and the frame structure block of the image block to derive the coding cost. The encoded picture structure determination step determines the encoded picture structure of the image block based on the encoding cost of the inter prediction step. In the intra prediction step, intra prediction is performed on each block having the determined encoded picture structure to derive an encoding cost. The encoding prediction scheme determination step determines the encoding prediction scheme for each block of the image block having the determined encoded picture structure by comparing the encoding cost by inter prediction and the encoding cost by intra prediction.

  In this method, since the intra prediction step performs intra prediction only for each block of the coded picture structure determined by the coded picture structure determining step, the intra prediction step performs intra prediction for all of the field structure block and the frame structure block. There is no need to do it. Since the number of intra predictions with a high processing load can be reduced in this way, the processing load for determining the coding prediction method for an image block can be reduced.

  An encoding mode determination method according to a thirty-fourth aspect is an encoding mode determination method for determining an encoding mode of an image block, and includes a simple motion estimation step and an encoded picture structure determination step. The simple motion estimation step derives an encoding cost by simple motion estimation for each of the field structure block and the frame structure block of the image block. The coded picture structure determination step compares the coding costs of the field structure block and the frame structure block of the image block based on the coding cost of the simple motion estimation step, and determines the coded picture structure.

In this method, a coding mode (specifically, a coded picture structure) of an image block is determined based on simple motion estimation. For this reason, it is possible to reduce the processing amount for determining the encoding mode.
An encoding mode determination program according to a thirty-fifth aspect causes a computer to perform the following method. The encoding mode determination method is an encoding mode determination method that determines at least one encoding mode of an image block from a plurality of candidates, and includes a simple motion estimation step, an encoding mode selection step, and a complex motion estimation And a coding mode determination step. The simple motion estimation step derives the coding cost of each coding mode based on simple motion estimation for a small block which is a partition of an image block obtained by each coding mode. The encoding mode selection step selects some encoding modes from the plurality of encoding modes based on the encoding cost derived by the simple motion estimation step. The complex motion estimation step derives the coding cost of each coding mode based on the complex motion estimation for the small block obtained by at least some of the coding modes. The encoding mode determination step determines the encoding mode of the image block based on the encoding cost derived by the complex motion estimation step.

  In this program, the encoding mode selection step narrows down the encoding mode from the encoding cost obtained by the simple motion estimation step. Further, the complicated motion estimation step performs a complicated motion estimation on the narrow blocks of the narrowed coding mode. Here, the complicated motion estimation has a larger processing amount than the simple motion estimation because, for example, it is necessary to apply a filter. However, in this program, all small motion estimations are performed when determining the encoding mode. There is no need to perform complex motion estimation for blocks. For this reason, the number of times of complicated motion estimation can be reduced, and the processing amount for determining the coding mode can be reduced. In addition, since a complicated motion estimation is performed for a necessary small block, it is possible to determine an encoding mode with appropriate encoding efficiency.

  A coding mode determination program according to a thirty-sixth aspect causes a computer to perform the following method. The encoding mode determination method is an encoding mode determination method for determining an encoding mode of an image block, and includes an inter prediction step, an encoded picture structure determination step, an intra prediction step, an encoding prediction method determination step, It has. In the inter prediction step, inter prediction is performed on each of the field structure block and the frame structure block of the image block to derive the coding cost. The encoded picture structure determination step determines the encoded picture structure of the image block based on the encoding cost of the inter prediction step. In the intra prediction step, intra prediction is performed on each block having the determined encoded picture structure to derive an encoding cost. The encoding prediction scheme determination step determines the encoding prediction scheme for each block of the image block having the determined encoded picture structure by comparing the encoding cost by inter prediction and the encoding cost by intra prediction.

  In this program, since the intra prediction step performs intra prediction only for each block of the coded picture structure determined by the coded picture structure determining step, the intra prediction step performs intra prediction for all of the field structure block and the frame structure block. There is no need to do it. Since the number of intra predictions with a high processing load can be reduced in this way, the processing load for determining the coding prediction method for an image block can be reduced.

  An encoding mode determination program according to a thirty-seventh aspect causes a computer to perform the following method. The encoding mode determination method is an encoding mode determination method for determining the encoding mode of an image block, and includes a simple motion estimation step and an encoded picture structure determination step. The simple motion estimation step derives an encoding cost by simple motion estimation for each of the field structure block and the frame structure block of the image block. The coded picture structure determination step compares the coding costs of the field structure block and the frame structure block of the image block based on the coding cost of the simple motion estimation step, and determines the coded picture structure.

  In this program, an encoding mode (specifically, an encoded picture structure) of an image block is determined based on simple motion estimation. For this reason, it is possible to reduce the processing amount for determining the encoding mode.

  According to the present invention, it is possible to provide an encoding mode determination device, an image encoding device, an encoding mode determination method, and an encoding mode determination program that enable selection of an appropriate encoding mode with a smaller processing amount.

[First Embodiment]
An encoder as a first embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a block diagram illustrating the structure of an encoder 1 as a first embodiment of the present invention. The encoder 1 is, for example, an image encoding device that MPEG-4 encodes an input image signal 30 and outputs the encoded image signal 31 as an encoded image signal 31, and is provided in a personal computer (PC), a mobile phone, or the like.

<Configuration of encoder 1>
An encoder 1 shown in FIG. 1 includes an intra prediction unit 2 that performs intra prediction of an input image signal 30, an inter prediction unit 3 that performs inter prediction of the input image signal 30, and a switching unit that switches between prediction results of intra prediction and inter prediction. 4, an encoding unit 5 that encodes the output of the switching unit 4 and outputs an encoded image signal 31, and a reference image generation unit 6 that generates a local decode signal 32 of the input image signal 30.

The intra prediction unit 2 performs intra prediction on the input image signal 30 for each image block, and outputs a difference signal from the intra predicted image to the switching unit 4.
The inter prediction unit 3 outputs the inter prediction result to the switching unit 4 with the input image signal 30 as the first input and the local decode signal 32 as the second input. Furthermore, the inter prediction unit 3 outputs information related to encoding, such as a motion vector, among the inter prediction results to the encoding unit 5 as a second output.

  The inter prediction unit 3 uses the input image signal 30 as a first input and the local decode signal 32 as a second input, performs motion estimation, and outputs the motion estimation unit 10 as a first input, A predicted image generating unit 11 that outputs the predicted image by using the decoded signal 32 as a second input; and a subtracting unit 12 that uses the input image signal 30 as a first input and the output of the predicted image generating unit 11 as a second input; It is composed of Of the output of the motion estimation unit 10, encoding information such as a motion vector and an encoding mode is also given to an input of a variable length encoding unit 22 described later.

The motion estimation unit 10 mainly includes a full-pel prediction unit 13, a division method candidate selection unit 14, a sub-pel prediction unit 15, and a division method determination unit 16 (the operation will be described later).
The switching unit 4 uses the intra prediction result as the first input and the inter prediction result as the second input, and outputs one of the inputs to the encoding unit 5.

The encoding unit 5 uses the output of the switching unit 4 as a first input, and outputs an encoded image signal 31 through a DCT (Discrete Cosine Transform) unit 20, a quantization unit 21, and a variable length encoding unit 22.
In the reference image creation unit 6, the output of the quantization unit 21 is input to the inverse quantization unit 23, and the output of the inverse quantization unit 23 is given to the first input of the addition unit 25 through the inverse DCT unit 24. . The adding unit 25 uses the output of the predicted image creating unit 11 as a second input, and outputs the addition result to the memory 26. The memory 26 outputs the local decode signal 32 to the second input of the predicted image creation unit 11 and the second input of the motion estimation unit 10.

<Operation of encoder 1>
Next, the operation of the encoder 1 will be described. First, the input image signal 30 is input in units of image blocks that are basic units of encoding processing.
The image block to be intra-coded is intra-predicted by the intra prediction unit 2 using the pixel coefficients of other image blocks in the same picture. The intra-predicted image block is subjected to discrete cosine transform (DCT) in the DCT unit 20, quantized in the quantization unit 21, and variable-length coded in the variable-length coding unit 22.

  On the other hand, the DCT coefficient quantized by the quantizing unit 21 is inversely quantized by the inverse quantizing unit 23, is inversely DCTed by the inverse DCT unit 24, is locally decoded, and is stored in the memory 26 as the local decoded signal 32. . The local decoded signal 32 stored in the memory 26 is used when the image block is inter-coded in the inter prediction unit 3.

The motion estimation unit 10 performs motion estimation on the image block to be inter-coded. Here, the detailed operation of the motion estimation unit 10 will be described later.
The predicted image creation unit 11 creates a predicted image based on the motion estimation result of the motion estimation unit 10 and the local decode signal 32 stored in the memory 26. The subtraction unit 12 obtains a difference image block from the difference between the image block and the created predicted image. The difference image block is subjected to discrete cosine transform in the DCT unit 20 and quantized in the quantization unit 21. The differential image block subjected to discrete cosine transform and quantization is subjected to variable length coding in the variable length coding unit 22 together with the result of motion estimation.

<Operation of Motion Estimation Unit 10>
The motion estimation unit 10 determines an image block coding mode (image block division method, prediction direction, etc.) that minimizes the coding cost of the image block, and derives a motion vector.

  The features of the motion estimation unit 10 will be described with reference to FIG. The motion estimator 10 performs motion estimation with integer pixel accuracy for all the partition sizes of the image block and all the reference pictures at the time of motion estimation (steps S701 to S703). Further, based on the result of motion estimation with integer precision, a candidate for a partition size and a reference picture for reducing the coding cost is selected, and motion estimation with non-integer pixel precision is performed on the selected candidate (step S704). ).

  As a result, it is not necessary to perform motion estimation with non-integer pixel accuracy for all partition sizes and all reference pictures, and it is possible to reduce the processing amount for determining the partition size and reference picture at the time of encoding. It becomes. In addition, since the motion estimation with non-integer pixel accuracy is performed on the selected candidate, it is possible to realize appropriate encoding efficiency.

The operation of the motion estimation unit 10 will be further described with reference to FIG.
FIG. 3 is a block diagram illustrating a processing flow for determining an encoding mode for an image block. The processing flow for determining the coding mode for the image block in FIG. 3 includes a full-pel prediction step S41 executed by the full-pel prediction unit 13, a division method candidate selection step S42 executed by the division method candidate selection unit 14, and a sub-pel prediction. It comprises a sub-pel prediction step S43 executed by the unit 15 and a division method determination step S44 executed by the division method determination unit 16.

The full pel prediction step S41 includes a small block full pel prediction step S45, a prediction direction selection step S46, and an encoding cost derivation step S47.
In the small block full-pel prediction step S45, M × N ((M, N) = (16,16), (16,8), (8, 16), (8, 8)) for each of the small blocks Sb1 to Sb9 (see FIG. 25), motion estimation with integer pixel accuracy is performed to derive the coding cost and motion vector for each small block. Specifically, forward prediction steps S451 to S454 and backward prediction steps S455 to S458 are performed for each of the small blocks Sb1 to Sb9. That is, in the forward prediction steps S451 to S454 and the backward prediction steps S455 to S458, the number of processes corresponding to the number of small blocks divided by the respective division method candidates is performed. In FIG. 3, this number is represented by the number of arrows from the processing block.

  The prediction direction selection step S46 selects a part of the encoding modes from the plurality of encoding modes based on the encoding cost derived by the full-pel prediction step S45. Specifically, the prediction direction selection step S46 compares the encoding cost of the forward prediction steps S451 to S454 with the encoding cost of the backward prediction steps S455 to S458, and reduces the encoding cost for each small block. The prediction direction (picture reference direction) to be selected is selected.

  In the encoding cost deriving step S47, the encoding costs in the prediction direction selected in the prediction direction selecting step S46 are summed for each division method candidate, and the encoding cost in units of image blocks is derived. Here, since the full-pel prediction step S45 selects the picture reference direction with the lowest coding cost for each small block, the combination of the small blocks with the lowest coding cost is possible in the coding mode for each division method candidate. It becomes.

In the division method candidate selection step S42, the coding costs in units of image blocks derived by the coding cost deriving step S47 are compared, and two types of division method candidates with low coding costs are selected.
The sub-pel prediction step S43 performs motion estimation with non-integer pixel accuracy for each of the small blocks divided by the two types of division method candidates selected in the division method candidate selection step S42. Here, the motion estimation with non-integer pixel accuracy is performed in the same manner as the sub-pel prediction step S301 described with reference to FIG. That is, motion estimation with non-integer pixel accuracy is performed for each of the small blocks divided by the two selected division method candidates based on the motion vector derived in the small block full-pel prediction step S45. In sub-pel prediction step S43, forward prediction steps S431 and S434, backward prediction steps S432 and S435, and bidirectional prediction steps S433 and S436 are performed for each of the small blocks. As a result, for each small block, encoding costs for three types of prediction directions are derived. The forward prediction steps S431 and S434, the backward prediction steps S432 and S435, and the bidirectional prediction steps S433 and S436 are performed in accordance with the number of small blocks divided by the two selected division method candidates. Is being processed.

  In the division method determination step S44, the prediction direction for each small block is determined from the encoding cost that is the minimum for each small block divided by the two types of division method candidates selected in the division method candidate selection step S42. Deriving the coding cost for each image block. Further, the derived encoding cost for each image block is compared for two types of division method candidates, and the division method candidate having the minimum encoding cost is determined as the image block division method. At the same time, a motion vector for a small block is obtained.

  With reference to FIG. 4, a detailed description will be given of the processing in the full-pel prediction step S41 and the division method candidate selection step S42. As described above, the full-pel prediction step S41 includes a small block full-pel prediction step S45, a prediction direction selection step S46, and an encoding cost derivation step S47.

  The small block full-pel prediction step S45 performs forward prediction (denoted as fw in FIG. 4) and backward prediction (denoted as bw in FIG. 4) with integer pixel accuracy for all of the small blocks Sb1 to Sb9. Then, the encoding cost for each reference direction is derived. FIG. 4 illustrates the respective encoding costs. For example, in the small block Sb2, the encoding cost for forward prediction is (21), and the encoding cost for backward prediction is (22).

The prediction direction selection step S46 compares the encoding costs of the forward prediction and the backward prediction for each small block, and selects a prediction direction with a smaller encoding cost. For example, in the small block Sb2, forward prediction is selected.
In the coding cost deriving step S47, the coding cost for each image block is derived from the coding cost for each small block selected in the prediction direction selecting step S46. For example, for the 16 × 8 division method, the forward prediction is selected in the small block Sb2, and the backward prediction is selected in the small block Sb3. Therefore, the coding cost for each 16 × 16 image block is (41).

  In the division method candidate selection step S42, the coding costs in units of image blocks derived in the full-pel prediction step S41 are compared for each division method candidate, and two types of division method candidates with low coding costs are selected. In FIG. 4, a 16 × 16 division method (encoding cost (40)) and a 16 × 8 division method (encoding cost (41)) are selected as division method candidates.

<Effect of encoder 1>
In the encoder 1, the division method candidate selection step S42 narrows down the division method candidates from the encoding cost obtained in the full-pel prediction step S41. Further, the sub-pel prediction step S43 performs sub-pel prediction for the narrowed-down candidate blocks. Here, the sub-pel prediction needs to apply a filter and has a larger processing amount than the full-pel prediction, but in this apparatus, it is necessary to perform sub-pel prediction for all the small blocks Sb1 to Sb9 when determining the encoding mode. No. For this reason, the number of sub-pel predictions can be reduced, and the processing amount for determining the coding mode can be reduced. In addition, since sub-pel prediction is performed for the necessary small blocks, it is possible to determine an encoding mode with appropriate encoding efficiency.

<Modification of Encoder 1>
The present invention is not limited to the above-described embodiment, and various changes or modifications can be made without departing from the scope of the present invention.
(1) Modification of full-pel prediction unit 13 (1-1)
In the above-described embodiment, the full-pel prediction unit 13 that executes the full-pel prediction step S41 is described as executing the forward prediction steps S451 to S454 and the backward prediction steps S455 to S458 for each of the small blocks Sb1 to Sb9 ( Hereinafter, it is referred to as a first full-pel prediction method). In this case, since bidirectional prediction is not performed, the amount of processing can be reduced, and the processing time of full-pel prediction can be shortened.

  Here, the full-pel prediction step S41 may further perform bi-directional prediction to derive an encoding cost (hereinafter referred to as a second full-pel prediction method). In this case, since bi-directional prediction is performed, it is possible to improve the accuracy of full-pel prediction. For this reason, it becomes possible to select a more appropriate encoding mode. Moreover, the encoding cost when bi-directional prediction is executed may be estimated from the encoding costs derived by the forward prediction steps S451 to S454 and the backward prediction steps S455 to S458 (hereinafter referred to as “third”). Full-pel prediction method). In this apparatus, since the prediction result of bidirectional prediction is estimated, the full-pel prediction unit 13 does not need to perform bidirectional prediction, and the processing amount can be reduced. In addition, by reflecting the prediction result on the encoding cost by the full-pel prediction unit 13, it is possible to easily obtain the same effect as when bi-directional prediction is performed. For this reason, encoding efficiency can be easily improved.

The first to third full-pel prediction methods executed for the 8 × 16 small block Sb4 and the small block Sb5 (see FIG. 25) obtained by dividing the 16 × 16 image block into two will be described with reference to FIG. To do.
FIG. 5A is a processing flow for explaining the first full-pel prediction method. In the first full-pel prediction method, the forward prediction step S453 and the backward prediction step S457 for the small blocks Sb4 and Sb5 are performed, and the encoding costs C4f and C5f by the forward prediction step S453 for the small blocks Sb4 and Sb5 are: The encoding costs C4b and C5b in the backward prediction step S457 for the small blocks Sb4 and Sb5 are derived. The derived encoding costs C4f, C5f, C4b, and C5b are compared for each small block in a small block prediction method selection step S463 (see FIG. 3), which is a prediction method selection step S46 for each small block, and are encoded with a small encoding. A prediction direction having a cost is selected. More specifically, the coding cost C4f and C4b for the small block Sb4 are compared in the comparison step S463a in the small block prediction method selection step S463, and the coding cost C5f for the small block Sb5 is compared in the comparison step S463b. And C5b are compared, and for each small block, the prediction direction with the lower coding cost is selected.

  FIG. 5B is a processing flow for explaining the second full-pel prediction method. The difference from the first full-pel prediction method is that a bidirectional prediction step S459 is performed. For example, for the small block Sb4, prediction using the motion vectors MV4f and MV4b detected in the forward prediction step S453 and the backward prediction step S457 is performed. Specifically, the reference area on the reference picture indicated by MV4f and MV4b is averaged to be a predicted image, and the encoding cost C4g of the bidirectional prediction step S459 is derived. Similarly, for the small block Sb5, the encoding cost C5g is derived using the MV 5f and the MV 5b.

  The derived encoding costs C4g and C5g of the bidirectional prediction step S459 are the same as those of the forward prediction step S453 and the backward prediction step S457 in the small block prediction method selection step S465 as a modification of the small block prediction method selection step S463. It is compared with the coding costs C4f, C5f, C4b, C5b. Specifically, in the comparison step S465a, the encoding costs C4f, C4b, and C4g for the small block Sb4 are compared, and in the comparison step S465b, the encoding costs C5f, C5b, and C5g for the small block Sb5 are compared. As a result, the prediction direction with the lowest coding cost for each small block is selected.

In the second full-pel prediction method, more accurate motion detection is possible for small blocks, and an improvement in coding efficiency can be expected.
FIG. 5C is a processing flow for explaining the third full-pel prediction method. The difference from the first full-pel prediction method is that the encoding cost estimation step S468 of the encoding cost when bi-directional prediction is performed is performed.

  The coding cost estimation step S468 is an estimated code that is an estimated value of the coding cost when bi-directional prediction is performed from the coding costs C4f, C5f, C4b, and C5b of the forward prediction step S453 and the backward prediction step S457. Derivation costs C4h and C5h are derived. Specifically, when the encoding costs C4f and C4b for the small block Sb4 are “near values”, the estimated encoding cost C4h is slightly smaller than the smaller one of the encoding costs C4f and C4b, for example, It is estimated that 90% of the smaller value.

  Here, the “near value” is, for example, the encoding cost C4f when the formula abs ([C4f] − [C4b]) * K <abs (abs ([C4f]) + abs ([C4b])) is true. C4b is determined to be “near value.” Here, [C4f] and [C4b] indicate values of the encoding costs C4f and C4b, and K is a predetermined constant.

  Further, the estimated encoding costs C4h and C5h are compared with the encoding costs C4f, C5f, C4b, and C5b in comparison steps S466a and S466b as modifications of the comparison steps S463a and S463b. Specifically, in the comparison step S466a, the estimated coding cost C4h and the coding costs C4f and C4b are compared, and in the comparison step S466b, the estimated coding cost C5h and the coding costs C5f and C5b are compared. . As a result, the prediction direction with the lowest coding cost for each small block is selected.

In the third full-pel prediction method, it is not necessary to perform bi-directional prediction, and the processing amount can be reduced. In addition, it is possible to easily obtain the same effect as when bidirectional prediction is performed. For this reason, encoding efficiency can be easily improved.
(1-2)
In the above embodiment, the small block full-pel prediction step S45 and the prediction direction selection step S46 may be sequentially processed or may be processed in parallel.

The processing order of the small block full-pel prediction step S45 and the prediction direction selection step S46 for the 16 × 16 small block Sb1 (see FIG. 25) obtained by dividing the 16 × 16 image block into one will be described with reference to FIG. To do.
FIG. 6A shows a processing flow when the small block full-pel prediction step S45 and the prediction direction selection step S46 are sequentially processed. Detailed description has been made in the above embodiment with reference to FIG.

  FIG. 6B shows a processing flow when the small block full pel prediction step S45 and the prediction direction selection step S46 are processed in parallel. Here, the forward prediction step S451, the backward prediction step S455, and the respective encoding costs are compared in parallel. Specifically, two reference pictures are stored in the memory 26 of the encoder 1 for the forward prediction step S451 and the backward prediction step S455, and motion estimation and encoding cost calculation are executed in parallel. Compare the best values of the first few cost calculations and abort the motion estimation in the reference direction with the highest coding cost.

  Usually, in motion estimation, encoding cost calculation is performed for a probable search start position and its surroundings, and the best candidate is selected. At this time, the encoding cost is calculated 10 to 1000 times. In the case of the present invention, the motion estimation process unnecessary for selection of the prediction direction can be stopped halfway, and the processing amount of full-pel prediction can be reduced.

Here, in order to make the allocation amount of the memory 26 the same as when one reference picture is stored, motion estimation may be performed using two reference pictures obtained by thinning out pixel information.
Further, the small block full pel prediction step S45 and the prediction direction selection step S46 may be performed not only in parallel for each small block but also in parallel for all small blocks.

Referring to FIG. 7, the processing order of small block full-pel prediction step S45 and prediction direction selection step S46 for all small blocks Sb1 to Sb9 when a 16 × 16 image block is divided by four types of division methods is used. explain.
In FIG. 7, the small block full pel prediction step S45 and the prediction direction selection step S46 are executed in parallel for all the small blocks Sb1 to Sb9. Further, as described with reference to FIG. 6B, the motion estimation process in the unnecessary prediction direction is terminated for each small block. Furthermore, the motion estimation process for a small block whose encoding cost does not decrease is terminated by comparing the encoding costs for each small block.

  That is, for each small block, not only the motion estimation process in the unnecessary prediction direction is aborted, but also the motion estimation process to the small block unnecessary for selection of the division method can be aborted. Thereby, unnecessary motion estimation processing can be further reduced, and the processing amount of full-pel prediction can be further reduced.

(1-3)
In the above embodiment, the small block full-pel prediction step S45 and the prediction direction selection step S46 may be sequentially processed for each small block.
The processing order of the small block full-pel prediction step S45 and the prediction direction selection step S46 for 16 × 8 small blocks Sb2 and Sb3 (see FIG. 25) obtained by dividing a 16 × 16 image block into two using FIG. Will be described.

  When the small block full-pel prediction step S45 and the prediction direction selection step S46 are sequentially processed (see FIG. 8A), the forward prediction step S452, the backward prediction step S456, and the prediction method selection step for each small block The small block prediction method selection step S463, which is S46, is performed in the following order. Forward prediction step S452 ′ for small block Sb2, backward prediction step S456 ′ for small block Sb2, forward prediction step S452 ″ for small block Sb3, backward prediction step S456 ″ for small block Sb3, small block The order is the small block prediction method selection step S462 ′, which is the prediction method selection step S46 for Sb2, and the small block prediction method selection step S462 ″ for the small block Sb3.

  On the other hand, when the small block full-pel prediction step S45 and the prediction direction selection step S46 are sequentially processed for each small block (see FIG. 8B), first, processing for the small block Sb2 is performed. Thereafter, the process for the small block Sb3 is performed. That is, first, a forward prediction step S452 ', a backward prediction step S456', and a small block prediction method selection step S462 'for the small block Sb2 are performed. Thereafter, the forward prediction step S452 ", backward prediction step S456", and small block prediction method selection step S462 "for the small block Sb3 are performed. At this time, the processing for each small block is (1-2 For example, the forward prediction step S452 ", the backward prediction step S456", and the small block prediction method selection step S462 "for the small block 1 may be executed in parallel. .

(1-4)
A modification of the full-pel prediction unit 13 will be described with reference to FIGS. 9 and 10. FIG. 9 is a block diagram illustrating a processing flow for determining an encoding mode for an image block. The processing flow for determining the coding mode for the image block in FIG. 9 includes a full-pel prediction step S41 ′ executed by the full-pel prediction unit 13 and a division method candidate selection step S42 ′ executed by the division method candidate selection unit 14. I have.

The full-pel prediction step S41 ′ includes a small block full-pel prediction step S45 and an encoding cost conversion step S66.
In the small block full-pel prediction step S45, M × N ((M, N) = (16,16), (16,8), (8, 16), (8, 8)) for each of the small blocks Sb1 to Sb9 (see FIG. 25), motion estimation with integer pixel accuracy is performed to derive the coding cost and motion vector for each small block. Specifically, forward prediction steps S451 to S454 and backward prediction steps S455 to S458 are performed on each of the small blocks Sb1 to Sb9. That is, in forward prediction steps S451 to S454 and backward prediction steps S455 to S458, processing is performed a number of times according to the number of small blocks divided by the respective division method candidates. In FIG. 9, this number is represented by the number of arrows from the processing block.

  In the encoding cost conversion step S66, the encoding costs in the forward prediction steps S451 to S454 and the encoding costs in the backward prediction steps S455 to S458 are individually converted into image block units. Specifically, the converted value converted into image block units is a value obtained by multiplying the encoding cost of each prediction code of each small block obtained by the small block full-pel prediction step S45 by the number of divisions of the partition. is there.

In the division method candidate selection step S42 ′, the coding costs in units of image blocks derived by the coding cost deriving step S47 are compared, and two types of division method candidates with low coding costs are selected.
With reference to FIG. 10, a detailed description will be given of the processing of the full-pel prediction step S41 ′ and the division method candidate selection step S42 ′. As described above, the full-pel prediction step S41 ′ includes a small block full-pel prediction step S45 and an encoding cost conversion step S66.

  The small block full-pel prediction step S45 includes forward prediction (denoted as fw in FIG. 10) and backward prediction (denoted as bw in FIG. 10) with integer pixel accuracy for all of the small blocks Sb1 to Sb9. Bidirectional prediction (bid in FIG. 10) is performed, and the coding cost for each reference direction is derived. FIG. 10 illustrates each encoding cost. For example, in the small block Sb1, the encoding cost for forward prediction is (40), and the encoding cost for backward prediction is (70).

  In the encoding cost conversion step S66, the encoding costs in the forward prediction steps S451 to S454 and the encoding costs in the backward prediction steps S455 to S458 are individually converted into the encoding costs for each image block. . Specifically, the coding cost of fw, bw, and bid of Sb1 is multiplied by 1, the coding cost of fw, bw, and bid of Sb2 to Sb5 is doubled, and the codes of FW, bw, and bid of Sb6 to Sb9 The cost will be quadrupled.

  In the said embodiment, as shown to Fig.11 (a), the full pel prediction part 13 which performs full pel prediction step S41 'performs forward direction prediction step S451-S454 and back direction with respect to each small block Sb1-Sb9. It has been described that only the prediction steps S455 to S458 are executed (hereinafter referred to as a first full-pel prediction method). Here, the full-pel prediction step S41 'may further perform bi-directional prediction to derive an encoding cost (hereinafter referred to as a second full-pel prediction method). Moreover, the encoding cost when bi-directional prediction is executed may be estimated from the encoding costs derived by the forward prediction steps S451 to S454 and the backward prediction steps S455 to S458 (hereinafter referred to as “third”). Full-pel prediction method).

  In the division method candidate selection step S42 ', the encoding costs for each image block derived in the full-pel prediction step S41' are compared, and two types of division method candidates with low encoding costs are selected. In FIG. 10, fw (encoding cost (40)) of the 16 × 16 division method and bw (encoding cost (70)) of the 16 × 16 division method are selected as division method candidates.

<Effect of encoder 1>
In the encoder 1, in the division method candidate selection step S42 ′, the encoding costs for each image block derived by the full-pel prediction step S41 ′ are compared, and two types of division method candidates with low encoding costs are selected. It is not necessary to perform sub-pel prediction for all the small blocks Sb1 to Sb9. For this reason, the frequency | count of subpel prediction can be reduced and it becomes possible to reduce a processing amount. Further, since sub-pel prediction is performed for the necessary small blocks, it is possible to maintain the coding efficiency.

  In particular, in this embodiment, unlike the previous embodiment, the prediction direction of each division method has not been narrowed down until the division method candidate selection step S42 ′, that is, the encoding cost for each prediction direction of each division method. Is a comparison target in the division method candidate selection step S42 ′. In other words, since the full-pel prediction unit 13 derives the encoding mode by converting the encoding cost for each picture reference direction of the small block into the image block unit, the encoding mode of the different picture reference direction is also divided in one small block. It becomes a comparison object in method candidate selection step S42. Therefore, in the case of the image block of the embodiment shown in FIG. 10, the fw (coding cost (40)) of the 16 × 16 division method, which is the minimum coding cost, and the bw (coding cost) of the 16 × 16 division method. (70)) is selected as two types of division method candidates. When the apparatus of the above embodiment is applied to the image block of this embodiment, bw is abandoned for the 16 × 16 division method in the full-pel prediction step S41, and thus, for example, 16 × 8 division ( sb2 is bid, sb3 is bid, and the coding cost is 77).

  In addition, as shown in FIG.11 (b), you may perform encoding cost conversion step S66 within small block full pel prediction step S45. For example, since the encoding conversion process is a simple calculation such as 2 or 4 times, it may be merged into the small block full-pel prediction step S45. The converted value may be calculated for each search position during the small block full pel prediction step S45, or may be obtained after the small block full pel prediction step S45.

(2) Modification of Division Method Candidate Selection Unit 14 The division method candidates selected by the division method candidate selection step S42 are not limited to two types. Any one of 1 to 3 types of division method candidates may be selected from the four types of division method candidates.
(3) Modification of sub-pel prediction unit 15 (3-1)
In the above embodiment, the sub-pel prediction step S43 performs forward prediction, backward prediction, and bidirectional prediction for each of the small blocks divided by the two types of division method candidates selected in the division method candidate selection step S42. It has been described that sub-pel prediction in three types of prediction directions is performed.

  Here, the sub-pel prediction step S43 determines what is actually performed among the three types of prediction directions in each division candidate based on the motion estimation result of the full-pel prediction step S41, and performs sub-pel prediction only for the determined one. May be. This will be described with reference to FIG.

  First, the full-pel prediction step S41 performs motion estimation with forward and backward integer pixel accuracy for each partition. In the division method candidate selection step S42, a division method candidate is selected based on the motion estimation with integer pixel accuracy in the full-pel prediction step S41. Further, the sub-pel prediction step S43 determines the prediction direction of the sub-pel prediction for the small block divided by the division method candidate selected in the division method candidate selection step S42.

More specifically, the prediction direction is determined by the following three cases.
The first case is a case where the encoding cost for forward prediction and the encoding cost for backward prediction substantially match. In this case, motion estimation with non-integer pixel accuracy is performed for three types of prediction directions: forward prediction, backward prediction, and bidirectional prediction. Further, in this case, motion estimation with non-integer pixel accuracy may be executed only for two types of prediction directions: forward prediction and backward prediction.

  The second case is a case where the encoding cost for forward prediction is smaller than the encoding cost for backward prediction except for the first case. In this case, motion estimation with non-integer pixel accuracy by forward prediction is executed, and motion estimation with non-integer pixel accuracy by backward prediction and bidirectional prediction is not executed.

  The third case is a case where the encoding cost of the forward prediction is larger than the encoding cost of the backward prediction except for the first case. In this case, motion estimation with non-integer pixel accuracy by backward prediction is executed, and motion estimation with non-integer pixel accuracy by forward prediction and bidirectional prediction is not executed.

When the encoding costs of forward prediction and backward prediction are different as in the second and third cases, only the one with the smaller encoding cost is selected. When one of the encoding costs is large, bi-directional prediction is selected. This is because the encoding cost cannot be expected to be small.
As described above, since it is possible to perform motion estimation with non-integer pixel accuracy by referring to the required reference direction by the determination of the above three cases, it is possible to reduce the processing amount of sub-pel prediction, The processing time for prediction can be shortened.

(3-2)
In addition to the determination in (3-1) above, the sub-pel prediction step S43 performs sub-pel prediction on a part of the division method candidates selected in the division method candidate selection step S42. There may be. That is, in this case, some of the division method candidates selected in the division method candidate selection step S42 are not subjected to sub-pel prediction. That is, it is not necessary to perform sub-pel prediction for all of some of the encoding modes selected from the plurality of encoding modes, and the processing amount can be reduced. It is also possible to select at least some of the coding modes among some of the coding modes so as to keep the processing amount constant.

  For example, based on the prediction direction determined in (3-1) above, the required processing amount for each small block to be subjected to sub-pel prediction is estimated. Further, narrow block candidates to be subjected to sub-pel prediction are narrowed down so that the total required processing amount for the entire image block does not exceed the processing margin allocated to the sub-pel prediction of the image block. For this reason, the sub-pel prediction step S43 may not select all of the coding modes (specifically, the division method candidates) selected in the division method candidate selection step S42, but even in that case, the coding cost is low. Since the division method candidates are selected, there are few problems.

  More specifically, the operation processing flow for sub-pel prediction in FIG. 13 will be described. For convenience of explanation, the required processing amount per prediction direction of the 16 × 16 small block Sb1 is [4], and the required processing amounts of the 16 × 8 and 8 × 16 small blocks Sb2 to Sb5 are [2]. , 8 × 8 small blocks Sb6 to Sb9 will be described as [1]. This is because the required processing amount of sub-pel prediction per prediction direction of a small block is proportional to the number of pixels of the small block.

  Processing is performed in units of image blocks (steps S30 to S37). First, the processing amount allocated to the sub-pel prediction of the 16 × 16 image block is set as the processing margin amount (step S30). Next, processing for each candidate division method is performed (steps S31 to S37).

  The processing for each division method candidate is performed in order from the division method candidate selected in the division method candidate selection step S42 in ascending order of encoding cost by full-pel prediction. First, the prediction direction of sub-pel prediction for each small block is selected by the method described in (3-1), and the necessary processing amount of sub-pel prediction for each small block is estimated. Further, the estimated necessary processing amount for each small block is summed up in units of division method candidates, and the necessary processing amount of the entire division method candidates is calculated (step S31).

  For example, when one prediction direction is selected for the 16 × 8 small block Sb2 (for example, the second case or the third case (3-1)), 1 of the small block Sb2 is selected. A value [2] obtained by multiplying the required processing amount [2] per prediction direction by a constant [1] determined from the prediction direction is calculated as the required processing amount of the small block Sb2. When three prediction directions are selected (for example, the first case of (3-1)), the required processing amount per prediction direction [2] of the small block Sb2 is determined from the prediction direction. A value [4] multiplied by the constant [2] is calculated as the required processing amount of the small block Sb2. Here, the reason that the constant determined from the prediction direction when the three prediction directions are selected is [2]. For bidirectional prediction, the motion estimation process is not performed, and the forward prediction and the backward prediction are performed. This is because prediction using the result can be performed (the method described in FIG. 5B or 5C can be used for sub-pel prediction). The necessary processing amount for each small block estimated in this way is summed up in units of division methods, and the necessary processing amount of the division method candidate is calculated.

The calculated required processing amount is compared with the processing margin amount set in step S30, and if the required processing amount is not larger than the processing margin amount, it is determined that there is a processing margin (step S32).
If it is determined that there is a processing margin, sub-pel prediction is performed for each small block in the prediction direction selected in (3-1) (step S33). Further, the difference between the processing margin amount and the required processing amount of the division method candidate is set as the processing margin amount, and processing for the next division method candidate is started.

  If it is determined that there is no processing margin, one prediction direction determined to indicate the minimum coding cost based on full-pel prediction is selected for each small block (step S55), and necessary processing for each small block is performed. The amounts are totaled in units of division method candidates, and the necessary processing amount of the division method candidates is calculated (step S35). For example, for the 16 × 8 small blocks Sb2 and Sb3, the necessary processing amount [2] per prediction direction of the small blocks Sb2 and Sb3 is totaled, and the necessary processing amount of the 16 × 8 division method candidate is [4 ] Is calculated. The calculated required processing amount is compared with the processing margin amount set in step S30. If the required processing amount is smaller than the processing margin amount, it is determined that there is a processing margin (step S36).

  If it is determined that there is a processing margin, sub-pel prediction is performed for one prediction direction that is determined to indicate the minimum coding cost based on full-pel prediction (step S37). Further, the difference between the processing margin amount and the necessary processing amount of the division method candidate calculated in step S35 is set as the processing margin amount (step 34), and processing for the next division method candidate is started.

If it is determined in step S36 that there is no processing margin, sub-pel prediction is not performed and processing of the next image block is started.
(3-2-1)
Next, a first specific example will be described with reference to FIG. In this specific example, in the division method candidate selection step 42, the 16 × 16 division method (encoding cost (40)) is selected as the first candidate, and the 16 × 8 division method (encoding cost (43)) is selected as the second candidate. ) Is selected.

  As shown in FIG. 14, the process is performed in units of image blocks (step S30 to step S37). First, the processing amount allocated to the sub-pel prediction of the 16 × 16 image block is set as the processing margin amount [8] (step S30). Next, processing is performed for each candidate division method (steps S31 to S37).

The processing for each division method candidate is performed in order from the division method candidate selected in the division method candidate selection step S42 in ascending order of encoding cost by full-pel prediction.
Initially, the 16 × 16 division method (encoding cost (40)) is targeted. Specifically, in the 16 × 16 division method, first, the prediction direction of the sub-pel prediction of the small block Sb1 is selected by the method described in (3-1). This case is a second case where the coding cost of the forward prediction fw is smaller than the coding cost of the backward prediction bw. Therefore, motion estimation with non-integer pixel accuracy by forward prediction is executed, and motion estimation with non-integer pixel accuracy by backward prediction and bidirectional prediction is not executed. As a result, the necessary processing amount [4] for sub-pel prediction of the small block Sb1 is estimated. Further, the required processing amount [4] of the 16 × 16 division method is calculated (step S31).

The calculated required processing amount [4] is compared with the processing margin amount [8] set in step S50. Since the required processing amount [4] is not larger than the processing margin amount [8], there is a processing margin. Determination is made (step S32).
In this case, sub-pel prediction of the small block Sb1 with respect to the prediction direction (fw) selected in (3-1) is performed (step S33). Further, the difference between the processing margin [8] and the necessary processing amount [4] of the division method candidate is set as the processing margin [4] (step S34).

  Next, the 16 × 8 division method (encoding cost (42)) is targeted. Specifically, in the 16 × 8 division method, first, the prediction direction of the sub-pel prediction of the small block Sb2 is selected as the second case by the method described in (3-1) (fw), and the small block Sb2 The required processing amount [2] for sub-pel prediction is estimated. As a third case, the prediction direction of the sub-pel prediction of the small block Sb3 is selected (bw), and the necessary processing amount [2] of the sub-pel prediction of the small block Sb3 is estimated. Further, the estimated necessary processing amount [2] of the small block Sb2 and the necessary processing amount [2] of the small block Sb3 are summed to calculate the necessary processing amount [4] of the 16 × 8 division method candidate (step). S31).

The calculated required processing amount [4] is compared with the processing margin amount [4] set in step S34, and the required processing amount [4] is not larger than the processing margin amount [4]. Determination is made (step S32).
In this case, sub-pel prediction of the small block Sb2 for the prediction direction (fw) selected in (3-1) is performed, and further, the sub-pel of the small block Sb3 for the prediction direction (bw) selected in (3-1). Prediction is performed (step S33).

Furthermore, the difference between the processing margin [4] and the necessary processing amount [4] of the division method candidate calculated in step 35 is set as the processing margin (step S34), and the value becomes [0]. Therefore, the process for the next division method candidate is not performed.
(3-2-2)
Next, a second specific example will be described with reference to FIG. In this specific example, in the division method candidate selection step 42, the 16 × 16 division method (encoding cost (40)) is selected as the first candidate, and the 16 × 8 division method (encoding cost (43)) is selected as the second candidate. ) Is selected.

  As shown in FIG. 15, the processing is performed in units of image blocks (steps S30 to S37). First, the processing amount allocated to the sub-pel prediction of the 16 × 16 image block is set as the processing margin amount [8] (step S30). Next, processing for each candidate division method is performed (steps S31 to S37).

The processing for each division method candidate is performed in order from the division method candidate selected in the division method candidate selection step S42 in ascending order of encoding cost by full-pel prediction.
Initially, the 16 × 16 division method (encoding cost (40)) is targeted. Specifically, in the 16 × 16 division method, first, the prediction direction of the sub-pel prediction of the small block Sb1 is selected by the method described in (3-1). In the first case, the encoding cost of the forward prediction fw and the encoding bw cost of the backward prediction are substantially the same. Therefore, motion estimation with non-integer pixel accuracy is executed for two types of prediction directions, ie, forward prediction fw and backward prediction bw. As a result, the necessary processing amount [4] for sub-pel prediction of the forward prediction fw of the small block Sb1 and the necessary processing amount [4] for sub-pel prediction of the forward prediction bw of the small block Sb1 are estimated. The estimated necessary processing amount [4] for each small block Sb1 is summed up in units of division method candidates, and the necessary processing amount [8] of the division method candidates is calculated (step S31).

The calculated required processing amount [8] is compared with the processing margin amount [8] set in step S30, and the required processing amount [8] is not larger than the processing margin amount [8]. Determination is made (step S32).
In this case, sub-pel prediction of the small block Sb1 with respect to the prediction direction (fw) selected in (3-1) is performed, and further, sub-pel prediction of the small block Sb1 with respect to the prediction direction (bw) is performed (step S33).

Furthermore, the difference between the processing margin [8] and the necessary processing amount [8] of the division method candidate calculated in step 55 is set as the processing margin (step S34), and the value becomes [0]. Therefore, the process for the next division method candidate is not performed.
In this specific example, although the 16 × 8 division method (encoding cost (42)) is the division method candidate selected in the division method candidate selection step S42, sub-pel prediction is not performed.

Effect of (3-2) The sub-pel prediction unit 15 can control the processing amount of the sub-pel prediction. In particular, when the control for minimizing the processing amount is performed, in the case of a software encoder, there is an effect of shortening the processing time, and in the case of a hardware encoder, there is an effect of reducing power consumption. In addition, when the processing time must be constant as in the real-time encoder, it is possible to improve the compression performance by allocating the surplus processing amount to other candidates.

(3-3)
In the said embodiment, it demonstrated as follows. That is, the division method determination step S44 determines the prediction direction for each small block from the minimum coding cost for each small block divided by the two types of division method candidates selected in the division method candidate selection step S42. In addition, the encoding cost for each image block is derived. Further, the derived encoding cost for each image block is compared for two types of division method candidates, and the division method candidate having the minimum encoding cost is determined as the image block division method.

Here, the division method candidates selected in the division method candidate selection step S42 are not limited to two types, and more candidates may be selected.
Further, the division method determination step S44 may select more methods without determining one image block division method. For example, in the division method determination step S44, two encoding modes may be selected, and the final encoding mode may be determined in yet another step.

(4) Other modifications (4-1)
It has been described that the full-pel prediction unit 13 performs motion estimation with integer pixel accuracy, and the sub-pel prediction unit 15 performs motion estimation of non-integer pixels. Here, the accuracy of motion estimation is not limited to these.

For example, the full-pel prediction unit 13 may perform simple motion estimation, and the sub-pel prediction unit 15 may perform complex motion estimation.
More specifically, complex motion estimation is more complex motion estimation than simple motion estimation. For example, complex motion estimation is more detailed accuracy than simple motion estimation with integer pixel accuracy. (E.g., non-integer pixel accuracy such as ½ pixel accuracy, ¼ pixel accuracy, etc.), motion estimation with more detailed accuracy for simple motion estimation of non-integer pixels, reduced image (pixel information For example, a motion estimation that refers to a more detailed image with respect to a simple motion estimation that refers to a thinned image).

Further, the simple motion estimation may be motion estimation such as 2-pixel accuracy and 1 / 2-pixel accuracy for a reduced image.
Accordingly, it is possible to enjoy the effect of appropriate motion estimation by complex motion estimation while reducing the amount of processing of complex motion estimation.

Here, the encoding mode is selected by motion estimation with two stages of accuracy, but the selection may be performed by more stages.
For example, the coding mode may be selected using motion estimation with three stages of accuracy, such as integer pixel accuracy, 1/2 pixel accuracy, and 1/4 pixel accuracy.

(4-2)
The full-pel prediction unit 13 or the sub-pel prediction unit 15 may change the motion estimation method so that the processing amount of each motion estimation is kept substantially constant.
Conventionally, a predetermined processing time is assigned to each motion estimation process, and the motion estimation process is performed on a fixedly selected partition size and reference picture within the predetermined processing time. Yes. In this case, processing time is allocated assuming that the amount of processing is considered to be the largest (worst case). Therefore, when the processing target is not the worst case, there is room in processing time, and efficient motion estimation processing is performed. It is hindered.

  Therefore, in the present invention, when there is a margin in processing, the motion estimation method is changed to perform efficient motion estimation processing. Here, whether or not there is a margin for the processing is determined, for example, according to the image attribute of the input image configured by the image block. Image attributes include, for example, image size, image encoding method (picture type [I picture, P picture, B picture], etc.), image format (scanning method (progressive, interlaced), color difference format, etc.) And the amount of motion of the image.

(4-2-1)
For example, the motion estimation method (number of reference pictures, direction, partition size for motion estimation is set so that the product of the input image size composed of image blocks, the number of reference pictures, and the number of partition sizes is substantially constant. Variation, motion search range, etc.). More specifically, when the input image size is small, it is possible to increase the number of reference pictures and the number of partition sizes, and to perform full-pel prediction or sub-pel prediction processing with higher accuracy.

(4-2-2)
Further, for example, the number of reference pictures of B picture is made smaller than that of P picture so that the processing amount of motion estimation is kept almost constant for each picture. More specifically, the following variations can be considered. <1> The front four pictures are referred to in the P picture, and the front two pictures and the rear two pictures are referred to in the B picture. <2> The front 3 frames are referred to in the P picture, and the front 2 frames and the rear 1 frame are referred to in the B picture. <3> The front picture is referred to in the P picture, and the front picture and the rear picture are referred to in the B picture.

(4-2-3)
Further, for example, the number of partition sizes of a B picture is made smaller than that of a P picture, so that the processing amount of motion estimation for each picture is kept almost constant. More specifically, the following variations can be considered. <1> It is assumed that prediction is performed with four partition sizes of 16 × 16, 16 × 8, 8 × 16, and 8 × 8 with reference to the front one in the P picture. On the other hand, in the B picture, any two of the above four sizes are selected, and each performs forward prediction and backward prediction. <2> It is assumed that prediction is performed with four partition sizes of 16 × 16, 16 × 8, 8 × 16, and 8 × 8 with reference to the rear one in the P picture. On the other hand, in the B picture, any two of the above four sizes are selected, and each performs forward prediction and backward prediction.

(4-2-4)
Also, for example, when the input image is interlaced, the number of reference pictures or the number of partition sizes is reduced more than in the case of progressive. This is because in the case of interlace, it is necessary to refer to two fields, a top field and a bottom field. More specifically, the following variations can be considered. In the case of a <1> P picture, the front two frames are referred to in the progressive P picture, and the two front fields (one frame in time) are referred to in the interlaced P picture. <2> In the case of a P picture, the progressive P picture refers to the front one frame, and the partition size is predicted in four types (16 × 16 to 8 × 8). In the interlaced P picture, the front two fields (in terms of time) One frame) is referenced, and two types of partition sizes are predicted.

(4-2-5)
Also, for example, the number of reference pictures or the number of partition sizes is changed according to the motion of the image. In motion vector search, there is a method in which search processing time is affected by the size of a motion vector. When such a method is used, if the motion is small, the processing time of each partition size and each reference picture is short. For this reason, when the motion is small, motion estimation is performed using a larger number of partition sizes and reference pictures. On the other hand, when the motion is large, the processing time for each motion estimation becomes long. For this reason, when the motion is large, the number of reference pictures is reduced, or the number of partition sizes is reduced.

[Second Embodiment]
An encoder as a second embodiment of the present invention will be described with reference to FIGS. 16 and 17.
FIG. 16 is a block diagram illustrating the structure of an encoder 60 as the second embodiment of the present invention. The encoder 60 is, for example, an image encoding device that performs MPEG-4 encoding on the input image signal 30 and outputs the encoded image signal 31 as an encoded image signal 31, and is provided in a personal computer (PC), a mobile phone, or the like. In addition, it is an apparatus that encodes the input image signal 30 in units of image block pairs 73 introduced in AVC (see FIG. 30).

<Configuration of Encoder 60>
The encoder 60 illustrated in FIG. 16 includes an intra prediction unit 61 that performs intra prediction of the input image signal 30, an inter prediction unit 62 that performs inter prediction of the input image signal 30, a coding mode determination unit 63, and intra prediction and inter prediction. A switching unit 64 that switches prediction results of prediction, an encoding unit 5 that encodes the output of the switching unit 64 and outputs an encoded image signal 31, and a reference image generation unit that generates a local decode signal 32 of the input image signal 30 6 is provided.

  The intra prediction unit 61 is controlled by a control unit (not shown) and performs intra prediction on a block having a picture structure (field structure block or frame structure block) determined by the encoded picture structure determination unit 67. As a result, the intra prediction unit 61 performs intra prediction on the input image signal 30 for each image block, and outputs the intra prediction result to the switching unit 4.

  The inter prediction unit 62 outputs the inter prediction result to the switching unit 4 with the input image signal 30 as the first input and the local decode signal 32 as the second input. Further, the inter prediction unit 62 outputs information related to encoding, such as a motion vector, among the inter prediction results to the encoding unit 5 as a second output.

  The inter prediction unit 3 uses the input image signal 30 as a first input and the local decode signal 32 as a second input, and performs a motion estimation unit 65 that performs motion estimation, and outputs the motion estimation unit 65 as a first input, A predicted image generating unit 11 that outputs the predicted image by using the decoded signal 32 as a second input; and a subtracting unit 12 that uses the input image signal 30 as a first input and the output of the predicted image generating unit 11 as a second input; It is composed of The motion estimation unit 65 performs motion estimation and derives an encoding cost. Of the output of the motion estimation unit 65, the encoded information such as the motion vector and the encoding mode is also given to the input of the variable length encoding unit 22.

The switching unit 4 uses the intra prediction result as the first input and the inter prediction result as the second input, and outputs one of the inputs to the encoding unit 5 in accordance with the switching signal from the encoding mode determination unit 63.
Since the structures and functions of the encoding unit 5 and the reference image creation unit 6 are the same as those in the above embodiment, description thereof is omitted here.

  The encoding mode determination unit 63 includes an encoded picture structure determination unit 67 and an intra / inter determination unit 68. The encoded picture structure determination unit 67 receives the encoding cost information from the motion estimation unit 65 as an input. The encoded picture structure determining unit 67 determines the encoded picture structure by summing the encoding costs for the top and the bottom for each encoded picture structure. The encoded picture structure determination unit 67 outputs the determined encoded picture structure to the intra / inter selection unit 68.

  The intra / inter selection unit 68 receives the encoding cost of intra prediction from the intra prediction unit 61 and the encoding cost of inter prediction from the inter prediction unit 62 as inputs. The intra / inter selection unit 68 compares the encoding costs of intra prediction and inter prediction, and determines the encoding mode. The intra / inter selection unit 68 notifies the switching unit 64 of the result. As a result, the switching unit 64 operates.

Note that the control unit may be included in the encoding mode determination unit 63.
FIG. 17 is a block diagram illustrating a processing flow of determining an encoding mode (determining an encoded picture structure and determining an encoding prediction scheme for an image block pair). The processing flow of FIG. 17 includes an inter prediction step S51 by the motion estimation unit 65, an encoded picture structure determination step S52 by the encoded picture structure determination unit 67, an intra prediction step S53 by the intra prediction unit 61, and intra / inter selection. And an encoding prediction method determination step S54 by the unit 68.

  The inter prediction step S51 derives motion estimation results for the field structure block pairs 75 and 76 and the frame structure block pairs 77 and 78 of the image block pair 73 (see FIG. 30). Specifically, the inter prediction step S51 includes a first inter prediction step S511 for the frame structure top MB77 and a second inter prediction step S512 for the bottom MB78. In the first inter prediction step S511, inter prediction is performed on the frame structure top MB77, and an encoding cost (cost top0) is derived. In the second inter prediction step S512, inter prediction is performed on the bottom MB 78 of the frame structure block pair, and a coding cost (cost bot0) is derived. The respective coding costs cost top0 and cost bot0 are sent to the coded picture structure determination step S52. Further, the respective coding costs cost top0 and cost bot0 are summed to obtain the coding cost cost0 of the frame structure block pair 77 and 78, which is sent to the coded picture structure determination step S52. In this embodiment, cost top0 is 1500, cost bot0 is 1300, and cost0 is 2800. The inter prediction step S51 further includes a third inter prediction step S513 for the top MB 75 of the field structure block pair 75 and 76 and a fourth inter prediction step S514 for the bottom MB 76. In the third inter prediction step S513, inter prediction is performed on the top MB 75 of the field structure block pair 75 and 76, and a coding cost (cost top1) is derived. In the fourth inter prediction step S514, inter prediction is performed on the bottom MB 76 of the field structure block pair 75 and 76, and a coding cost (cost bot1) is derived. The respective coding costs cost top1 and cost bot1 are sent to the coded picture structure determination step S52. Further, the respective coding costs cost top1 and cost bot1 are summed to obtain the coding cost cost1 of the field structure block pair 75 and 76, which is sent to the coded picture structure determination step S52. In this embodiment, cost top1 is 1400, cost bot1 is 1300, and cost1 is 2700.

  The first to fourth inter prediction steps S511 to S514 are motions including a 16 × 16 division method, a 16 × 8 division method, an 8 × 16 division method, and an 8 × 8 division method, respectively. It represents the entire estimation operation. That is, the first embodiment of the present invention can be applied to the first to fourth inter prediction steps S511 to S514. Moreover, although 1st-4th inter prediction step S511-step S514 may perform both full pel prediction and sub pel prediction, you may perform only full pel prediction for processing amount reduction.

As described above, only the inter prediction is performed for deriving the coding cost of the coded picture structure. However, since the accuracy of determination of inter prediction is better than that of intra prediction, sufficient accuracy can be obtained.
The encoded picture structure determination step S52 determines the encoded picture structure of the image block pair 73 based on the motion estimation result. Specifically, the coded picture structure determination step S52 compares the coding cost cost0 of the frame structure block pair 77 and 78 from the inter prediction step S51 with the coding cost cost1 of the field structure block pair 75 and 76. Frame / field selection. In this embodiment, since the coding cost cost1 (2700) of the field structure block pair 75 and 76 is smaller than the coding cost cost0 (2800) of the frame structure block pair 77 and 78, the field is selected. As a result, the inter coding cost cost top1 of the top MB 75 and the inter coding cost cost bot1 of the bottom MB 76 of the field structure block pair 75 and 76 are provided to the coding prediction method determination step S54.

  In the intra prediction step S53, an intra prediction result for a block pair having the determined coded picture structure is derived. Specifically, the intra prediction step S53 includes a first intra prediction step S531 for the top MB and a second intra prediction step S532 for the bottom MB. The first intra prediction step S531 derives the intra coding cost cost top2 for the top MB 75 of the selected coded picture structure block pair (in this case, the field structure block pair 75, 76), and the coding prediction method determination step S54. To provide. In the second intra prediction step S532, an intra coding cost cost bot2 is derived for the bottom MB 76 of the selected coded picture structure block pair (in this case, the field structure block pair 75, 76), and a coding prediction method determination step S54 is performed. To provide. In this embodiment, cost top2 is 1500 and cost bot2 is 1400. Further, the intra prediction may be a process in which the accuracy is reduced by thinning out pixels in order to reduce the processing amount, and the intra 4 × 4 may be omitted.

  The coding prediction scheme determination step S54 determines a coding prediction scheme for each block of the block pair having the determined coded picture structure based on the inter prediction result and the intra prediction result. Specifically, the encoding prediction method determination step S54 includes a first encoding prediction method determination step S541 for the top MB and a second encoding prediction method determination step S542 for the bottom MB. The first encoding prediction scheme determination step S541 is the top MB inter encoding cost from the encoded picture structure determination step S52 (specifically, the inter MB encoding cost cost top1 of the top MB 75 of the field structure block pair 75, 76). ) And the intra coding cost cost top2 of the top MB 75 from the first intra prediction step S531, and intra / inter selection for the top MB is performed. In this case, since inter coding cost cost top1 (1400) is smaller than intra coding cost cost top2 (1500), inter is selected. The second encoding prediction method determination step S542 is a bottom MB inter encoding cost from the encoded picture structure determination step S52 (specifically, an inter encoding cost cost bot1 of the bottom MB 76 of the field structure block pair 75, 76). ) And the intra coding cost cost bot2 of the bottom MB76 from the second intra prediction step S532, and performs intra / inter selection for the bottom MB76. In this case, inter is selected because the inter coding cost cost bot1 (1300) is smaller than the intra coding cost cost bot2 (1400) in this case.

  In this embodiment, the encoding prediction scheme (intra / inter) is the same for the top MB and the bottom MB, but may be different. However, the top MB and the bottom MB are not encoded with different encoded picture structures. This is because the encoded picture structure is determined in the encoded picture structure determining step S52.

  In this embodiment, since the intra prediction step S53 performs intra prediction only for the image block pair having the coded picture structure determined by the coded picture structure determining step S52, the intra prediction step S53 includes the field structure block and the frame structure block. There is no need to make intra predictions for everything. As described above, since the number of intra predictions with a high processing load can be reduced, the processing load for determining the coding prediction method of the image block pair can be reduced.

<Modification>
In the second embodiment, the contents described in the first embodiment can be appropriately modified and applied. Here, a modified example characteristic of the second embodiment will be described.
(1)
In the said embodiment, 1st-4th inter prediction step S511-step S514 described that you may perform both full pel prediction and sub pel prediction.

  Here, when performing both full-pel prediction and sub-pel prediction, sub-pel prediction may be performed on a combination of partition size, reference picture, and picture structure narrowed down by full-pel prediction. More specifically, as shown in FIG. 18, an appropriate combination of partition size, reference picture, and picture structure is selected by motion estimation with integer pixel accuracy (step S710), and a non-integer is selected for the selected combination. Further narrowing is performed by motion estimation with pixel accuracy (step S711). Further, intra prediction is performed on the combination of partition size, reference picture, and picture structure obtained as a result of narrowing down (step S712), and intra prediction / inter prediction is selected.

As a result, it is not necessary to perform motion estimation and intra prediction with non-integer pixel accuracy for all partition sizes, all reference pictures, and all picture structures, and the processing amount can be reduced.
(2)
In the above embodiment, the apparatus that encodes the input image signal 30 in units of the image block pair 73 has been described. Here, encoding may not be performed in units of image block pairs. For example, encoding may be performed in units of square image blocks. In this case, the method described in the above embodiment is applied to a rectangular field structure block pair and a rectangular frame structure block pair.

[Third Embodiment]
An encoder as a third embodiment of the present invention will be described with reference to FIGS.
<Configuration of encoder 1>
19 includes an intra prediction unit 91 that performs intra prediction of the input image signal 30, an inter prediction unit 92 that performs inter prediction of the input image signal 30, a coding mode determination unit 93, and intra prediction and inter prediction. A switching unit 94 that switches prediction prediction results, an encoding unit 5 that encodes the output of the switching unit 94 and outputs an encoded image signal 31, and a reference image generation unit that generates a local decode signal 32 of the input image signal 30 6 is provided.

  The intra prediction unit 91 can perform simple intra prediction and complex intra prediction. Simple intra prediction is, for example, intra prediction for a compressed image, and complex intra prediction is, for example, intra prediction for an uncompressed image. The intra prediction unit 91 is controlled by a control unit 99 (described later) in the encoding mode determination unit 93, performs simple intra prediction, and derives an encoding cost. As a result, the intra prediction unit 91 performs intra prediction on the input image signal 30 for each image block, and outputs the intra prediction result to the switching unit 94.

  The inter prediction unit 92 outputs the inter prediction result to the switching unit 94 with the input image signal 30 as the first input and the local decode signal 32 as the second input. Further, the inter prediction unit 92 outputs information related to encoding, such as a motion vector, among the inter prediction results to the encoding unit 5 as a second output.

  The inter prediction unit 92 uses the input image signal 30 as a first input and the local decoded signal 32 as a second input, and performs a motion estimation unit 95 that performs motion estimation, and outputs the motion estimation unit 95 as a first input, A predicted image generating unit 11 that outputs the predicted image by using the decoded signal 32 as a second input; and a subtracting unit 12 that uses the input image signal 30 as a first input and the output of the predicted image generating unit 11 as a second input; It is composed of The motion estimation unit 95 performs full-pel inter prediction or sub-pel inter prediction and derives an encoding cost. Also, of the output of the motion estimation unit 95, the encoding information such as the motion vector and the encoding mode is also given to the input of the variable length encoding unit 22.

The switching unit 94 uses the intra prediction result as the first input and the inter prediction result as the second input, and outputs one of the inputs to the encoding unit 5 in accordance with the switching signal from the encoding mode determination unit 93.
Since the structures and functions of the encoding unit 5 and the reference image creation unit 6 are the same as those in the above embodiment, description thereof is omitted here.

  The encoding mode determination unit 93 includes a determination unit 96 and a control unit 99. The determination unit 96 includes an intra / inter selection unit 97 and an encoded picture structure determination unit 98. The determination unit 96 receives the encoding cost from the motion estimation unit 95 and the encoding cost from the intra prediction unit 91 as inputs. The intra / inter selector 97 determines intra / inter. An encoded picture structure determination unit 98 determines a field / frame. The control unit 99 controls the intra prediction unit 91 or the motion estimation unit 95 to cause the image block pair 73 having the determined coded picture structure to perform motion estimation. That is, the control unit 99 causes the intra prediction unit 91 to perform complex intra prediction or causes the motion estimation unit 95 to perform sub-pel / inter prediction. The control unit 99 further operates the switching unit 94 to encode the intra prediction result or the inter prediction result.

The control unit only needs to be somewhere in the encoder 90. The encoding mode determination unit 93 may not be provided.
FIG. 20 is a flowchart of the processing operation for determining the coding mode of the image block pair 73. This processing operation includes a simple motion estimation step S61 executed by the intra prediction unit 91 or the motion estimation unit 95, an intra / inter selection step S62 executed by the intra / inter selection unit 97, and an encoded picture structure determination unit 98. And a coded picture structure determination step S63 of the image block pair 73 executed by step S63. Note that, after the encoded picture structure determination step S63, a complex motion estimation step S64 executed by the intra prediction unit 91 or the motion estimation unit 95 is provided.

  The simple motion estimation step S61 performs full-pel inter prediction and simple intra prediction on the top MB and the bottom MB of the frame / field structure, and derives their coding costs. The simple motion estimation step S61 includes first to eighth estimation steps S611 to S618. The first estimation step S611 performs full-pel inter prediction for the top MB 77 of the frame structure block pair 77, 78, and the second estimation step S612 performs simple / intra prediction for the top MB 77 of the frame structure block pair 77, 78. Do. The third estimation step S613 performs full-pel inter prediction on the bottom MB78 of the frame structure block pair 77 and 78, and the fourth estimation step S614 performs simple intra prediction on the bottom MB78 of the frame structure block pair 77 and 78. . The fifth estimation step S615 performs full-pel inter prediction on the top MB 75 of the field structure block pair 75 and 76, and the sixth estimation step S616 performs simple intra prediction on the top MB 75 of the field structure block pair 75 and 76. . The seventh estimation step S617 performs full-pel inter prediction on the bottom MB 76 of the field structure block pair 75, 76, and the eighth estimation step S818 performs simple intra prediction on the bottom MB 76 of the field structure block pair 75, 76. . As described above, since the simple motion estimation step S61 derives the coding costs of the frame structure block pairs 77 and 78 and the field structure block pairs 75 and 76 using inter prediction and intra prediction, either inter prediction or intra prediction is used. Even in the case of the image block pair 73 (71, 72) in which the compression rate is improved, it is possible to determine a coded picture structure that provides the best compression rate.

The intra / inter selection step S62 compares the intra and inter coding costs for each of the four types (frame, field) * (top, bottom), and selects the smaller one.
The intra / inter selection step S62 includes first to fourth selection steps S621 to S624. The first selection step S621 compares the coding costs of the first estimation step S611 and the second estimation step S612, and selects intra / inter for the frame structure top MB77. In this case, the encoding cost (1300) of the second estimation step S612 is selected. The second selection step S622 compares the coding costs of the third estimation step S613 and the fourth estimation step S614, and selects intra / inter for the frame structure bottom MB78. In this case, the encoding cost (1200) of the fourth estimation step S614 is selected. The coding cost (1300) of the frame structure top MB 77 for which intra / inter is selected and the coding cost (1200) of the bottom MB 78 are summed, and the coding cost (2500) of the frame structure block pair 77 and 78 is obtained. The third selection step S623 compares the coding costs of the fifth estimation step S615 and the sixth estimation step S616, and selects intra / inter for the field structure top MBs 75 and 76. In this case, the encoding cost (1400) of the fifth estimation step S615 is selected. The fourth selection step S624 compares the coding costs of the seventh estimation step S617 and the eighth estimation step S618 and selects intra / inter for the field structure bottom MB76. In this case, the encoding cost (1300) of the seventh estimation step S617 is selected. The coding cost (1400) of the field structure top MB 75 for which intra / inter is selected and the coding cost (1300) of the bottom MB 76 are summed, and the coding cost (2700) of the field structure block pair 75 and 76 is obtained.

  The coded picture structure determination step S63 compares the coding cost of the frame structure block pair 77 and 78 with the coding cost of the field structure block pair 75 and 76 to determine the field / frame of the image block pair 73. In this case, since the coding cost (2500) of the frame structure block pair 77, 78 is smaller than the coding cost (2700) of the field structure block pair 75, 76, the frame structure block pair 77, 78 is selected.

  In the complex motion estimation step S64, complex motion estimation (one of sub-pel / inter or complex intra) is performed for each of the top MB and the bottom MB 77 and 78 of the image block pair 73 having the determined coded picture structure. The complex motion estimation step S64 includes first to fourth estimation steps S641 to S644. The first estimation step S641 performs sub-pel inter prediction for the top MB77. In the second estimation step S642, complex intra prediction for the top MB 77 is performed. Note that only one of the first estimation step S641 and the second estimation step S642 is executed. In the third estimation step S643, sub-pel inter prediction is performed on the bottom MB78. The fourth estimation step S644 performs complex intra prediction for the bottom MB78. Note that only one of the third estimation step S643 and the fourth estimation step S644 is executed.

  As described above, the encoding mode (specifically, the encoded picture structure) is determined by the encoded picture structure determination step S63 based on the simple inter prediction / intra prediction in the simple motion estimation step S61. ing. For this reason, it is possible to reduce the processing amount for determining the encoding mode.

  Further, after the coding mode is determined, the complex motion estimation step S64 performs complex motion estimation. Thus, since the image block pair 73 is encoded by complicated prediction, the compression efficiency is improved. In addition, since complicated prediction is performed only for the image block pair 73 having the determined coded picture structure, the number of times of complicated prediction can be reduced as compared with the conventional case. As a result, the processing amount can be reduced while maintaining the encoding efficiency.

The top MB and the bottom MB are not encoded with different encoded picture structures, but may be encoded with different encoding prediction schemes (intra / inter).
In the third embodiment, the contents described in the first and second embodiments can be appropriately modified and applied.
[Fourth Embodiment]
Furthermore, application examples of the moving picture coding apparatus shown in the above embodiment and a system using the same will be described.

FIG. 21 is a block diagram showing an overall configuration of a content supply system ex100 that implements a content distribution service. The communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed radio stations, are installed in each cell.
The content supply system ex100 includes, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, a camera via the Internet ex101, the Internet service provider ex102, the telephone network ex104, and the base stations ex107 to ex110. Each device such as the attached mobile phone ex115 is connected.

However, the content supply system ex100 is not limited to the combination as shown in FIG. 21, and may be connected in any combination. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.
The camera ex113 is a device capable of shooting a moving image such as a digital video camera. The mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global System for Mobile Communications) system mobile phone, Alternatively, PHS (Personal Handyphone System) or the like may be used.

  In addition, the streaming server ex103 is connected from the camera ex113 through the base station ex109 and the telephone network ex104, and live distribution or the like based on the encoded data transmitted by the user using the camera ex113 becomes possible. The encoded processing of the captured data may be performed by the camera ex113 or may be performed by a server or the like that performs data transmission processing. Further, the moving image data shot by the camera 116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device that can shoot still images and moving images, such as a digital camera. In this case, the encoding of the moving image data may be performed by the camera ex116 or the computer ex111. The encoding process is performed in the LSI ex117 included in the computer ex111 and the camera ex116. Note that image encoding / decoding software may be incorporated into any storage medium (CD-ROM, flexible disk, hard disk, etc.) that is a recording medium readable by the computer ex111 or the like. Furthermore, you may transmit moving image data with the mobile telephone ex115 with a camera. The moving image data at this time is data encoded by the LSI included in the mobile phone ex115.

  In this content supply system ex100, the content (for example, video shot of music live) captured by the user with the camera ex113, camera ex116, etc. is encoded and transmitted to the streaming server ex103 as in the above embodiment. On the other hand, the streaming server ex103 distributes the content data to the requested client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, and the like that can decode the encoded data. In this way, the content supply system ex100 can receive and reproduce the encoded data at the client, and also realize personal broadcasting by receiving, decoding, and reproducing in real time at the client. It is a system that becomes possible.

The moving picture coding apparatus described in each of the above embodiments may be used for coding of each device constituting this system.
A mobile phone will be described as an example.
FIG. 22 is a diagram illustrating the mobile phone ex115 using the video encoding device described in the above embodiment. The cellular phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a camera such as a CCD camera, a camera unit ex203 capable of taking a still image, a video shot by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display that displays data obtained by decoding received video and the like, a main body unit composed of a group of operation keys ex204, an audio output unit ex208 such as a speaker for audio output, and audio input To store encoded data or decoded data such as a voice input unit ex205 such as a microphone, captured video or still image data, received mail data, video data or still image data, etc. Recording medium ex207, and slot portion ex20 for enabling recording medium ex207 to be attached to mobile phone ex115 The has. The recording medium ex207 stores a flash memory element which is a kind of EEPROM (Electrically Erasable and Programmable Read Only Memory) which is a nonvolatile memory that can be electrically rewritten and erased in a plastic case such as an SD card.

  Further, the cellular phone ex115 will be described with reference to FIG. The cellular phone ex115 controls the power supply circuit ex310, the operation input control unit ex304, and the image coding for the main control unit ex311 which is configured to control the respective units of the main body unit including the display unit ex202 and the operation key ex204. Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / reproducing unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via a synchronization bus ex313 Are connected to each other.

When the end call and power key are turned on by a user operation, the power supply circuit ex310 activates the camera-equipped digital mobile phone ex115 by supplying power from the battery pack to each unit. .
The mobile phone ex115 converts the voice signal collected by the voice input unit ex205 in the voice call mode into digital voice data by the voice processing unit ex305 based on the control of the main control unit ex311 including a CPU, a ROM, a RAM, and the like. The modulation / demodulation circuit unit ex306 performs spread spectrum processing, and the transmission / reception circuit unit ex301 performs digital analog conversion processing and frequency conversion processing, and then transmits the result via the antenna ex201. In addition, the cellular phone ex115 amplifies the received signal received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-digital conversion processing, performs spectrum despreading processing by the modulation / demodulation circuit unit ex306, and analog audio by the voice processing unit ex305. After conversion into a signal, this is output via the audio output unit ex208.

  Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmits the text data to the base station ex110 via the antenna ex201.

  When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

  The image encoding unit ex312 has a configuration including the image encoding device described in the present invention, and an encoding method using the image data supplied from the camera unit ex203 in the image encoding device described in the above embodiment. The encoded image data is converted into encoded image data by compression encoding, and sent to the demultiplexing unit ex308. At the same time, the cellular phone ex115 sends the sound collected by the audio input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 as digital audio data via the audio processing unit ex305.

  The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and the multiplexed data obtained as a result is a modulation / demodulation circuit unit A spectrum spread process is performed in ex306, a digital analog conversion process and a frequency conversion process are performed in the transmission / reception circuit unit ex301, and then the signal is transmitted through the antenna ex201.

When receiving data of a moving image file linked to a home page or the like in the data communication mode, the received signal received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexing is obtained. Is sent to the demultiplexing unit ex308.
In addition, in order to decode the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data to generate an encoded bitstream of image data and an encoded bitstream of audio data. The encoded image data is supplied to the image decoding unit ex309 via the synchronization bus ex313, and the audio data is supplied to the audio processing unit ex305.

  Next, the image decoding unit ex309 generates reproduction moving image data by decoding the encoded bit stream of the image data with a decoding method corresponding to the encoding method described in the above embodiment, and generates the reproduced moving image data. This is supplied to the display unit ex202 via the control unit ex302, and thereby, for example, moving image data included in a moving image file linked to a home page is displayed. At the same time, the audio processing unit ex305 converts the audio data into an analog audio signal, and then supplies the analog audio signal to the audio output unit ex208. Thus, for example, the audio data included in the moving image file linked to the home page is reproduced. The

  It should be noted that the present invention is not limited to the above-described system, and recently, digital broadcasting using satellites and terrestrial waves has become a hot topic, and as shown in FIG. Can do. Specifically, in the broadcasting station ex409, the encoded bit stream of the video information is transmitted to the communication or broadcasting satellite ex410 via radio waves. Receiving this, the broadcasting satellite ex410 transmits a radio wave for broadcasting, and receives the radio wave with a home antenna ex406 having a satellite broadcasting receiving facility, such as a television (receiver) ex401 or a set top box (STB) ex407. The device decodes the encoded bit stream and reproduces it. In addition, the image decoding apparatus described in the above embodiment can also be mounted on a playback apparatus ex403 that reads and decodes an encoded bitstream recorded on a storage medium ex402 such as a CD or DVD as a recording medium. is there. In this case, the reproduced video signal is displayed on the monitor ex404. Further, a configuration in which an image decoding device is mounted in a set-top box ex407 connected to a cable ex405 for cable television or an antenna ex406 for satellite / terrestrial broadcasting, and this is reproduced on the monitor ex408 of the television is also conceivable. At this time, the image decoding apparatus may be incorporated in the television instead of the set top box. It is also possible to receive a signal from the satellite ex410 or the base station ex107 by the car ex412 having the antenna ex411 and reproduce a moving image on a display device such as the car navigation ex413 that the car ex412 has.

  Further, the image signal can be encoded by the image encoding device shown in the above embodiment and recorded on a recording medium. As a specific example, there is a recorder ex420 such as a DVD recorder that records an image signal on a DVD disk ex421 or a disk recorder that records on a hard disk. Further, it can be recorded on the SD card ex422. If the recorder ex420 includes the image decoding device described in the above embodiment, the image signal recorded on the DVD disc ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.

  For example, the configuration of the car navigation ex413 may be a configuration excluding the camera unit ex203, the camera interface unit ex303, and the image encoding unit ex312 in the configuration illustrated in FIG. 23. The same applies to the computer ex111 and the television (receiver). ) Ex401 can also be considered.

In addition to the transmission / reception type terminal having both the encoder and the decoder, the terminal such as the mobile phone ex114 has three mounting formats: a transmitting terminal having only an encoder and a receiving terminal having only a decoder. Can be considered.
As described above, the moving picture coding apparatus described in the above embodiment can be used in any of the above-described devices and systems, and by doing so, the effects described in the above embodiment can be obtained. .

[Modifications common to all embodiments]
(1)
In the above embodiment, a macroblock partition obtained by dividing a 16 × 16 macroblock by each division candidate has been treated as a unit of motion estimation as a small block. In this case, as shown in FIG. 25, the small block obtained by the 8 × 8 division method can be further divided into 8 × 8, 8 × 4, 4 × 8, and 4 × 4 sub-macroblock partitions. Therefore, the present invention can be applied by using this sub-macroblock partition as a small block of the present invention.

(2)
Each functional block in the block diagrams (for example, FIG. 1, FIG. 16, FIG. 19, FIG. 23, etc.) is typically implemented as an LSI that is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them.

  More specifically, the motion estimation unit 10 in FIG. 1 may be integrated into one chip. Furthermore, functional blocks other than the memory 26 in FIG. 1 may be integrated into one chip. Further, the inter prediction unit 3, the coding mode determination unit 63, and the intra prediction unit 61 in FIG. 16 may be integrated into one chip. Furthermore, functional blocks other than the memory 26 in FIG. 16 may be integrated into one chip. Further, the inter prediction unit 92 and the encoding mode determination unit 93 in FIG. 19 may be integrated into one chip. Furthermore, functional blocks other than the memory of FIG. 19 may be integrated into one chip.

Note that the name used here is LSI, but it may also be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.
Further, the method of circuit integration is not limited to LSI, and implementation with a dedicated circuit or a general-purpose processor is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Further, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. Biotechnology can be applied.

  The encoding mode determination device, the image encoding device, the encoding mode determination method, and the encoding mode determination program according to the present invention enable selection of an appropriate encoding mode with a smaller amount of processing, which is useful in the above field. is there.

The block diagram of the image coding apparatus which concerns on one Embodiment of this invention. The processing flowchart of the motion estimation part which concerns on this invention. The figure which shows the processing flow of the motion estimation part which concerns on this invention. The figure which shows the selection method of the division candidate by the division candidate selection part which concerns on this invention. The figure which shows the processing flow of a full pel prediction part. The figure which shows the processing flow of a full pel prediction part. The figure which shows the modification of the processing flow of a full pel prediction part. The figure which shows the modification of the processing flow of a full pel prediction part. The figure which shows the modification of the processing flow of a full pel prediction part and a division | segmentation candidate selection part. The figure which shows the encoding cost conversion by the encoding cost conversion part and a division | segmentation candidate selection part, and the selection method of a division | segmentation candidate. The figure which shows the modification of the processing flow of a full pel prediction part and an encoding cost conversion part. The processing flowchart concerning a 1st embodiment of the present invention. The processing flowchart of the sub pel prediction part which concerns on 1st Embodiment of this invention. The figure which shows the example of 1 processing amount distribution of subpel prediction. The figure which shows the example of 1 processing amount distribution of subpel prediction. The block diagram of the image coding apparatus which concerns on 2nd Embodiment of this invention. The figure which shows the processing flow of an intra estimation part, a motion estimation part, and an encoding mode determination part. The process flowchart concerning 2nd Embodiment of this invention. The block diagram of the image coding apparatus which concerns on 3rd Embodiment of this invention. The figure which shows the processing flow of an intra estimation part, a motion estimation part, and an encoding mode determination part. The block diagram which shows the whole structure of a content supply system. The example of the mobile phone using the moving image encoding method and the moving image decoding method. A block diagram of a mobile phone. An example of a system for digital broadcasting. The figure which shows the division | segmentation method candidate of the conventional macroblock. The figure which shows the relationship between the coding picture by the division | segmentation method candidate of the conventional macroblock, and a reference picture. The figure which shows the prediction direction of the conventional macroblock. The figure which shows the processing flow of the conventional motion estimation. The figure which shows the processing flow of the conventional motion estimation. The figure for demonstrating the concept of the image block pair in MPEG-4AVC. The figure which shows the processing flow of the conventional encoding picture structure determination and encoding prediction system determination. The figure which shows the process flow of the encoding picture structure determination and encoding prediction system determination at the time of assuming that the prior art is applied to MPEG-4AVC although it is not a prior art.

Explanation of symbols

1 Encoder 2 Intra Prediction Unit 3 Inter Prediction Unit 4 Switching Unit 10 Motion Estimation Unit 13 Full Pel Prediction Unit 14 Division Method Candidate Selection Unit 15 Subpel Prediction Unit 16 Division Method Determination Unit 60 Encoder 61 Intra Prediction Unit 62 Inter Prediction Unit 63 Coding Mode Determining unit 64 Switching unit 65 Motion estimation unit 67 Coding picture structure determination unit 68 Intra / inter selection unit 91 Intra prediction unit 92 Inter prediction unit 93 Coding mode determination unit 94 Switching unit 95 Motion estimation unit 96 Determination unit 97 Intra / inter Selection unit 98 Encoded picture structure determination unit 99 Control unit

Claims (31)

An encoding mode determining apparatus that determines at least one encoding mode of an image block from a plurality of candidates,
A simple motion estimation unit that derives a first encoding cost that is an encoding cost of each encoding mode based on simple motion estimation for a small block that is a partition of an image block obtained by each encoding mode;
An encoding mode selection unit that selects a part of the encoding modes from the plurality of encoding modes based on the first encoding cost derived by the simple motion estimation unit;
Complex motion estimation for deriving a second coding cost that is a coding cost of each coding mode based on a complex motion estimation for a small block obtained by at least a part of the coding modes. And
An encoding mode determination unit that determines an encoding mode of the image block based on a second encoding cost derived by the complex motion estimation unit;
Equipped with a,
The complex motion estimator is
Based on the simple motion estimation in the simple motion estimation unit, determine a picture reference direction in the complex motion estimation,
As a result of simple motion estimation for the small block in the simple motion estimation unit, if the first encoding cost for forward prediction and the first encoding cost for backward prediction are the same, both are selected, and if different, the code is Selecting only the one with the smaller encoding cost and calculating the second encoding cost;
Encoding mode determination device. When deriving the first coding cost, the simple motion estimation unit performs simple motion estimation in a plurality of picture reference directions for each small block obtained by each coding mode, and reduces the first coding cost. Next, for each small block, a picture reference direction with the low first coding cost is selected, and the first coding cost of all small blocks related to the selected picture reference direction is then determined for each division method candidate. To derive the first encoding cost of the encoding mode for each candidate division method,
The encoding mode determination device according to claim 1. When deriving the first coding cost, the simple motion estimation unit performs simple motion estimation in a plurality of picture reference directions for each small block obtained by each coding mode, and reduces the first coding cost. calculated, then converted to the first cost of coding each picture reference direction of the small blocks into image blocks, the first coding cost of coding mode for each picture reference direction of each candidate dividing To derive,
The encoding mode determination device according to claim 1. The simple motion estimation in the reference direction of the plurality of pictures of the simple motion estimation unit includes only forward prediction referring to a temporally forward picture and backward prediction referring to a temporally backward picture. ,
The encoding mode determination device according to claim 2 or 3. The simple motion estimation in the plurality of picture reference directions of the simple motion estimator includes forward prediction referring to a temporally forward picture, backward prediction referring to a temporally backward picture, temporal Bi-directional prediction with reference to bi-directional pictures,
The encoding mode determination device according to claim 2 or 3. The simple motion estimation in the plurality of picture reference directions of the simple motion estimation unit includes forward prediction that refers to a temporally forward picture and backward prediction that refers to a temporally backward picture. ,
The simple motion estimator derives a coding cost when bi-directional prediction referring to a bi-directional picture is performed based on the forward prediction and the backward prediction.
The encoding mode determination device according to claim 2 or 3. The complex motion estimation unit further selects at least a part of the coding mode from the part of the coding modes based on simple motion estimation for the small block in the simple motion estimation unit;
The encoding mode determination device according to any one of claims 1 to 6 .   The complex motion estimation unit selects each coding mode from the first coding cost in ascending order and within a range in which the sum of the processing amounts of the coding modes does not exceed the allowable value of the image block. Make an estimate,
  The encoding mode determination device according to any one of claims 1 to 6. The complex motion estimator selects each coding mode in ascending order of the first coding cost, and while the sum of the processing amount of the selected coding mode does not exceed the processing margin amount , When the first coding cost is repeated, the second coding cost is derived, and when the sum of the processing amounts of the selected coding mode exceeds the processing margin amount, the selection of the coding mode is performed. Discontinue, do not perform the second encoding cost derivation process after the sum of the processing amount of the selected encoding mode exceeds the processing margin amount,
The encoding mode determination device according to claim 1 .   The processing amount is calculated to be proportional to the number of pixels of the small block.
  The encoding mode determination apparatus according to claim 8 or 9.   The processing amount is calculated to be proportional to the number of directions referring to a picture.
  The encoding mode determination apparatus according to claim 8 or 9.   The throughput is
  When the simple motion estimation of the plurality of picture reference directions of the simple motion estimation unit is executed by bidirectional prediction that refers to temporally bidirectional pictures, the number of directions referring to pictures is not counted,
  When simple motion estimation in the plurality of picture reference directions of the simple motion estimation unit is performed by other than bidirectional prediction that refers to temporally bidirectional pictures, the number of directions referring to pictures is counted.
  Calculated to be proportional to the number of reference directions obtained by
  The encoding mode determination device according to claim 11. The simple motion estimation unit, so that the processing amount of the motion estimation process is maintained substantially constant, the motion estimation scheme definitive in the simple motion estimation, is changed according to the image attribute,
Coding mode determining apparatus according to any one of claims 1 to 1 2.   The complex motion estimation unit changes a motion estimation method in the complex motion estimation according to an image attribute so that a processing amount of the motion estimation process is kept substantially constant.
The encoding mode determination apparatus according to any one of claims 1 to 12.   The simple motion estimation unit and the complex motion estimation unit are configured so that the sum of the motion estimation processing amount by the simple motion estimation unit and the processing amount of the motion estimation processing by the complex motion estimation unit is kept substantially constant. , Respectively, change the motion estimation method according to the image attributes,
The encoding mode determination apparatus according to any one of claims 1 to 12.   The image attribute is at least one of image size, encoding method (I picture, P picture, B picture), format (interlace format, progressive format), color difference format, and image motion amount. ,
The encoding mode determination device according to any one of claims 13 to 15.   The simple motion estimator and / or the complex motion estimator changes the motion estimation method so that a product of an input image size composed of image blocks, a reference picture number, and a partition size number is constant. ,
The encoding mode determination device according to claim 16.   The simple motion estimator and / or the complex motion estimator changes the motion estimation method so that a product of an input image size composed of image blocks, a reference picture number, and a partition size number is constant. ,
The encoding mode determination device according to claim 16.   The simple motion estimation unit and / or the complex motion estimation unit reduces the number of reference pictures of a B picture to be less than that of a P picture so that the processing amount of motion estimation is kept constant for each picture.
The encoding mode determination device according to claim 16.   The simple motion estimator and / or the complex motion estimator is
  (1) P picture refers to the front 4 pictures, B picture refers to the front 2 pictures and the back 2 pictures,
  (2) The P picture refers to the front 3 pictures, and the B picture refers to the front 2 pictures and the rear 1 picture.
  (3) P picture refers to the front two pictures, B picture refers to the front one and the rear one
  By any one of the above, the processing amount of motion estimation is kept constant for each picture.
The encoding mode determination device according to claim 16.   The simple motion estimator and / or the complex motion estimator is configured such that the number of partition sizes of a B picture is smaller than that of a P picture, and the processing amount of motion estimation in units of pictures is kept constant.
  The encoding mode determination device according to claim 16.   The simple motion estimator and / or the complex motion estimator is
  (1) With reference to the front one in the P picture, prediction is performed with four partition sizes of 16 pixels × 16 pixels, 16 pixels × 8 pixels, 8 pixels × 16 pixels, and 8 pixels × 8 pixels. Select any two partition sizes from the four partition sizes, and perform forward prediction and backward prediction for the two selected partition sizes, respectively.
  (2) In the P picture, the back one is referenced, and prediction is performed with four partition sizes of 16 pixels × 16 pixels, 16 pixels × 8 pixels, 8 pixels × 16 pixels, and 8 pixels × 8 pixels. Select any two partition sizes from the four partition sizes, and perform forward prediction and backward prediction for the two selected partition sizes, respectively.
  To ensure that the amount of motion estimation processing per picture is kept constant,
The encoding mode determination device according to claim 16.   The simple motion estimator and / or the complex motion estimator reduces the number of reference pictures or the number of partition sizes when the input image is an interlaced image than when the input image is a progressive image.
  The encoding mode determination device according to claim 16.   The simple motion estimator and / or the complex motion estimator is
  (1) In the case of a P picture, a progressive P picture refers to the front two frames, and an interlaced P picture refers to the front two fields.
  (2) In the case of a P picture, the progressive P picture refers to the front one frame, and has four partition sizes of 16 pixels × 16 pixels, 16 pixels × 8 pixels, 8 pixels × 16 pixels, and 8 pixels × 8 pixels. In the interlaced P picture, reference is made to the two front fields, and the four partition sizes of 16 pixels × 16 pixels, 16 pixels × 8 pixels, 8 pixels × 16 pixels, and 8 pixels × 8 pixels are respectively predicted. Select any two partition sizes from and perform prediction for each of the two selected partition sizes.
The encoding mode determination device according to claim 16.   The simple motion estimation unit and / or the complex motion estimation unit changes the number of reference pictures or the number of partition sizes according to the motion of an image.
The encoding mode determination device according to claim 16. The simple motion estimation is motion estimation with integer pixel accuracy;
The complex motion estimation is motion estimation with non-integer pixel accuracy;
The encoding mode determination device according to any one of claims 1 to 25 . The encoding mode determination device according to any one of claims 1 to 26 is included.
Integrated circuit. The encoding mode determination device according to any one of claims 1 to 26 ,
An encoding device for encoding the image block based on the encoding mode of the image block determined by the encoding mode determination device;
An image encoding device comprising: An image encoding device according to claim 28 ,
Integrated circuit. An encoding mode determination method for determining an encoding mode of an image block as at least one of a plurality of candidates,
A simple motion estimation step for deriving a first encoding cost that is an encoding cost of each encoding mode based on simple motion estimation for a small block that is a partition of an image block obtained by each encoding mode;
An encoding mode selection step of selecting a part of the encoding modes from the plurality of encoding modes based on the first encoding cost derived by the simple motion estimation step;
Complex motion estimation for deriving a second coding cost that is a coding cost of each coding mode based on a complex motion estimation for a small block obtained by at least a part of the coding modes. Steps,
A coding mode determining step for determining a coding mode of the image block based on the second coding cost derived by the complex motion estimation step;
Including
In the complex motion estimation step,
Based on the simple motion estimation in the simple motion estimation step, determine a picture reference direction in the complex motion estimation,
As a result of simple motion estimation for the small block in the simple motion estimation step, if the first encoding cost for forward prediction and the first encoding cost for backward prediction are the same, both are selected. Deriving the second encoding cost by selecting only the one with the smaller encoding cost,
Coding mode determination method. An encoding mode determination program for determining at least one encoding mode of an image block from a plurality of candidates by a computer,
The encoding mode determining program is stored in a computer.
A simple motion estimation step for deriving a first encoding cost that is an encoding cost of each encoding mode based on simple motion estimation for a small block that is a partition of an image block obtained by each encoding mode;
An encoding mode selection step of selecting a part of the encoding modes from the plurality of encoding modes based on the first encoding cost derived by the simple motion estimation step;
Complex motion estimation for deriving a second coding cost that is a coding cost of each coding mode based on a complex motion estimation for a small block obtained by at least a part of the coding modes. Steps,
A coding mode determining step for determining a coding mode of the image block based on the second coding cost derived by the complex motion estimation step;
Including
In the complex motion estimation step,
Based on the simple motion estimation in the simple motion estimation step, determine a picture reference direction in the complex motion estimation,
As a result of simple motion estimation for the small block in the simple motion estimation step, if the first encoding cost for forward prediction and the first encoding cost for backward prediction are the same, both are selected. A coding mode determination method for selecting only the one with the smaller coding cost and deriving the second coding cost is performed.
Encoding mode determination program.

Images