JP5355226B2 - Encoding apparatus, encoding method, and program - Google Patents

Description

The present invention relates to an encoding apparatus and the like that can control the amount of code based on the relative importance of a plurality of faces existing in an image.

JP 2007-201980 A

  In recent years, the number of pixels of the image sensor has increased, and products that handle high-resolution video (HD video) of 1920 pixels × 1080 pixels have been developed one after another. Digital video cameras that handle such HD video data have been commercialized. In general, a consumer digital video camera compresses and encodes moving image data in order to reduce the amount of recording data, and then records the data on a recording medium such as a flash memory or a hard disk.

  As a moving image compression system, MPEG-2 or H.264 is used. H.264 is widely known. In these moving image compression systems, efficient and high-quality data compression is realized by assigning code amounts to image areas that are visually important.

  One of the image areas that are visually important is the face of a person. Image quality can be improved by detecting a face part before encoding and assigning a larger amount of code to the detected face part.

  However, even in a region that is detected as a face in the same way, it is considered that there are visually important faces and unimportant faces. For example, when taking a picture centering on a child at an athletic meet or the like, the face of the child is a very important face, but the face of the other person shown as the background is not so important. As a method for determining whether or not the face is important, a method of calculating importance from face size information, position information, or tilt information is known (see Patent Document 1).

  The conventional method calculates the absolute importance of the face, and determines the code allocation amount according to the absolute importance. Therefore, in the conventional example, even when the importance is relatively low, the image quality may be targeted, and as a result, the coding efficiency may be impaired as a whole.

An object of the present invention is to eliminate such inconveniences and to determine an assigned code amount based on the relative importance of a plurality of faces existing in an image .

Engaging Ru marks Goka apparatus of the present invention is an encoding apparatus for encoding image data, a detecting means for detecting a face region from the image data, a plurality of face areas detected by said detecting means The first exclusion means for excluding a face area having a size smaller than the first size from the above, and the number of face areas not excluded by the first exclusion means is two or more. The face with the highest importance is determined from the faces not excluded by the exclusion means, the second size is determined based on the size of the face with the highest importance, and the face is excluded by the first exclusion means A second exclusion means for excluding faces smaller than the second size from the faces that have not been selected, the size of the most important face, and the face that has not been excluded by the second exclusion means Based on size and above Importance determining means for determining the importance of the face not excluded by the two exclusion means, and the importance of the face not excluded by the second exclusion means by the second exclusion means And a control means for controlling a quantization step size for a face area that has not been excluded .
Engaging Ru marks Goka method according to the present invention, there is provided a coding method for encoding image data, a detection step of detecting a face region from the image data, a plurality of face areas detected in said detecting step A first excluding step of excluding a face area having a size smaller than the first size from the above, and the number of face areas not excluded in the first excluding step is two or more. The face having the highest importance is determined from the faces not excluded in the exclusion step, the second size is determined based on the size of the face having the highest importance, and the face is excluded in the first exclusion step. A second exclusion step of excluding a face smaller than the second size from the faces that have not been processed, the size of the most important face, and the face that has not been excluded in the second exclusion step The importance level determining step for determining the importance level for the face that is not excluded in the second exclusion step, and the importance level for the face that is not excluded in the second exclusion step. And a control step for controlling a quantization step size for a face region that is not excluded in the second exclusion step .
One of the programs according to the present invention is a program for causing a computer to function as an encoding device for encoding image data, the detection means for detecting a face area from the image data, and the detection means A first exclusion means for excluding a face area having a size smaller than the first size from the plurality of detected face areas; and the number of face areas not excluded by the first exclusion means is two or more. In some cases, the most important face is determined from the faces that are not excluded by the first exclusion means, the second size is determined based on the size of the most important face, A second exclusion means for excluding faces smaller than the second size from the faces not excluded by the first exclusion means; the size of the most important face; and the second exclusion means Therefore, based on the size of the face that has not been excluded, importance determining means for determining the importance of the face that has not been excluded by the second excluding means, and the face that has not been excluded by the second excluding means Is a program for causing the computer to function as a control unit that controls a quantization step size for a face area that is not excluded by the second exclusion unit.
One of the programs according to the present invention is a program for causing a computer to execute an encoding method for encoding image data. In the detection step for detecting a face area from the image data, and in the detection step, A first exclusion step of excluding a face region having a size smaller than the first size from a plurality of detected face regions; and the number of face regions not excluded in the first exclusion step is two or more. In some cases, the most important face is determined from the faces not excluded in the first exclusion step, a second size is determined based on the size of the most important face, A second excluding step of excluding faces smaller than the second size from the faces not excluded in the first excluding step; and the size of the most important face. And an importance level determining step for determining an importance level for the face not excluded in the second exclusion step based on the size of the face not excluded in the second exclusion step; For causing the computer to execute a control step of controlling a quantization step size for a face region not excluded in the second exclusion step based on the importance of the face not excluded in the exclusion step It is a program.

According to the present invention, the allocated code amount can be determined based on the relative importance of a plurality of faces existing in an image .

It is a schematic block diagram of one Example of this invention. It is a schematic block diagram of the face importance degree determination apparatus of the present embodiment. It is an operation | movement flowchart of the face importance degree determination process of a present Example. It is an example of a face importance degree determination. It is another image example of face importance determination. It is another operation | movement flowchart of a face importance determination process.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram of a schematic configuration of an image encoding apparatus according to an embodiment of the present invention. Image data taken by a camera (not shown) is input to the input terminal 10. The face importance level determination device 12 detects a human face from input image data from an input terminal, and determines the importance level of each face. Then, the face importance level determination device 12 supplies the quantization control unit 14 with a face detection result that excludes an unimportant face according to the determined face importance level.

  Input image data from the input terminal 10 is stored in the frame buffer 16 in units of frames. The reference frame buffer 20 stores reference image data for interframe predictive coding. The motion prediction unit 18 performs block matching between the input image data of the frame buffer 16 and the reference image data of the reference frame buffer 20, and calculates a motion vector. The motion prediction unit 18 generates predicted image data in which the reference image data is moved within the screen according to the calculated motion vector, calculates a difference (difference image data) between the input image data and the predicted image data, and the orthogonal transform unit 22. To supply. The orthogonal transform unit 22 performs discrete cosine transform on the difference image data and supplies transform coefficients to the quantization unit 24. The quantization unit 24 quantizes the transform coefficient from the orthogonal transform unit 22 with a quantization step size specified by the quantization control unit 14. The quantized transform coefficient is supplied to the entropy encoding unit 26 and is supplied to the inverse quantization unit 28 for creating a locally decoded image.

  The entropy encoding unit 26 performs variable length encoding on the quantized transform coefficient by zigzag scanning, alternate scanning, or the like to generate code data. The entropy encoding unit 26 also adds a variable length encoded encoding method information such as a motion vector, a quantization step size, and macroblock division information to the code data to generate an encoded stream. The entropy encoding unit 26 further calculates the generated code amount for each macroblock and sends it to the quantization control unit 14.

  The quantization control unit 14 refers to the face detection information from the face importance degree determination device 12 and applies different encoding according to whether or not the macroblock to be quantized is a face region. Specifically, when the macroblock is not a face region, the quantization control unit 14 determines the quantization step size that becomes the target code amount according to the generated code amount from the entropy encoding unit 26. Set to. When the macroblock is a face region, the quantization control unit 14 sets a quantization step size finer than that of normal processing in the quantization unit 24 according to the importance of the face. By making the quantization step size finer, a larger amount of code is allocated to the face area, and image quality deterioration can be suppressed.

  The inverse quantization unit 28 inversely quantizes the quantized transform coefficient from the quantization unit 24, generates a transform coefficient for local decoding, and supplies the transform coefficient to the inverse orthogonal transform unit 30. The inverse orthogonal transform unit 30 performs inverse discrete cosine transform on the transform coefficient from the inverse quantization unit 28, generates difference image data, and supplies the difference image data to the motion compensation unit 32. The motion compensation unit 32 reads the reference image at the motion vector position from the reference frame buffer 20 and adds the difference image data. The image data from the motion compensation unit 32 is output to the deblocking filter 34. The deblocking filter 34 applies a deblocking filter to the image data from the motion compensation unit 32. The image data after the deblocking filter is stored in the reference frame buffer 20 as locally decoded image data.

  FIG. 2 shows a schematic block diagram of the face importance degree determination device 12. The face detection unit 40 detects a face area from the input image data, and supplies the number of detected face areas and the size information of each face to the most important face determination unit 42 and the face importance degree calculation unit 44. The most important face determination unit 42 selects one face having the maximum size from the detected face size information as the most important face, and supplies the face size as a face reference size to the face importance degree calculation unit 44. The face importance degree calculation unit 44 compares the face reference size with the sizes of other faces, and calculates the face importance degree based on the relative size with respect to the most important face. The face importance calculation unit 44 also determines that a face having a size smaller than a certain size relative to the size of the most important face is not an important face, and excludes it from the face detection result.

  FIG. 3 shows an operation flowchart of the face importance degree determination device 12. First, the face detection unit 40 detects a region that is a face from the input image (S1). If no face is detected, or if only one face is detected, the process ends as it is as a result of face detection (S2).

  When there are a plurality of detected faces, the most important face determination unit 42 compares the sizes of all the faces, considers that the face with the largest size is the closest face, and regards this as the most important face. Judged to be a face. This face size is set as the face reference size (S3). Next, the face importance degree calculation unit 44 acquires the size (size) of a face other than the face determined to be the most important (S4). If the size (size) of the face other than the most important is smaller than a certain ratio (α) with respect to the reference face reference size (base_size) (S5), it is determined that the face is not important and is excluded from the face detection result ( S6).

For example, if the base face size is base_size, the size of the face to be judged is size, and the ratio of excluded sizes is α (<1),
size <base_size × α
In this case, it is excluded from the face detection result. On the other hand,
size> = base_size × α
In this case, the face detection result is maintained.

  The size referred to here may be the area of the face, and any size may be used as long as it represents the size of the face, such as the length of the face in the vertical and horizontal directions or the length of the diagonal line. Good.

For the faces that are not excluded, the face importance degree calculation unit 44 calculates the face importance degree based on the relative size from the face reference size (S7). For example, when the face importance is evaluated from 1 to 10 (10 is the most important), the face importance is calculated by the following formula (1). That is,
Face importance = [10 × (size − base_size × α) / (base_size × (1-α))] (1)
Here, [] indicates an operation rounded up to an integer.

  Steps S4 to S7 are repeated for all faces (S8). By doing so, faces that are detected as faces but are not relatively important can be excluded, and relative face importance can be calculated for the remaining faces.

  FIG. 4 shows a face detection result of the face detection unit 40 and an example of exclusion by the face importance degree calculation unit 44. In FIG. 4, it is assumed that, as a criterion for exclusion from face detection, when the ratio of the size from the reference face size is smaller than 1/2, it is excluded from the face detection result. That is, α = ½.

In step S1, a face in the input image is detected. In FIG. 4, two faces, a face 50 and a face 52, are detected. Here, the size of the face 50 is size1, the size of the face 52 is size2,
size1> size2 × 2 (2)
Is assumed to hold.

  Since there are a plurality of faces in the image (S2), the largest face 50 is determined as the most important face, and its size size1 is set as the face reference size (S3). That is, base_size = size1.

Next, the size of the face 52 other than the reference face is acquired (S4). That is, size = size2. To determine whether to exclude from face detection results, when comparing size with base_size × α,
size <base_size × α
Is established. From this result, the face 52 is determined to be relatively unimportant, and is excluded from the face detection result. As a result, in the image example shown in FIG. 4, the face detection result is only the face 50 with high importance. Since the face 50 is the most important face, the face importance degree is evaluated as 10.

FIG. 5 shows an example of face detection determination in the case of a group photo. The condition excluded from the face detection is set to α = ½ as in the example of FIG. In the example illustrated in FIG. 5, the face detection unit 40 detects four faces 60, 62, 64, 66. The size of these faces 60-66 is size3, size4, size5, size6, respectively.
size3> size4 = size5 = size6 = size2 = size3 × 9/10 (3)
Is assumed to hold. The sizes of the faces 62, 64, and 66 are assumed to be the same as those of the face 52 described with reference to FIG.

The sizes of the five faces 60 to 66 are compared, and the size size3 of the face 60 that is the largest face is set as the face reference size base_size. The size size4 of the face 62, which is one of the faces other than the reference face, is acquired and substituted into size. Comparing size with base_size × α,
size> = base_size × α
Is established. From this condition, the face 62 is determined to be relatively important and is not excluded from the face detection result.

Next, the face importance degree is calculated based on the formula (1). From the relationship of α = 1/2 and equation (3), substituting size = base_size × 9/10 into equation (1)
Face importance = [(((base_size × 9/10 − base_size × 1/2) / (base_size × (1-1 / 2))) × 10]
= 8
The result is obtained.

  Similar processing is performed for the faces 64 and 66. As a result, the faces 64 and 66 are not excluded from the face detection results, and the importance of these faces is calculated as 8.

  As a result of the above, the face importance degree determination device 12 determines that all the faces 60, 62, 64, 66 are relatively important faces.

Comparing the example shown in FIG. 4 with the example shown in FIG. 5, the absolute face size is the same for the faces 52, 62, 64 and 66. However, due to the difference in relative size, the face 52 is excluded from the face detection result, and the faces 62, 64, and 66 are left as important faces in the face detection result. In the example shown in FIG. 4, it is determined that the front person is important and the back person is not important. In the example shown in FIG. 5, although the absolute face size is small, it is determined that the importance of all the faces is high.

  In this way, by determining the importance according to the relative face size regardless of the absolute face size, relatively unimportant faces can be excluded, and only important faces can be extracted. The important face area is encoded with a smaller quantization step size according to the importance of the face, and the face area excluded from the face detection result is encoded under the same conditions as other image parts. To do. Thereby, high image quality can be realized while maintaining encoding efficiency.

  Although the embodiment according to the hardware configuration has been described, it is obvious that the present invention can be realized in part or in whole by computer software.

  In addition to excluding a face having a relatively small size with respect to the face reference size, a face having an absolutely small size may be excluded. FIG. 6 shows another operation flowchart of the face importance degree determination device 12 modified as described above.

  First, the face detection unit 40 detects a region that is a face from the input image (S21). For the detected face area, those smaller than the minimum size minimum_size are excluded from the face detection result (S22). If no faces remain after this exclusion, or if only one face remains, the process ends as it is as a result of face detection (S23).

  When there are a plurality of remaining faces (S23), the most important face determination unit 42 determines that the largest size face is the most important face from the remaining faces. This face size is set as the face reference size (S24).

  The face importance degree calculation unit 44 acquires the size (size) of a face other than the face determined to be the most important (S25). When the size (size) of the face other than the most important is smaller than a fixed ratio (α) with respect to the face reference size (base_size) (S26), it is determined that the face is not important and is excluded from the face detection result (S27). When the size of the face other than the most important (size) is equal to or greater than a certain ratio (α) to the reference face reference size (base_size) (S26), the face importance is calculated based on the relative size from the face reference size. (S28).

  Steps S25 to S28 are repeated for all faces (S29).

  According to the processing shown in FIG. 6, since the non-important face is excluded from both the relative size from the size of the most important face and the absolute size, the encoding efficiency and the image quality improvement of the important face area are improved. Can be further improved.

DESCRIPTION OF SYMBOLS 12 Face importance determination apparatus 14 Quantization control part 16 Frame buffer 18 Motion estimation part 22 Orthogonal transformation part 24 Quantization part 26 Entropy encoding part

Claims (8)

An encoding device for encoding image data ,
Detecting means for detecting a face region from the image data;
First exclusion means for excluding a face area having a size smaller than the first size from a plurality of face areas detected by the detection means;
When the number of face areas that are not excluded by the first exclusion means is two or more, the face having the highest importance is determined from the faces that are not excluded by the first exclusion means, Second exclusion means for determining a second size based on the size of the face having high importance, and excluding faces smaller than the second size from the faces not excluded by the first exclusion means When,
The importance of determining the importance of the face not excluded by the second exclusion means based on the size of the face having the highest importance and the size of the face not excluded by the second exclusion means A degree determination means;
Control means for controlling a quantization step size for a face area not excluded by the second exclusion means based on the importance of the face not excluded by the second exclusion means;
Marks Goka device you further comprising a. The second exclusion means determines a face selected as the face of the largest size from the faces not excluded by the first exclusion means as the face with the highest importance. Item 4. The encoding device according to Item 1. 3. The encoding apparatus according to claim 1, wherein the second size is determined based on a size of the most important face and a predetermined ratio. 4. The significance decision means, according to the size of not excluded face is smaller than the size of the face of high the least important by the second exclusion means, not excluded by the second exclusion means face It marks Goka device according to any one of claims 1 to 3, characterized in the lower that importance for. Before SL control means, the importance of not excluded face by said second exclusion means according higher due, the quantization step size for the region of the not excluded by the second exclusion means face marks Goka device according to claim 1 in any one of 4, characterized in that to reduce. An encoding method for encoding image data , comprising:
A detection step of detecting a face region from the image data;
A first excluding step of excluding a face area having a size smaller than the first size from the plurality of face areas detected in the detecting step;
When the number of face regions that are not excluded in the first exclusion step is 2 or more, a face having the highest importance is determined from the faces that are not excluded in the first exclusion step, A second exclusion step of determining a second size based on the size of the face of high importance and excluding faces smaller than the second size from the faces not excluded in the first exclusion step When,
The importance of determining the importance of the face not excluded in the second exclusion step based on the size of the most important face and the size of the face not excluded in the second exclusion step A degree determination step;
A control step for controlling a quantization step size for a face region not excluded in the second exclusion step based on the importance of the face not excluded in the second exclusion step;
Marks Goka how to, comprising a. A program for causing a computer to function as an encoding device for encoding image data,
Detecting means for detecting a face region from the image data;
First exclusion means for excluding a face area having a size smaller than the first size from a plurality of face areas detected by the detection means;
When the number of face areas that are not excluded by the first exclusion means is two or more, the face having the highest importance is determined from the faces that are not excluded by the first exclusion means, Second exclusion means for determining a second size based on the size of the face having high importance, and excluding faces smaller than the second size from the faces not excluded by the first exclusion means When,
The importance of determining the importance of the face not excluded by the second exclusion means based on the size of the face having the highest importance and the size of the face not excluded by the second exclusion means A degree determination means;
Control means for controlling the quantization step size for the face area not excluded by the second exclusion means based on the importance of the face not excluded by the second exclusion means
A program for causing the computer to function as A program for causing a computer to execute an encoding method for encoding image data,
A detection step of detecting a face region from the image data;
A first excluding step of excluding a face area having a size smaller than the first size from the plurality of face areas detected in the detecting step;
When the number of face regions that are not excluded in the first exclusion step is 2 or more, a face having the highest importance is determined from the faces that are not excluded in the first exclusion step, A second exclusion step of determining a second size based on the size of the face of high importance and excluding faces smaller than the second size from the faces not excluded in the first exclusion step When,
The importance of determining the importance of the face not excluded in the second exclusion step based on the size of the most important face and the size of the face not excluded in the second exclusion step A degree determination step;
  A control step for controlling a quantization step size for a face region not excluded in the second exclusion step based on the importance of the face not excluded in the second exclusion step.
For causing the computer to execute.

Images