JP5111412B2 - Image display apparatus and image processing method - Google Patents

Description

  The present invention relates to an image display apparatus and an image processing method for dividing image data into a plurality of areas and changing the correction amount of the image data for each area.

  Image data such as television includes various objects such as people, vehicles, sky, rivers, buildings, etc., and image data for each object or for each area such as foreground and background to improve image quality. There is a need to correct this.

  Patent Document 1 proposes a technique for dividing an image into a plurality of regions by dividing an image using luminance color information and texture information and then integrating the regions using luminance color information. Yes.

JP 2004-258752 A

  However, when areas are integrated using luminance color information as described above, an area including an edge and an area not including an edge are integrated, and contrast correction, noise reduction, or the like of image data is performed using the area. , Noise and blurring occur when displaying moving images.

  In addition, detection of edge information in the horizontal direction, vertical direction, oblique direction, etc. is particularly important in dividing an image into regions. In Patent Document 1 of the prior art, there is a problem that the region division is not appropriately performed depending on the edge information calculation method, and the region division cannot be used for image quality correction.

  In view of the above problems, an object of the present invention is to perform image quality correction by performing region division more suitably, and to improve the image quality of images more suitably.

  One embodiment of the present invention may be configured as described in the claims, for example.

  According to the present invention, it is possible to perform image quality correction by performing region division more preferably, and to improve the image quality of the image more suitably.

1 is a block diagram showing an image display device according to an embodiment of the present invention. The figure explaining an example of operation | movement of the block extraction part which concerns on one Example of this invention. The figure explaining an example of the orthogonal transformation result of the orthogonal transformation part which concerns on one Example of this invention. The figure explaining an example of the orthogonal transformation coefficient grouping of the orthogonal transformation coefficient characteristic determination part which concerns on one Example of this invention. The flowchart which shows operation | movement of the orthogonal transformation coefficient characteristic determination part which concerns on one Example of this invention. The flowchart which shows operation | movement of the brightness | luminance determination part which concerns on one Example of this invention. The figure explaining an example of the brightness | luminance threshold value calculation of the brightness | luminance determination part which concerns on one Example of this invention. The figure explaining an example of the area | region determination of the area | region determination part which concerns on one Example of this invention. The figure explaining an example of contrast correction concerning one example of the present invention. The figure explaining an example of the combination of contrast correction of the image quality correction part which concerns on one Example of this invention. The figure explaining an example of the noise reduction of the image quality correction | amendment part of this invention which concerns on one Example of this invention.

  Embodiments of the present invention will be described below with reference to the drawings. In each drawing, the same code | symbol was provided to the common component.

  FIG. 1 is a block diagram showing an image display apparatus according to a first embodiment of the present invention. The configuration of the image display device 1 includes an input unit 10, a block extraction unit 11, an orthogonal transform unit 12, an orthogonal transform coefficient feature determination unit 13, a luminance determination unit 14, a region determination unit 15, an image quality correction unit 16, an image display unit 17, A control unit 20 is provided.

  The operation of each part will be described. Moving image data is input to the input unit 10. For example, the input unit 10 includes an image input terminal, a network connection terminal, and the like, or may be a TV broadcast tuner. The input unit 10 performs image preprocessing on the input moving image data as necessary, and outputs the processed display data to the block extraction unit 11 and the luminance determination unit 14.

  The block extraction unit 11 extracts pixels belonging to a block of a predetermined size (hereinafter referred to as a pixel block) from the input image.

  The orthogonal transform unit 12 performs orthogonal transform on the pixel block by Hadamard transform, and calculates orthogonal transform coefficients, that is, Hadamard transform coefficients. For the Hadamard transform, a Hadamard transform matrix or a fast Walsh-Hadamard transform may be used. Regarding the Hadamard transform technique, an existing technique may be used.

  The orthogonal transform coefficient feature determination unit 13 divides the Hadamard transform coefficients obtained by the orthogonal transform unit 12 into a plurality of groups, obtains a sum Si of absolute values of each Hadamard transform coefficient for each group, and calculates a sum Si of each group. By comparing, the feature of the image, that is, the orthogonal transformation coefficient feature amount m of the target pixel in the pixel block is determined for each pixel.

  The luminance determination unit 14 determines the luminance value of each pixel input from the input unit 10 and determines a luminance feature amount n indicating which luminance range each pixel is included in.

  In the region determination unit 15, the region to which each pixel belongs is determined for each pixel using the orthogonal transformation coefficient feature amount m obtained by the orthogonal transformation coefficient feature determination unit 13 and the luminance feature amount n obtained by the luminance determination unit 14. Determine the type of area.

  The image quality correction unit 16 prepares in advance a plurality of correction equations y = ax + b having several types of inclinations a and intercepts b, correction equations for e (the natural logarithm base) to the aX power, and the like. A plurality of correction formulas are combined for each region, and the image data is corrected for each region.

  The image display unit 17 displays an image based on the image data corrected by the image correction unit 16.

  The control unit 20 is connected to each element in the display device. The operation of each element of the display device operates according to the autonomous operation of each component described above or an instruction from the control unit 20.

  As described above, in the image display device 1 according to the present embodiment, the orthogonal transform coefficient feature quantity m obtained by the orthogonal transform coefficient feature judgment unit 13 and the luminance feature quantity n obtained by the brightness judgment unit 14 are used for each pixel. The area to which the pixel belongs is determined, and the image correction unit 16 corrects the image data using a different correction formula for each area. Hereinafter, the configuration and operation of each unit will be described in detail.

  FIG. 2 is a diagram for explaining an example of the operation of the block extraction unit 12. In FIG. 2, the horizontal axis indicates the horizontal direction of the image, and the vertical axis indicates the vertical direction of the image. Accordingly, when the horizontal axis moves one to the right, the coordinate value Xi increases by one. Further, when the vertical axis moves downward one, the coordinate value Yi increases by one. As shown in FIG. 2, the block extraction unit 12 extracts a pixel block having the target pixel as a base point from the input image. Also, the next pixel block is extracted by moving the target pixel rightward by one. However, any pixel in the pixel block may be used as the target pixel.

  The orthogonal transform unit 12 performs orthogonal transform on the pixel block obtained by the block extraction unit 12 by Hadamard transform, and calculates an orthogonal transform coefficient shown in FIG. 3, that is, a Hadamard transform coefficient Dij.

  FIG. 5 is a flowchart showing the operation of the orthogonal transform coefficient feature determination unit 13.

  The orthogonal transform coefficient feature determination unit 13 determines the image feature, that is, the orthogonal transform coefficient feature amount m of the target pixel in the pixel block for each pixel. The procedure will be described.

  In step 501, the orthogonal transformation coefficient Dij belonging to the pixel block of the target pixel obtained by the orthogonal transformation unit 12 is extracted.

  In step 502, the absolute value Fij of each orthogonal transform coefficient Dij is calculated.

  In step 503, the absolute values Fij of the Hadamard transform coefficients are divided into groups as shown in FIG. 4, and the sum Si (i = 1 to 4) of the absolute values Fij is calculated for each group Gi.

  In step 504, Si is compared with each threshold value. That is, three comparisons of S2> threshold 1, S3> threshold 2, and S4> threshold 3 are performed. If at least one of the three comparisons satisfies the condition, the process proceeds to step 505. In step 505, 1 is set to the feature of the image, that is, the orthogonal transformation coefficient feature amount m. If none of the conditions are satisfied under the above conditions, the process proceeds to step 506. In step 506, 0 is set to the orthogonal transformation coefficient feature value m.

  In step 507, the obtained orthogonal transform coefficient feature value m is output.

  In step 508, steps 501 to 507 are repeated until the orthogonal transformation coefficient feature quantity m is calculated for all pixels in the target frame.

  As described above, the orthogonal transform coefficient feature determination unit 13 according to the present embodiment determines the feature of each target pixel using the orthogonal transform coefficient in the pixel block including the target pixel, and the orthogonal transform coefficient for all the pixels of the target frame. The feature amount m is calculated. Therefore, it is possible to detect more edge information in the horizontal direction, the vertical direction, the oblique direction, etc. by placing emphasis on the detection of the medium-frequency components of the groups G2 to G4, not the low-frequency components of the group G1. .

  FIG. 6 is a flowchart showing the operation of the luminance determination unit 14.

  In step 601, the luminance value Yij of the target pixel obtained by the input unit 10 is acquired.

  In step 602, it is determined whether the luminance value Yij <the luminance threshold value 1 is satisfied. If the condition is satisfied, the process proceeds to step 603, and the luminance feature amount n is set to 0 (low luminance area). If the condition is not satisfied, the process proceeds to step 604.

  In step 604, it is determined whether the luminance value Yij <the luminance threshold value 2 is satisfied. If the condition is satisfied, the process proceeds to step 605 and the luminance feature quantity n is set to 1 (medium luminance area). If the condition is not satisfied, the process proceeds to step 606, where the luminance feature quantity n is set to 2 (high luminance area).

  In step 607, the luminance feature amount n is output.

  In step 608, steps 601 to 607 are repeated until the luminance feature amount n is calculated for all pixels in the target frame.

  As described above, the luminance determination unit 14 according to the present exemplary embodiment determines the range of the luminance value of each target pixel, and calculates the luminance feature amount n for all the pixels of the target frame. Here, the luminance thresholds 1 and 2 are used to classify each pixel in the target frame into three regions, a low luminance region, a medium luminance region, and a high luminance region. Each luminance threshold value may be the same fixed value for each frame, or may be a variable value for each frame by creating a luminance histogram for each frame. For example, in the case of a fixed value, 48 is set for the luminance threshold 1 and 120 is set for the luminance threshold 2. Next, an example of how to obtain the variable value of each luminance threshold will be described with reference to FIG. For each target frame, a luminance histogram obtained by obtaining the count number N of each luminance value shown in FIG. Here, in FIGS. 7A and 7B, the horizontal axis represents the luminance value Yij, and the vertical axis represents the count number. A is the total number of pixels of the target frame, “A * 2/3” is the number of 2/3 of the total number of pixels, “A * 1/3” is the number of 1/3 of the total number of pixels, and a to i Represents the count number N of each luminance value. Next, as shown in FIG. 7B, the cumulative number of counts of each luminance value is sequentially calculated from the minimum luminance value. First, the cumulative value “a + b” of the count number “a” of the minimum luminance value “30” and the count number “b” of the next largest luminance value is calculated. Similarly, the cumulative value of the count number is sequentially calculated up to the count number i of the maximum luminance value. In the process of accumulating the count number, if the accumulated value of the obtained count number exceeds “A * 1/3” of the determination value 1, the luminance value having the count number at that time is set as the luminance threshold value 1. That is, in FIG. 7B, the luminance value 60 having the count number d is set as the luminance threshold value 1. When the accumulated value of the obtained count number exceeds “A * 2/3” of the determination value 2, the luminance value having the count number at that time is set as the luminance threshold value 2. That is, in FIG. 7B, the luminance value 120 having the count number g is set as the luminance threshold value 2. This makes it possible to calculate each luminance threshold variable for each frame, and to determine the luminance feature of each pixel using each obtained luminance threshold. In this embodiment, the case where the number of luminance threshold values and the number of determination values is two has been described, but any number of luminance threshold values and determination values may be set as long as they are within the range of the number of luminance values in the target frame. .

  FIG. 8 shows the contents of the area determination performed by the area determination unit 15. As shown in FIG. 8, the region determination unit 15 uses a combination of the orthogonal transformation coefficient feature amount m obtained by the orthogonal transformation coefficient feature determination unit 13 and the luminance feature amount n obtained by the luminance determination unit 14 for each pixel. And determine the type of area. In FIG. 8, the orthogonal transformation coefficient feature amount m represents the feature of the frequency component. That is, when m = 0, it indicates that there are few middle and high frequency components, and when m = 1, it indicates that there are many middle and high frequency components. The luminance feature amount n represents which luminance region each pixel belongs to. That is, when n = 0, the target pixel belongs to the low luminance region, when n = 1, the target pixel belongs to the middle luminance region, and when n = 2, the target pixel belongs to the high luminance region. Represents. As shown in FIG. 8, for example, when m = 0 and n = 0, the target pixel area number AN is 0, and when m = 0 and n = 1, the target pixel area number AN is 1, m = 0. When n = 2, 2 is set to the area number AN of the target pixel. Further, when m = 1 and n = 0, the target pixel area number AN is 3, when m = 1 and n = 1, the target pixel area number AN is 4, m = 1 and n = 2, 5 is set in the area number AN of the target pixel. That is, by using the luminance feature amount n, the region of the orthogonal transform coefficient feature amount m = 0 (low mid-high frequency component) is divided into three types, and the orthogonal transform coefficient feature amount m = 1 (high mid-high frequency component). The area is divided into three types. As a result, each pixel of the target frame can be divided into six types of regions.

  The operation of the image quality correction unit 16 will be described with reference to FIG. 9A and 9B, the horizontal axis represents the input luminance value yin, and the vertical axis represents the output luminance value yout. FIG. 9A shows a polygonal line for contrast correction that satisfies the relations a1> b1 and a2 <b2. When the luminance value is corrected using the polygonal line in FIG. 9A, dark pixels are corrected darker and bright pixels are corrected brighter, so that the contrast is improved.

  FIG. 9B shows a polygonal line for contrast correction that satisfies the relationship of a1 <b1, a2> b2. When the luminance value is corrected using the broken line in FIG. 9B, dark pixels are corrected brighter and bright pixels are corrected darker. FIG. 9B is used when correcting a dark and invisible region so that it can be seen. The dotted lines in FIGS. 9A and 9B represent straight lines in which the input luminance value yin and the output luminance value yout are equal. By changing the magnitude relationship between a1 and b1 and a2 and b2, various broken lines can be expressed. As shown in FIG. 9C, when a1 and b1 are equal and a2 and b2 are equal, that is, when the input luminance value is not corrected, yin and yout are equal, and the same straight line as the dotted line can be expressed.

  The image quality correction unit 16 assigns various contrasts by assigning FIG. 9A and FIG. 9B in which each coefficient (a1, a2, b1, b2) is changed for each area number AN obtained by the area determination unit 15. Correction can be realized. 9B is assigned to the area number AN = 0, the polygonal line in FIG. 9A is assigned to the area number AN = 1, and the polygonal line in FIG. 9A is assigned to the area number AN = 2. This is a polygonal line for contrast correction shown in (a), that is, a line obtained by combining several straight lines. 9B is assigned to the area number AN = 3, the polygonal line in FIG. 9A is assigned to the area number AN = 4, and the polygonal line in FIG. 9C is assigned to the area number AN = 5. It becomes a polygonal line for contrast correction shown in (b). However, the coefficients (a1, a2, b1, b2) of FIGS. 9A and 9B assigned to each area number AN may be the same or different for each area number. Therefore, a suitable contrast correction can be realized for each region.

  According to the present embodiment, the target frame can be appropriately divided into regions by using the orthogonal transform coefficient feature and the luminance feature.

  Further, by using the orthogonal transform coefficient feature of Hadamard transform, it is possible to divide the image into regions with a small circuit scale.

  In addition, by changing the contrast correction amount for each divided area, it is possible to reduce noise generated when displaying a moving image and realize a suitable contrast correction for each area.

  That is, according to the present embodiment, the target frame is divided into regions using the orthogonal transform coefficient feature and the luminance feature, and the contrast correction amount is changed for each region. A suitable contrast correction can be realized.

  Therefore, according to the present embodiment, it is possible to improve the image quality of the image more suitably.

  The image display apparatus according to the second embodiment has the same configuration as that of FIG. 1 of the first embodiment, but the operation of the image quality correction unit 16 is changed. Therefore, the description of the same configuration is omitted here.

  In human visual characteristics, there is a characteristic that the visual sensitivity of noise of low frequency components (that is, flat portions) is higher than noise of high frequency components. Further, among the flat portions, there is a characteristic that noise is more conspicuous in a bright region than in a dark region.

  When noise reduction is performed on the entire surface of the target frame, noise is reduced. However, on the other hand, there is a problem in that even an area such as a dark flat portion where an edge or noise is not noticeable is blurred. Therefore, the image quality correction unit 16 of the present embodiment changes the correction amount and intensity by noise reduction for each region.

  FIG. 11 is a diagram for explaining an example of the operation of the image quality correction unit 16. FIG. 11A is a diagram illustrating an example in which the target frame is divided into six regions by the region determination unit 15. Here, AN indicates an area number. FIG. 11B is a diagram illustrating an example of the strength of noise reduction set for each region. Here, NR indicates noise reduction. As the noise reduction technique, an existing technique may be used. For example, a bilateral filter may be used. For the bilateral filter, an existing technique may be used. As shown in FIG. 11B, the application ratio of NR is set to be weak for the region of AN = 0 and AN = 3 to 5, and the application rate of NR is set to be strong for the region of AN = 1 to 2. Set. As a result, it is possible to suppress blurring in areas such as edges and dark flat portions, and to reduce noise in other areas where noise is conspicuous. Therefore, suitable noise reduction can be realized for each region. For example, a noise reduction effect can be strengthened in a region where noise is conspicuous compared to other regions.

  In this embodiment, two types of strength are set as the noise reduction setting amount, but a plurality of strength setting amounts may be used. In addition, combinations of noise reduction strengths set for each region may be combinations other than those in FIG. 11B as long as the same effect can be obtained.

  According to the present embodiment, it is possible to increase the noise reduction effect in the area where noise is conspicuous, as compared with other areas, while suppressing the blur of the area.

  Further, according to the present embodiment, in addition to the effects of the first embodiment, by changing the noise reduction amount for each region, it is possible to suppress the blurring of the region and suppress the noise more effectively than the other regions in the region where the noise is conspicuous. Can be strengthened.

  Therefore, according to the present embodiment, it is possible to improve the image quality of the image more suitably.

  In addition, about each Example demonstrated above, the following deformation | transformation is possible.

  Although the example using Hadamard transform as orthogonal transform in the orthogonal transform unit 12 has been described, orthogonal transform such as discrete Fourier transform, discrete cosine transform, discrete Hartley transform, Karhunen-Reeb transform, wavelet transform, etc. may be used.

  In the orthogonal transform coefficient feature determination unit 13, the example in which the orthogonal transform coefficients are divided into four groups and compared with the threshold value has been described, but the threshold determination may be performed in any number of groups as long as there are two or more groups.

  Moreover, although the example which calculates the two orthogonal transformation coefficient feature-values in the orthogonal transformation coefficient feature determination part 13 was demonstrated, as long as it is two or more, you may obtain | require how many orthogonal transformation coefficient feature-values.

  Moreover, although the example which calculates three brightness | luminance feature-values in the brightness | luminance determination part 14 was demonstrated, you may obtain | require brightness | luminance feature-values other than this. Alternatively, the region division may be performed using only the orthogonal transform coefficient feature amount without using the luminance feature amount.

  In addition, the luminance determination unit 14 has described an example in which the luminance thresholds 1 and 2 are calculated using the cumulative value of the count number N of each luminance value. However, the luminance having the largest count number among the low, medium, and high luminances. The luminance thresholds 1 and 2 may be obtained using the values, and the same effect is obtained.

  Further, any combination of the embodiments described above, such as each drawing and each method, can be an embodiment of the present invention.

  DESCRIPTION OF SYMBOLS 1 ... Image display apparatus, 10 ... Input part, 11 ... Block extraction part, 12 ... Orthogonal transformation part, 13 ... Orthogonal transformation coefficient characteristic determination part, 14 ... Luminance determination part, 15 ... Area | region determination part, 16 ... Image quality correction part, 17: Image display unit, 20: Control unit.

Claims (6)

An input unit for inputting image data;
A block extractor for extracting a part of the image data as a pixel block of a predetermined size;
An orthogonal transform unit that calculates an orthogonal transform coefficient for each of the pixel blocks extracted by the block extraction unit;
An orthogonal transform coefficient feature determination unit that calculates a first image feature amount based on the orthogonal transform coefficient;
A luminance determination unit that calculates a second image feature amount based on the luminance value of each pixel in the image data;
A region determination unit that determines the type of region for each pixel based on the first image feature amount obtained by the orthogonal transform coefficient feature determination unit and the second image feature amount obtained by the luminance determination unit;
An image quality correction unit that performs image quality correction processing on the image data of each region determined by the region determination unit;
An image display unit that displays an image based on the image data corrected by the image quality correction unit,
The orthogonal transform coefficient feature determination unit divides the orthogonal transform coefficients in the pixel block into a plurality of groups, and calculates a first image feature amount based on the orthogonal transform coefficients belonging to the plurality of groups. An image display device. The image display device according to claim 1,
The orthogonal transform unit performs Hadamard transform. The image display device according to claim 1,
The image display device, wherein the image quality correction unit changes a correction amount of image quality correction for each region. An input step of inputting the image data;
A block extraction step of extracting a part of the image data as a pixel block of a predetermined size;
An orthogonal transform step of calculating an orthogonal transform coefficient for each pixel block in the block extraction step;
An orthogonal transform coefficient feature determining step for calculating a first image feature amount based on the orthogonal transform coefficient;
A luminance determination step of calculating a second image feature amount based on a luminance value of each pixel in the image data;
A region determination step for determining the type of region for each pixel based on the first image feature amount obtained in the orthogonal transform coefficient feature determination step and the second image feature amount obtained in the luminance determination step;
An image quality correction step of performing image quality correction processing on the image data of each area determined in the area determination step,
The orthogonal transform coefficient feature determining step divides the orthogonal transform coefficients in the pixel block into a plurality of groups, and calculates a first image feature amount based on the orthogonal transform coefficients belonging to the plurality of groups. An image processing method. The image processing method according to claim 4,
The orthogonal transforming step performs Hadamard transform. The image processing method according to claim 4,
The image quality correction step is characterized in that a correction amount of image quality correction is changed for each area.

Images