JP6089470B2 - Image processing apparatus, image display apparatus, and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus that outputs an image projected on a projection plane in an image display apparatus that projects an image onto a projection plane by an image projection unit.

  In a projection display device such as a liquid crystal projector device or a rear projection television, image distortion occurs due to the positional relationship between the light source and the screen or the distortion of the optical system due to the aberration of the optical system. In order to correct this distortion, there is known a method of projecting an image that has undergone conversion of characteristics opposite to that of the distortion of the optical system.

JP 2010-252290 A

  However, the conventional techniques as described above are based on the assumption that the projection surface is a continuous surface. For this reason, when the projection surface is a discontinuous surface, the conventional method cannot correct the distortion of the optical system.

The image processing apparatus according to the present invention has a projection plane divided into first to Nth areas where N is a natural number, and the projection plane is not at the boundary of the projection plane divided into each area. Each pixel constituting the output image is in any one of the first to Nth areas of the projection plane corresponding to the projection plane that is continuous and is continuous within the range of the projection plane divided into each of the areas. Output image division data storage means for holding output image division data which is a value indicating whether or not, correction value storage means for holding data for coordinate conversion for geometrically converting at least a part of the input image, and the output and a geometrical transformation means for performing the geometric transformation corresponding to the projection plane which is divided into the area based on the image segmentation data and the coordinate conversion data for each projection plane divided into the area, the correction value The storage means is the output image For the representative pixel selected from the constituent pixels, information regarding the position of the pixel to be referred to in the input image when performing the geometric conversion is held as the data for coordinate conversion, and the geometric conversion means includes: Reference pixel position calculation means for obtaining the position of the pixel to be referred to in the input image when performing the geometric conversion with respect to each pixel of the output image by interpolating the coordinate conversion data. When J is a natural number from 1 to N, the pixel position calculation means gives a representative pixel included in the Jth area for a pixel included in the Jth area among the pixels of the output image. The interpolation is performed using only the coordinate conversion data .

  The image processing apparatus of the present invention can correct distortions of different optical systems that occur for each discontinuous surface.

1 is a block diagram illustrating a configuration of an image processing device according to a first embodiment. 1 is a configuration diagram illustrating a configuration of an image display device using an image processing device according to Embodiment 1. FIG. It is a schematic diagram showing the distortion of the optical system of a horizontal direction. It is a schematic diagram showing the distortion contrary to the distortion of the optical system of a horizontal direction. It is a figure showing the geometric transformation for giving the distortion contrary to the distortion of the optical system of a horizontal direction. It is a figure showing the data hold | maintained at the correction value memory | storage means 2H. The figure which shows an example of the interpolation coefficient C6A (i) It is a figure showing the projection surface comprised by a discontinuous surface. It is a figure showing the projection surface comprised by a discontinuous surface. It is a block diagram showing the reference pixel position calculation means 3H. It is a figure showing operation | movement of the interpolation means 3HB1. It is a figure showing operation | movement of the interpolation means 3HB2. It is a figure showing operation | movement of the interpolation means 3HC. 6 is a block diagram illustrating a configuration of an image processing apparatus according to Embodiment 2. FIG. It is a schematic diagram showing distortion of the optical system in the vertical direction. It is a schematic diagram showing the reverse distortion of the optical system of a perpendicular direction. It is a figure showing the geometric transformation for giving the reverse distortion of the optical system of a perpendicular direction. It is a figure showing the reference pixel position calculation means 3V. It is a figure showing operation | movement of the interpolation means 3VB1. It is a figure showing operation | movement of the interpolation means 3VB2. It is a figure showing operation | movement of the interpolation means 3VC. 10 is a flowchart illustrating an image processing method according to Embodiment 3. FIG. FIG. 10 is a block diagram illustrating an image processing apparatus that executes an image processing method according to a third embodiment. It is a flowchart showing reference pixel position calculation step ST3H. FIG. 10 is a flowchart showing an image processing method according to the fourth embodiment. It is a flowchart showing reference pixel position calculation step ST3V. FIG. 10 is a configuration diagram illustrating a configuration of an image display device according to a fifth embodiment.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 1 of the present invention. FIG. 2 is a configuration diagram showing the configuration of the projection display apparatus according to the first embodiment of the present invention. The image processing apparatus includes a correction value storage unit 2H, an output image division data storage unit 9, and a geometric conversion unit 4. The geometric conversion means 4 has a reference pixel position calculation means 3H. Further, the geometric conversion unit 4 can include an image data storage unit 1, an image data reading unit 5H, an interpolation coefficient calculation unit 6HA, and an interpolation calculation unit 7H. In addition, the image processing apparatus can include a control unit 8. The control unit 8 controls the image data storage unit 1, the correction value storage unit 2H, the reference pixel position calculation unit 3H, the image data reading unit 5H, the interpolation coefficient calculation unit 6HA, the interpolation unit 7H, and the output image divided data storage unit 9. These are operated in cooperation with each other. A control signal line from the control means 8 to each means is not shown.

  The image processing apparatus shown in FIG. 1 can be used as a part of an image display apparatus typified by a projection display apparatus such as a rear projection television shown in FIG.

  The rear projection television shown in FIG. 2 includes the image processing device U0, the image projection unit U1, and the projection plane U2 shown in FIG. The image projection unit U1 has a light source U1A and an optical system U1B. The optical system U1B has a mirror U1B1 and a lens U1B2.

  In the rear projection television shown in FIG. 2, an image corresponding to the input image DIN is projected on the projection plane U2 by the following procedure. First, the image processing device U0 outputs an image DU0 obtained by performing processing described later on the input image DIN. Next, light corresponding to the image DU0 is output from the light source U1A. The light output from the light source U1A is projected on the projection surface U2 via the optical system U1B including the mirror U1B1 and the lens U1B2.

  The projection surface U2 of the rear projection television (or projection display device) according to the first embodiment is constituted by discontinuous surfaces as shown in FIGS. Here, FIG. 8 shows a perspective view of the projection plane U2. FIG. 9 shows a three-view diagram of the projection plane U2. In this example, the projection plane U2 includes a first projection plane U2A and a second projection plane U2B. The first projection plane U2A and the second projection plane U2B are projection planes divided into regions. The projection plane divided into these areas is a discontinuous projection plane.

  Here, “discontinuous” means that the distance from the optical system U1B to the projection plane U2 changes discontinuously. The projection plane U2 has a discontinuous boundary between the projection plane divided into one area and the projection plane divided into another area. The projection plane U2 is continuous within the range of the projection plane divided into one area.

  However, for the sake of simplicity in the following description, the description will be continued assuming that the projection plane U2 is a single plane. A problem caused by the fact that the projection plane U2 is not configured as a single plane and a solution to the problem will be described separately. Furthermore, for the sake of simplicity, it is assumed that the image processing apparatus U0 does not include the output image division data storage unit 9. Also, the optical system U1B will be described as having a distortion in the first direction. Here, the first direction will be described as a horizontal direction, for example.

  In this case, the distortion of the optical system U1B is schematically represented as shown in FIGS. 3 (A) and 3 (B). That is, when a grid-like image composed of a horizontal straight line and a vertical straight line as shown in FIG. 3A is projected onto the projection plane U2 by the optical system U1B, the projection plane U2 as shown in FIG. 3B. An image in which the lattice is distorted in a trapezoidal shape is displayed. The broken line PA shown in FIG. 3B represents the range of the image displayed on the projection surface U2 when the upper side of the trapezoidal image is adjusted to match the width of the projection surface. is there.

  Thus, in the rear projection television shown in FIG. 2, the distorted image is projected on the projection plane U2 by the optical system U1B in the image projection means U1. Therefore, in the image processing device U0, an image DU0 in which the input image DIN is distorted with characteristics opposite to the distortion caused by the optical system U1B as shown in FIG. 4 is generated in advance. A range of an image DU0 obtained by distorting the input image DIN with characteristics opposite to those of the distortion caused by the optical system U1B is indicated by a symbol PB in FIG. When the image DU0 is input to the image projection unit U1, the input image DIN is projected on the projection surface U2 with no apparent distortion.

  Distorting the input image DIN can be realized by enlarging or reducing at least a part of the input image DIN in the horizontal direction. Such enlargement or reduction is achieved by performing coordinate transformation on the input image DIN to obtain the output image DU0. This process can also be regarded as a geometric transformation for the input image DIN. This process is a process for obtaining an output image DU0 by performing coordinate transformation on the input image DIN.

  As described above, the image processing device U0 can generate an image DU0 in which the input image DIN is distorted (distortion corrected) with characteristics opposite to those of the distortion caused by the optical system U1B. For this reason, the input image DIN is projected on the projection surface U2 in an apparently undistorted state.

  The operation of the image processing apparatus shown in FIG. 1 will be described in more detail. The image processing apparatus shown in FIG. 1 performs interpolation and coordinate conversion on the input image D0. The input image D0 corresponds to the input image DIN shown in FIG. Then, the image processing device generates and outputs an output image D7. The output image D7 corresponds to the image DU0 shown in FIG. This process can also be regarded as a process in which the geometric conversion means 4 performs geometric conversion on the input image D0 and outputs the output image D7.

  The image data storage unit 1 stores the pixel value of each pixel of the input image D0. The input image D0 has pixels arranged in a matrix in different first directions (for example, horizontal direction) and second directions (for example, vertical direction). Similarly to the input image D0, the output image D7 generated by the image processing device is also a pixel arrayed in a matrix in a first direction (for example, the horizontal direction) and a second direction (for example, the vertical direction) that are different from each other. have. The pixel position of the input image and the pixel position of the output image are represented by two-dimensional coordinates. In such a two-dimensional coordinate system, the interval in the first direction of the pixels represented by the difference in coordinate values is “1”. Further, the interval between the pixels in the second direction represented by the difference in coordinate values is “1”.

  The correction value storage unit 2H holds data related to the position of a pixel to be referred to in order to correct distortion caused by the optical system U1B. The data relating to the position of the pixel to be referred to is data representing the coordinate position before coordinate conversion for enlarging or reducing the image for at least some of the pixels of the output image. For example, the data related to the position of the pixel to be referenced represents a relative position with respect to the position of the pixel of the output image. Specifically, the relative position is represented by a difference in position. By performing enlargement or reduction according to this difference, distortion correction is performed. For this reason, this difference is also called a “distortion correction value”.

  FIG. 5 is a diagram for explaining the position of a pixel to be referred to (pixel position before coordinate conversion). The horizontal coordinate is shown on the uppermost side. Below the horizontal coordinate, the pixel position of a row of the input image D0 is shown. A certain line is one line selected as an example of the input image D0. The pixel position of the input image D0 is a pixel position before coordinate conversion. Below the input image D0, pixel positions in the same row as the input image D0 of the output image D7 are shown. The pixel position of the output image D7 is a pixel position after coordinate conversion.

  From FIG. 5, the pixel of the output image D7 whose horizontal coordinate is on the left side of the value Ah may refer to the pixel of the input image D0 on the left side of the pixel of the output image D7 itself. Further, the pixel of the output image D7 whose horizontal coordinate is on the right side of the value Bh may refer to the pixel of the input image D0 on the right side of the pixel of the output image D7 itself. The pixel of the output image D7 whose horizontal coordinate is between the value Ah and the value Bh may refer to the pixel of the input image D0 at the same position as the output image D7 itself.

  The correction value storage unit 2H holds data representing the relative position of the reference pixel in the image D0 for a part of the pixels in the output image D7. Specifically, the number of pixels to the right or the left of each pixel with respect to the same coordinate in the image D0 is held.

  For example, when it is appropriate to refer to the right of three pixels, data “plus 3” is held. When it is appropriate to refer to the left of four pixels, data of “minus 4” is written. When it is appropriate to refer to a pixel having the same coordinate, data of “zero” is written. In short, the difference between the horizontal coordinate value of each pixel in the output image D7 and the horizontal coordinate value of the pixel to be referred to in the image D0 is written. The horizontal coordinate of the pixel to be referred to is the value of the pixel position before coordinate conversion. The difference is a value with a positive or negative sign.

  Here, if distortion correction values are held for all the pixels in the output image D7, the number of data to be held becomes enormous. Here, the distortion correction value is data indicating the distortion correction value. The number of data means the amount of data. Therefore, data is held at predetermined pixel intervals in both the horizontal direction and the vertical direction. In other words, pixels at predetermined pixel intervals are used as representative pixels. Then, the distortion correction value is held only for each representative pixel. As a result, the number of data to be held can be reduced.

  As will be described later, the pixel value of the pixel to be referred to for the pixels other than the representative pixel is obtained by interpolation calculation. The pixel to be referred to is a pixel before coordinate conversion. For this reason, the distortion correction value for the representative pixel is not limited to an integer (representing an integer multiple of the pixel interval), and may be a real number including an integer part and a decimal part. For example, if it is desired to refer to the right of 3.5 pixels, a value of “plus 3.5” may be set.

  FIG. 6 is a diagram illustrating, as an example, a case where every 64 pixels in the horizontal direction and the vertical direction are representative pixels in the output image D7. The distortion correction value P2 of each representative pixel is shown at the position of each representative pixel. The position of the representative pixel is represented by a horizontal coordinate of 64 × M and a vertical coordinate of 64 × N, where M and N are integers. The distortion correction value P2 for each representative pixel is represented by P2 (M, N).

  The reference pixel position calculation unit 3H performs linear interpolation for each pixel of the output image D7 based on the distortion correction value P2 (M, N) of the representative pixel held in the correction value storage unit 2H. The reference pixel position calculation unit 3H calculates the horizontal coordinate U and the vertical coordinate V of the position of the pixel to be referred to in the input image D0 (position before coordinate conversion) for each of all the pixels of the output image D7. The position of the pixel to be referred to is the position before coordinate conversion.

  That is, an output image D7 in which the horizontal coordinate is represented by X = 64 × J + A and the vertical coordinate is represented by Y = 64 × K + B (J and K are integers greater than or equal to zero and A and B are integers greater than or equal to zero and less than 63). The horizontal coordinate U and the vertical coordinate V of the reference pixel position (pixel position before coordinate conversion) in the input image D0 are obtained by calculation using the following equation (1).

  The coordinates (U, V) obtained by Expression (1) indicate the absolute position in the input image D0. The coordinates (U, V) are output as reference pixel position data P3.

  From Equation (1), U is a real number, so it is divided into an integer part and a decimal part. The integer part of U (hereinafter referred to as UINT) and the coordinates (UINT, V) represented by V are input to the image reading means 5H as data P3INT. The decimal part of U (hereinafter referred to as UDEC) is input to the interpolation coefficient calculation means 6HA as data P3DEC. UINT can be defined as the largest integer that does not exceed U. UDEC can be defined as a value obtained by subtracting UINT from U.

  The image data reading unit 5H receives the data P3INT, and outputs the coordinates of a plurality of pixels including each pixel and pixels located in the vicinity thereof as data P5. The coordinates of a plurality of pixels are the coordinates of the pixels used for the interpolation calculation described later for four points. The coordinates of these four points are coordinates indicating the positions of the four pixels in the input image D0.

  When the coordinates (UINT, V) are input as the data P3INT, the image data reading means 5H receives the coordinates (UINT-1, V), coordinates (UINT, V), coordinates (UINT + 1, V), coordinates as the data P5. Data indicating the coordinates of the four points represented by (UINT + 2, V) is output. In this way, the image data reading unit 5H inputs the data P3INT and reads the pixel at the position indicated by the data P3INT and the pixels in the vicinity thereof. Neighboring pixels are, for example, a plurality of pixels composed of left and right pixels. Further, reading out a pixel means extracting the pixel.

  The image data storage means 1 inputs and stores the input image D0. Further, the image data storage means 1 inputs data P5 from the image reading means 5H. The data P5 is the coordinates of four points: coordinates (UINT-1, V), coordinates (UINT, V), coordinates (UINT + 1, V), and coordinates (UINT + 2, V).

  The image data storage means 1 reads the pixel value of the pixel in the input image D0 located at the coordinates of the four points indicated by the data P5 and outputs it as the pixel value D1. Here, the pixel values of the pixels in the input image D0 located at the coordinates of the four points indicated by the data P5 are the pixel value D0 (UINT-1, V), the pixel value D0 (UINT, V), and the pixel value D0 (UNIT + 1). , V) and a pixel value D0 (UINT + 2, V). The pixel value D1 is represented by a pixel value D1 (−1), a pixel value D1 (0), a pixel value D1 (1), and a pixel value D1 (2). That is, when i = −1 to 2, the image data storage unit 1 reads out and outputs the pixel value D0 (UINT + i, V) of the pixel in the input image D0 as the pixel value D1 (i).

  The relationship between the pixel value D0 (UINT + i, V) in the input image D0 and the read pixel value D (i) is expressed by the following equation (2).

  In addition, when the pixel in the input image D0 does not exist in the position represented by said coordinate, the value (for example, zero) showing black is output as a pixel value.

  The interpolation coefficient calculation means 6HA outputs an interpolation coefficient C6A based on the data P3DEC. FIG. 7 is a diagram showing how to obtain the interpolation coefficient C6A. The horizontal axis indicates the variable x, and the vertical axis indicates the function f (x). The interpolation coefficient calculation means 6HA obtains an interpolation coefficient C6A based on a predetermined function f (x) and data P3DEC. When the value of the data P3DEC is represented by UDEC, four values C6A (−1), C6A (0), C6A (1), and C6A (2) calculated by the following equation (3) are output as the interpolation coefficient C6A. Is done.

なお、関数ｆ（ｘ）は、以下の式（４）の条件が任意のα（０＜α＜１）について満足される関数である。例えば、関数ｆ（ｘ）は、ＳＩＮＣ関数に適当な窓関数を掛けたもので定義される。窓関数とは、例えばハミング窓関数、カイザー窓関数である。

The function f (x) is a function that satisfies the condition of the following formula (4) for any α (0 <α <1). For example, the function f (x) is defined by a SINC function multiplied by an appropriate window function. The window function is, for example, a Hamming window function or a Kaiser window function.

  FIG. 7 shows examples of f (x), f (x-UDEC), C6A (-1), C6A (0), C6A (1), and C6A (2).

  The interpolation calculation means 7H performs filter processing using the interpolation coefficient C6A and interpolation calculation of the following expression (5) using the image data D1 for the pixel at the position represented by the coordinates (X, Y) in the output image D7. Do. In Equation (5), the symbol D0 (g, h) represents the pixel value of the pixel at the position represented by the coordinates (g, h) in the input image D0.

  Thereby, the pixel value D7 (X, Y) of the pixel at the position represented by the coordinates (X, Y) is obtained. The output image D7 is obtained by performing the above processing for all the pixels.

  Further, according to the image processing apparatus according to the first embodiment, the pixel value at an arbitrary position in the input image D0 can be referred to by interpolation calculation. For this reason, arbitrary distortion can be given regarding the horizontal direction. That is, it is possible to correct any distortion of the optical system in the horizontal direction.

  However, when the distortion of the optical system is corrected as described above, the distortion of the optical system may not be sufficiently corrected depending on the shape of the projection plane U2. As shown in FIGS. 8 and 9, problems that occur when the projection plane U2 is constituted by the first projection plane U2A and the second projection plane U2B that are discontinuous with each other will be described below.

  In this case, the positional relationship between the first projection surface U2A and the optical system U1B is different from the positional relationship between the second projection surface U2B and the optical system U1B. For this reason, the characteristic of distortion generated in the image displayed on the first projection plane U2A by the optical system U1B is different from the characteristic of distortion generated in the image displayed on the second projection plane U2B.

  Therefore, when calculating the distortion correction value for the pixel displayed on the first projection plane U2A according to the equation (1), all the representative pixels that refer to the distortion correction value are areas displayed on the first projection plane U2A. It is preferable that it exists in. Similarly, when calculating the distortion correction value for the pixel displayed on the second projection plane U2B, all the representative pixels that refer to the distortion correction value are present in the area displayed on the second projection plane U2B. It is preferable.

  However, when the reference pixel position is obtained by the expression (1), the representative pixel that refers to the distortion correction value is located on the first projection plane U2A near the boundary between the first projection plane U2A and the second projection plane U2B. There are cases where the display area and the area displayed on the second projection plane U2B are arranged over both. In this case, an error from an ideal value may become large at the reference pixel position obtained by Expression (1). Further, when the error with respect to the reference pixel position becomes large, there arises a problem that the accuracy of correction of image distortion caused by the optical system U1B is also lowered.

  In order to solve this problem, the image processing apparatus according to the first embodiment is provided with the output image divided data storage means 9. The reference pixel position calculation unit 3H uses information held in the output image division data storage unit 9 when calculating the reference pixel position. When the reference pixel position calculation unit 3H calculates the distortion correction value for the pixel displayed on the first projection plane U2A, the distortion correction value of the representative pixel existing in the area displayed on the first projection plane U2A. Refer only to. In addition, when the reference pixel position calculation unit 3H calculates a distortion correction value for the pixel displayed on the second projection plane U2B, the reference pixel position calculation unit 3H may distort the representative pixel existing in the area displayed on the second projection plane U2B. Refer to the correction value only.

  Hereinafter, the reference pixel position calculation unit 3H and the output image division data storage unit 9 will be further described.

  The output image division data storage means 9 stores output image division data D9 that describes whether each representative pixel in the output image D7 is included in the first area or the second area. As the output image division data D9, for example, a first value (for example, 0) may be stored for the representative pixel included in the first region. The second value (for example, 1) may be stored for the representative pixels included in the second area.

  FIG. 10 is a block diagram showing the internal configuration of the reference pixel position calculation unit 3H. The reference pixel position calculation means 3H includes pixel position determination means 3HA1, 3HA2 and interpolation means 3HB1, 3HB2, 3HC. The operation of the reference pixel position calculation means 3H described below is different from the operation described above in obtaining the horizontal coordinate U of the position of the pixel to be referred to in the input image D0. The method for obtaining the vertical coordinate V is the same as that described above. Hereinafter, how to obtain the horizontal coordinate U will be described.

  The pixel position determination unit 3HA1 refers to the output image division data D9, and the representative pixels represented by [J, K] and [J, K + 1] enter the same region as the pixel represented by the coordinates (X, Y). Judge whether or not. The pixel position determination unit 3HA1 outputs the determination result as a determination result D3HA1.

  The pixel position determination unit 3HA2 refers to the output image division data D9, and the representative pixels represented by [J + 1, K] and [J + 1, K + 1] enter the same region as the pixel represented by coordinates (X, Y). Judge whether or not. The pixel position determination unit 3HA2 outputs the determination result as a determination result D3HA2.

  However, the representative pixel represented by [J, K] represents a representative pixel having a horizontal coordinate of 64 × J and a vertical coordinate of 64 × K. The representative pixel represented by [J, K + 1] represents a representative pixel having a horizontal coordinate of 64 × J and a vertical coordinate of 64 × (K + 1). The representative pixel represented by [J + 1, K] represents a representative pixel having a horizontal coordinate of 64 × (J + 1) and a vertical coordinate of 64 × K. The representative pixel represented by [J + 1, K + 1] represents a representative pixel having a horizontal coordinate of 64 × (J + 1) and a vertical coordinate of 64 × (K + 1).

  FIG. 11 is a diagram illustrating the operation of the interpolation unit 3HB1. FIG. 12 is a diagram illustrating the operation of the interpolation unit 3HB2. FIG. 13 is a diagram illustrating the operation of the interpolation unit 3HC. As shown in FIG. 11, the interpolation means 3HB1 obtains data corresponding to PK0 described in the equation (1) from the distortion correction value P2 (J, K), the distortion correction value P2 (J, K + 1), and the determination result D3HA1. . Further, the interpolation unit 3HB1 outputs a flag F3HB1 indicating whether the obtained value of PK0 is a highly reliable value or a low value. In FIG. 11, when “O” is written in the column indicated by [J, K], the representative pixel represented by [J, K] is in the same area as the pixel represented by coordinates (X, Y). Indicates that it is in. In addition, when “X” is written in the column indicated by [J, K], the representative pixel represented by [J, K] is in the same area as the pixel represented by coordinates (X, Y). Indicates no. In addition, when “列” is written in the column indicated by [J, K + 1], the representative pixel represented by [J, K + 1] enters the same area as the pixel represented by coordinates (X, Y). Indicates that Further, when “X” is written in the column indicated by [J, K + 1], the representative pixel represented by [J, K + 1] is in the same area as the pixel represented by the coordinates (X, Y). Indicates no.

  When the representative pixel represented by [J, K] and the representative pixel represented by [J, K + 1] are in the same region as the pixel represented by coordinates (X, Y), the interpolation means 3HB1 Then, PK0 is obtained by the same method as that of the equation (1). That is, PK0 is represented by the following formula (6). In this case, PK0 is considered to be a highly reliable value, and flag F3HB1 indicates that the value of PK0 is valid.

  When only the representative pixel represented by [J, K] is in the same region as the pixel represented by coordinates (X, Y), the interpolation means 3HB1 outputs P2 (J, K) as PK0. This means that P2 (J, K) is used in place of P2 (J, K + 1) in equation (6). In this case, PK0 is considered to be a highly reliable value, and flag F3HB1 indicates that the value of PK0 is valid.

  When only the representative pixel represented by [J, K + 1] is in the same region as the pixel represented by coordinates (X, Y), the interpolation unit 3HB1 outputs P2 (J, K + 1) as PK0. This means that P2 (J, K + 1) is used in place of P2 (J, K) in equation (6). In this case, PK0 is considered to be a highly reliable value, and flag F3HB1 indicates that the value of PK0 is valid.

  When the representative pixel represented by [J, K] and the representative pixel represented by [J, K + 1] are not in the same region as the pixel represented by coordinates (X, Y), the interpolation means 3HB1 Then, PK0 is obtained by the same method as that in Expression (6). However, in this case, PK0 is considered to be a value with low reliability, and the flag F3HB1 indicates that the value of PK0 is invalid.

  As shown in FIG. 12, the interpolation unit 3HB2 obtains data corresponding to PK1 described in the expression (1) from the distortion correction value P2 (J + 1, K), the distortion correction value P2 (J + 1, K + 1), and the determination result D3HA2. Ask. In addition, a flag F3HB2 indicating whether the obtained value of PK1 is a highly reliable value or a low value is output.

  In FIG. 12, when “O” is written in the column labeled [J + 1, K], the representative pixel represented by [J + 1, K] is in the same area as the pixel represented by coordinates (X, Y). Indicates that it is in. If “X” is written in the column labeled [J + 1, K], the representative pixel represented by [J + 1, K] is not in the same area as the pixel represented by coordinates (X, Y). Indicates.

  In addition, when “Ｋ” is written in the column indicated as [J + 1, K + 1], the representative pixel represented by [J + 1, K + 1] enters the same region as the pixel represented by coordinates (X, Y). Indicates that If “X” is written in the column labeled [J + 1, K + 1], the representative pixel represented by [J + 1, K + 1] is not in the same area as the pixel represented by coordinates (X, Y). Indicates.

  When the representative pixel represented by [J + 1, K] and the representative pixel represented by [J + 1, K + 1] are in the same region as the pixel represented by coordinates (X, Y), the interpolation means 3HB2 Then, PK1 is obtained by the same method as that in Expression (1). That is, PK1 is represented by the following formula (7). In this case, PK1 is considered to be a highly reliable value, and flag F3HB2 indicates that the value of PK1 is valid.

  When only the representative pixel represented by [J + 1, K] is in the same region as the pixel represented by coordinates (X, Y), the interpolation unit 3HB2 outputs P2 (J + 1, K) as PK1. This means that P2 (J + 1, K) is used instead of P2 (J + 1, K + 1) in equation (7). In this case, PK1 is considered to be a highly reliable value, and flag F3HB2 indicates that the value of PK1 is valid.

  When only the representative pixel represented by [J + 1, K + 1] is in the same region as the pixel represented by coordinates (X, Y), the interpolation unit 3HB2 outputs P2 (J + 1, K + 1) as PK1. This means that P2 (J + 1, K + 1) is used in place of P2 (J + 1, K) in equation (7). In this case, PK1 is considered to be a highly reliable value, and flag F3HB2 indicates that the value of PK1 is valid.

  When the representative pixel represented by [J + 1, K] and the representative pixel represented by [J + 1, K + 1] are not in the same region as the pixel represented by coordinates (X, Y), the interpolation means 3HB1 PK1 is obtained by the same method as Expression (7). In this case, however, PK1 is considered to be a value with low reliability, and flag F3HB1 indicates that the value of PK1 is invalid.

  As shown in FIG. 13, the interpolation means 3HC changes the method of obtaining the horizontal coordinate U according to the values of the flags F3HB1, F3HB2, the first interpolation value PK0, and the second interpolation value PK1.

  When the flag F3HB1 indicates that PK0 is valid and the flag F3HB2 indicates that PK1 is valid, the interpolating unit 3HC obtains the horizontal coordinate U by the same method as Expression (1). That is, the horizontal coordinate U is expressed by the following equation (8).

When the flag F3HB1 indicates that PK0 is valid and the flag F3HB2 indicates that PK1 is invalid, the interpolation unit 3HC obtains the horizontal coordinate U by the following equation. This means that PK0 is used instead of PK1 in equation (8).
U = X + PK0

When the flag F3HB1 indicates that PK0 is invalid and the flag F3HB2 indicates that PK1 is valid, the interpolation unit 3HC calculates the horizontal coordinate U by the following equation. This means that PK1 is used instead of PK0 in equation (8).
U = X + PK1

  When the flag F3HB1 indicates that PK0 is invalid and the flag F3HB2 indicates that PK1 is invalid, the interpolating unit 3HC obtains the horizontal coordinate U by the same method as in the equation (1). In this case, the value of the horizontal coordinate U is a low reliability value.

  The above is the operation of the image processing apparatus according to the first embodiment.

  According to the above operation, when the horizontal coordinate U is calculated, the data of the first area and the data of the second area are not confused. For this reason, the horizontal coordinate U can be obtained with high accuracy. Therefore, optical distortion can be accurately corrected even if the projection surface is constituted by a discontinuous surface.

  If only the representative pixels are arranged in a two-dimensional manner and regarded as an image, the output image division data D9 can be regarded as an image in which a portion corresponding to the first region is filled with the first value. It can be regarded as an image in which pixel values corresponding to the region are filled with the second value.

  As described above, it is known that the data size can be reduced by compressing an image painted in a certain area. As a compression method, for example, run length encoding can be used. Therefore, the output image division data storage means 9 can reduce the size by compressing and saving the output image division data D9. In this case, an output image divided data restoring unit that cancels the compression processing applied to the output image divided data D9 may be provided inside the geometric conversion unit 4.

  In the above example, the input image D1 and the output image D7 have the same number of pixels. However, the number of pixels may be different. In particular, by making the number of pixels of the input image D1 larger than the number of pixels of the output image D7, the accuracy of the interpolation calculation in the interpolation calculation means 7H can be increased. Alternatively, by making the resolution of the input image D1 higher than that of the output image D7, the accuracy of the interpolation calculation in the interpolation calculation means 7H can be increased. In other words, it is possible to prevent image deterioration due to interpolation calculation or geometric conversion to some extent. In this case, the distortion correction value stored in the reference pixel position storage unit 2H may be a value obtained on a coordinate system in which the pixel interval of the input image D1 is 1.

  As described above, the image processing apparatus according to the first embodiment includes the output image division data storage unit 9, the correction value storage unit 2H, and the geometric conversion unit 4. For this reason, the image processing apparatus according to the first embodiment can also be used in a projection-type image display apparatus configured with a discontinuous projection surface. In a projection-type image display device in which the projection surface is a discontinuous surface, different optical system distortion occurs between the projection optical system and each of the discontinuous surfaces constituting the projection surface. However, since the image processing apparatus according to the first embodiment includes the output image division data storage unit 9, the correction value storage unit 2H, and the geometric conversion unit 4, different optical systems generated for each discontinuous surface. Can be corrected.

  In the above example, the projection surface is composed of two discontinuous surfaces. However, the configuration of the projection surface to which the present invention can be applied is not limited to this. That is, the projection surface may be composed of first to Nth discontinuous surfaces. In this case, the output image division data storage unit 9 may hold a value indicating whether the corresponding pixel belongs to the region corresponding to any of the first to Nth surfaces for each pixel constituting the output image. For example, the J-th value may be held for pixels belonging to the region corresponding to the J-th surface. Here, J is any natural number from 1 to N.

  Note that the processing in the image processing apparatus according to Embodiment 1 can also be regarded as geometric transformation for the input image DIN. Therefore, the data held in the correction value storage unit 2H can be regarded as coordinate conversion data for geometrically converting at least a part of the input image.

  In the correction value storage unit 2H, the interval between representative pixels that hold the distortion correction value is not limited to every 64 pixels. Here, the distortion correction value is data indicating the distortion correction value. However, if the interval between the representative pixels is set to a power of 2, the division used when calculating the horizontal coordinate U can be substituted by a shift operation. Therefore, when the image processing apparatus according to the first embodiment is mounted on a circuit. It is advantageous.

Embodiment 2. FIG.
FIG. 14 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 2 of the present invention. The configuration is the same as that of the image processing apparatus described in the first embodiment. However, instead of the correction value storage means 2H, the reference pixel position calculation means 3H, the image data reading means 5H, the interpolation coefficient calculation means 6HA and the interpolation calculation means 7H in FIG. 1, the correction value storage means 2V and the reference pixel position calculation means 3V are used. An image data reading means 5V, an interpolation coefficient calculating means 6VA, and an interpolation calculating means 7V are provided. The geometric conversion means 4 according to the second embodiment includes a reference pixel position calculation means 3V, an image data storage means 1, an image data reading means 5V, an interpolation coefficient calculation means 6VA, and an interpolation calculation means 7V.

  The correction value storage means 2V, the reference pixel position calculation means 3V, the image data reading means 5V, the interpolation coefficient calculation means 6VA, and the interpolation calculation means 7V are respectively the correction value storage means 2H, the reference pixel position calculation means 3H, and the image data reading means 5H. This is the same as the interpolation coefficient calculation means 6HA and the interpolation calculation means 7H. However, as will be described later, the enlargement direction and the reduction direction are not the horizontal direction but the vertical direction, and the contents of the processing are different.

  The same components as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted. 1 are the image data storage unit 1, the control unit 8, and the output image divided data storage unit 9.

  The image processing apparatus according to the second embodiment can also be used as a part of an image display apparatus typified by a projection display apparatus such as a rear projection television shown in FIG.

  The image processing apparatus according to the first embodiment gives an arbitrary distortion in the horizontal direction. However, the image processing apparatus according to the second embodiment can give an arbitrary distortion in the vertical direction.

  For example, even if the optical system U1B shown in FIG. 2 has a distortion in the vertical direction as shown in FIGS. 15A and 15B, the image processing apparatus according to the second embodiment has the projection plane U2. It is possible to display an image with no distortion. FIG. 15 is a diagram schematically showing distortion of the optical system U1B. FIGS. 15A and 15B are diagrams schematically illustrating an example of distortion in the vertical direction of the optical system U1B. A lattice-like image composed of a horizontal straight line and a vertical straight line as shown in FIG. 15A causes a trapezoidal distortion as shown in FIG. 15B in the vertical direction by the optical system U1B. The broken line PA shown in FIG. 15B represents the range of the image displayed on the projection plane U2 when the left side of the trapezoidal image is adjusted so as to match the width of the projection plane. is there.

  When the optical system U1B shown in FIG. 2 has a distortion as shown in FIGS. 15A and 15B, the image processing device U0 has an image DU0 as shown in FIG. 16 with respect to the input image DIN. Is output to the image projection unit U1. An image DU0 shown in FIG. 16 is an image distorted with characteristics opposite to the distortion caused by the optical system U1B in the vertical direction. In FIG. 16, a range distorted with a characteristic opposite to the distortion caused by the optical system U1B is indicated by a symbol PB. Thereby, the input image DIN is projected on the projection plane U2 in a state where there is no apparent distortion.

  The operation of the image processing apparatus according to the second embodiment shown in FIG. 14 will be described below. The image processing apparatus shown in FIG. 14 performs interpolation and coordinate conversion on the input image D0 and outputs an image D7. Here, the input image D0 corresponds to the input image DIN shown in FIG. The image D7 corresponds to the image DU0 shown in FIG. This process can also be regarded as a process in which the geometric conversion means 4 performs geometric conversion on the input image D0 and outputs the output image D7.

  The image data storage means 1 stores the pixel value of each pixel of the input image D0.

  The correction value storage means 2V holds data representing the position of a pixel to be referred to in order to correct distortion caused by the optical system U1B. The pixel position to be referred to in order to correct the distortion caused by the optical system U1B is a pixel position before coordinate conversion. The data representing the position of the pixel to be referred to in order to correct the distortion caused by the optical system U1B is distortion correction value data.

  FIG. 17 is a diagram illustrating the position of a pixel to be referred to. The pixel position to be referred to is a pixel position before coordinate conversion. The vertical coordinate is shown on the far left. The pixel position of a certain column of the input image D0 is shown to the right of the vertical coordinate. A certain column is one column selected as an example of the input image D0. The pixel position of the input image D0 is a pixel position before coordinate conversion. The pixel positions in the same column as the input image D0 of the output image D7 are shown to the right of the input image D0. The pixel position of the output image D7 is a pixel position after coordinate conversion.

  From FIG. 17, the pixel of the output image D7 whose vertical coordinate is above the value Av in the output image D7 may refer to the pixel of the input image D0 above the pixel of the output image D7 itself. Further, the pixel of the output image D7 whose vertical coordinate is lower than the value Bv may refer to the pixel of the input image D0 below the pixel of the output image D7 itself. The pixel of the output image D7 whose vertical coordinate is between the value Av and the value Bv may refer to the pixel of the input image D0 at the same position as the output image D7 itself.

  The correction value storage unit 2V holds data representing the relative position of the reference pixel in the image D0 for some of the pixels in the output image D7. Data representing the relative position of the reference pixel is distortion correction value data. Specifically, the number of pixels below each pixel with the same coordinates in the image D0 is held. In addition, the number of pixels that each pixel should refer to from the pixel having the same coordinate in the image D0 is held.

  For example, when it is appropriate to refer to the area below 3 pixels, data “plus 3” is held in the correction value storage means 2V. When it is appropriate to refer to the upper four pixels, data of “minus 4” is written in the correction value storage means 2V. When it is appropriate to refer to pixels having the same coordinates, data of “zero” is written in the correction value storage means 2V. In short, in the correction value storage means 2V, the difference between the value of the vertical coordinate of each pixel in the output image D7 and the value of the vertical coordinate of the pixel to be referred to in the image D0 is written. This difference is a value with a positive or negative sign.

  Here, if distortion correction values are held for all the pixels in the output image D7, the number of data to be held becomes enormous. For this reason, data is held at predetermined pixel intervals in both the horizontal direction and the vertical direction. In other words, pixels at predetermined pixel intervals are set as representative pixels, and distortion correction values are held only for each representative pixel. As a result, the number of data to be held can be reduced. Here, the number of data refers to the amount of data.

  As will be described later, the pixel values of the pixels to be referred to for the pixels other than the representative pixels are obtained by interpolation calculation. The pixel to be referred to is a pixel at a position before coordinate conversion. For this reason, the distortion correction value for the representative pixel is not limited to an integer. That is, the value is not limited to a value representing an integer multiple of the pixel interval. The distortion correction value may be a real number including an integer part and a decimal part. For example, if it is desired to refer to 3.5 pixels below, a value of “plus 3.5” may be set.

  The output image division data storage means 9 stores output image division data D9. The output image division data D9 is data indicating whether each representative pixel in the output image D7 is included in the first region or the second region. For example, the output image division data D9 may indicate the first value for the representative pixels included in the first area. The first value is, for example, 0 (zero). Further, the output image division data D9 may indicate the second value for the representative pixels included in the second area. The second value is, for example, 1.

  Hereinafter, a process of holding the distortion correction value P2 of each representative pixel in the correction value storage unit 2V will be described. The representative pixel is a pixel every 64 pixels in both the horizontal direction and the vertical direction. The position of the representative pixel is represented by a horizontal coordinate of 64 × M and a vertical coordinate of 64 × N, where M and N are integers. The distortion correction value P2 for each representative pixel is represented by P2 (M, N) as shown in FIG.

  FIG. 18 is a block diagram showing the internal configuration of the reference pixel position calculation means 3V. The reference pixel position calculation means 3V includes pixel position determination means 3VA1, 3VA2 and interpolation means 3VB1, 3VB2, 3VC.

  The reference pixel position calculation unit 3V performs linear interpolation on each pixel in the output image D7 based on the distortion correction value P2 (M, N) for the representative pixel held in the correction value storage unit 2V. . The reference pixel position calculation unit 3V calculates the horizontal coordinate U and the vertical coordinate V of the position of the pixel to be referred to in the input image D0 for each of all the pixels of the output image D7. The position of the pixel to be referred to is the position before coordinate conversion.

  Here, since the image distortion occurs only in the vertical direction, the horizontal coordinate U of the position of the pixel to be referred to in the input image D0 is always equal to the horizontal coordinate of each pixel in the output image D7. That is, for the pixel at the position where the horizontal coordinate is represented by X = 64 × J + A and the vertical coordinate is represented by Y = 64 × K + B in the output image D7, the horizontal coordinate U of the reference pixel position in the input image D0 is X. Here, J and K are integers of zero or more. A and B are integers of zero or more and 63 or less.

  The operation of the pixel position determination means 3VA1, 3VA2 and the interpolation means 3VB1, 3VB2, 3VC and how to obtain the vertical coordinate V will be described below.

  The pixel position determination unit 3VA1 refers to the output image division data D9, and the representative pixel represented by [J, K] and [J + 1, K] is in the same area as the pixel represented by coordinates (X, Y). Judge whether it is in or not. The pixel position determination unit 3VA1 outputs the determination result as a determination result D3VA1.

  The pixel position determination unit 3VA2 refers to the output image division data D9, and the representative pixel represented by [J, K + 1] and [J + 1, K + 1] is placed in the same area as the pixel represented by the coordinates (X, Y). Judge whether it is in or not. The pixel position determination unit 3VA2 outputs the determination result as a determination result D3VA2.

  However, the representative pixel represented by [J, K] represents a representative pixel having a horizontal coordinate of 64 × J and a vertical coordinate of 64 × K. The representative pixel represented by [J, K + 1] represents a representative pixel having a horizontal coordinate of 64 × J and a vertical coordinate of 64 × (K + 1). The representative pixel represented by [J + 1, K] represents a representative pixel having a horizontal coordinate of 64 × (J + 1) and a vertical coordinate of 64 × K. The representative pixel represented by [J + 1, K + 1] represents a representative pixel having a horizontal coordinate of 64 × (J + 1) and a vertical coordinate of 64 × (K + 1).

  FIG. 19 is a diagram illustrating the operation of the interpolation unit 3VB1. FIG. 20 is a diagram illustrating the operation of the interpolation unit 3VB2. FIG. 21 is a diagram illustrating the operation of the interpolation unit 3VC. As shown in FIG. 19, the interpolation means 3VB1 obtains PK0 from the distortion correction value P2 (J, K), the distortion correction value P2 (J + 1, K) and the determination result D3VA1 according to the method described below. Further, the interpolation means 3VB1 outputs a flag F3VB1 indicating whether the obtained value of PK0 is a highly reliable value or a low value.

  In FIG. 19, when “O” is written in the column labeled [J, K], the representative pixel represented by [J, K] is in the same area as the pixel represented by coordinates (X, Y). Indicates that it is in. If “X” is written in the column indicated by [J, K], the representative pixel represented by [J, K] is not in the same area as the pixel represented by coordinates (X, Y). Indicates.

  In addition, when a circle is written in the column indicated as [J + 1, K], the representative pixel represented by [J + 1, K] enters the same area as the pixel represented by coordinates (X, Y). Indicates that If “X” is written in the column labeled [J + 1, K], the representative pixel represented by [J + 1, K] is not in the same area as the pixel represented by coordinates (X, Y). Indicates.

  When the representative pixel represented by [J, K] and the representative pixel represented by [J + 1, K] are in the same area as the pixel represented by coordinates (X, Y), the interpolation means 3VB1 uses PK0. Is calculated by the following equation (9). In this case, PK0 is considered as a highly reliable value, and the flag F3VB1 indicates that the value of PK0 is valid.

  When only the representative pixel represented by [J, K] is in the same region as the pixel represented by coordinates (X, Y), the interpolation means 3HB1 outputs P2 (J, K) as PK0. . This means that P2 (J, K) is used in place of P2 (J + 1, K) in equation (9). In this case, PK0 is considered as a highly reliable value, and the flag F3VB1 indicates that the value of PK0 is valid.

  When only the representative pixel represented by [J + 1, K] is in the same region as the pixel represented by coordinates (X, Y), the interpolation means 3VB1 outputs P2 (J + 1, K) as PK0. . This means that P2 (J + 1, K) is used in place of P2 (J, K) in equation (9). In this case, PK0 is considered as a highly reliable value, and the flag F3VB1 indicates that the value of PK0 is valid.

  When the representative pixel represented by [J, K] and the representative pixel represented by [J + 1, K] are not in the same region as the pixel represented by coordinates (X, Y), the interpolation means 3VB1 PK0 is obtained by the same method as equation (9). However, in this case, PK0 is considered to be a value with low reliability, and flag F3VB1 indicates that the value of PK0 is invalid.

  As shown in FIG. 20, the interpolation unit 3VB2 obtains PK1 from the distortion correction value P2 (J, K + 1), the distortion correction value P2 (J + 1, K + 1), and the determination result D3VA2 by the method described below. Further, a flag F3VB2 indicating whether the obtained value of PK1 is a highly reliable value or a low value is output.

  In FIG. 20, when “O” is written in the column indicated as [J, K + 1], the representative pixel represented by [J, K + 1] is in the same area as the pixel represented by coordinates (X, Y). Indicates that it is in. If “X” is written in the column labeled [J, K + 1], the representative pixel represented by [J, K + 1] is not in the same area as the pixel represented by coordinates (X, Y). Indicates.

  In addition, when “Ｋ” is written in the column indicated as [J + 1, K + 1], the representative pixel represented by [J + 1, K + 1] enters the same region as the pixel represented by coordinates (X, Y). Indicates that If “X” is written in the column labeled [J + 1, K + 1], the representative pixel represented by [J + 1, K + 1] is not in the same area as the pixel represented by coordinates (X, Y). Indicates.

  When the representative pixel represented by [J, K + 1] and the representative pixel represented by [J + 1, K + 1] are in the same region as the pixel represented by coordinates (X, Y), the interpolation means 3VB2 uses PK1. Is calculated by the following equation (10). In this case, PK1 is considered to be a highly reliable value, and flag F3VB2 indicates that the value of PK1 is valid.

  When only the representative pixel represented by [J, K + 1] is in the same region as the pixel represented by coordinates (X, Y), the interpolation means 3VB1 outputs P2 (J, K + 1) as PK1. . This means that P2 (J, K + 1) is used in place of P2 (J + 1, K + 1) in equation (10). In this case, PK1 is considered to be a highly reliable value, and flag F3VB2 indicates that the value of PK1 is valid.

  When only the representative pixel represented by [J + 1, K + 1] is in the same region as the pixel represented by coordinates (X, Y), the interpolation means 3VB2 outputs P2 (J + 1, K + 1) as PK1. . This means that P2 (J + 1, K + 1) is used in place of P2 (J, K + 1) in equation (10). In this case, PK1 is considered to be a highly reliable value, and flag F3VB2 indicates that the value of PK1 is valid.

  When the representative pixel represented by [J, K + 1] and the representative pixel represented by [J + 1, K + 1] are not in the same region as the pixel represented by coordinates (X, Y), the interpolation means 3VB1 PK1 is obtained by the same method as Expression (9). In this case, however, PK1 is considered to be a value with low reliability, and flag F3VB1 indicates that the value of PK1 is invalid.

  As shown in FIG. 21, the interpolation means 3VC calculates V from the values of the flags F3VB1, F3VB2, the first interpolation value PK0, and the second interpolation value PK1 by the method described below.

  When the flag F3VB1 indicates that PK0 is valid and the flag F3VB2 indicates that PK1 is valid, the interpolation unit 3VC calculates the vertical coordinate V by the following equation (11).

When the flag F3VB1 indicates that PK0 is valid and the flag F3VB2 indicates that PK1 is invalid, the interpolation means 3VC calculates the vertical coordinate V using the following equation. This means that PK0 is used in place of PK1 in equation (11).
V = Y + PK0

When the flag F3VB1 indicates that PK0 is invalid and the flag F3VB2 indicates that PK1 is valid, the interpolation unit 3VC obtains the vertical coordinate V using the following equation. This means that PK1 is used in place of PK0 in equation (11).
V = Y + PK1

  When the flag F3VB1 indicates that PK0 is invalid and the flag F3VB2 indicates that PK1 is invalid, the interpolating means 3VC calculates V by the same method as in the equation (11). In this case, the value of V is a low reliability value.

  The above is the operation of the image processing apparatus according to the second embodiment.

  The image data reading unit 5V receives the data P3INT, and outputs the coordinates of a plurality of pixels including each pixel and pixels located in the vicinity thereof as data P5. The coordinates of a plurality of pixels are the coordinates of the pixels used for the interpolation calculation described later for four points. The coordinates of these four points are coordinates indicating the positions of the four pixels in the input image D0.

  When the coordinates (U, VINT) are input as the data P3INT, the image data reading means 5V receives the coordinates (U, VINT-1), coordinates (U, VINT), coordinates (U, VINT + 1) and coordinates as the data P5. Data indicating the coordinates of the four points represented by (U, VINT + 2) is output. In this way, the image data reading unit 5V inputs the data P3INT and reads the pixel at the position indicated by the data P3INT and the pixels in the vicinity thereof. Neighboring pixels are, for example, a plurality of pixels composed of left and right pixels. Further, reading out a pixel means extracting the pixel.

  The image data storage means 1 inputs and stores the input image D0. Further, the image data storage means 1 inputs data P5 from the image reading means 5V. The data P5 is the coordinates of four points, coordinates (U, VINT-1), coordinates (U, VINT), coordinates (U, VINT + 1), and coordinates (U, VINT + 2).

  The image data storage means 1 reads the pixel value of the pixel in the input image D0 located at the coordinates of the four points indicated by the data P5 and outputs it as the pixel value D1. Here, the pixel values of the pixels in the input image D0 located at the coordinates of the four points indicated by the data P5 are the pixel value D0 (U, VINT-1), the pixel value D0 (U, VINT), and the pixel value D0 (U , VINT + 1) and pixel value D0 (U, VINT + 2). The pixel value D1 is represented by a pixel value D1 (−1), a pixel value D1 (0), a pixel value D1 (1), and a pixel value D1 (2). That is, when i = −1 to 2, the image data storage unit 1 reads out and outputs the pixel value D0 (U, VINT + i) of the pixel in the input image D0 as the pixel value D1 (i).

  The relationship between the pixel value D0 (U, VINT + i) in the input image D0 and the read pixel value D (i) is expressed by the following equation (12).

なお、上記の座標で表される位置に、入力画像Ｄ０内の画素が存在しない場合、画素値として、黒色を表す値を出力する。黒色を表す値とは、例えばゼロである。

Note that when a pixel in the input image D0 does not exist at the position represented by the above coordinates, a value representing black is output as the pixel value. The value representing black is, for example, zero.

  The operation of the interpolation coefficient calculation means 6VA is the same as that of the interpolation coefficient calculation means 6HA in the first embodiment. That is, the interpolation coefficient C6A output from the interpolation coefficient calculation means 6VA is four values C6A (−1), C6A (0), C6A (1), and C6A (2) obtained by replacing UDEC in Expression (3) with VDEC. expressed.

  The interpolation calculation means 7V is expressed by the following expression (13) using the filter processing using the interpolation coefficient C6A and the image data D1 for the pixel at the position represented by the coordinates (X, Y) in the output image D7. Perform interpolation calculation.

  Thereby, the pixel value D7 (X, Y) of the pixel at the position represented by the coordinates (X, Y) is obtained. The output image D7 is obtained by performing the above processing for all the pixels.

  The image processing apparatus according to the second embodiment can perform the same processing as the image processing apparatus according to the first embodiment in the vertical direction. For this reason, the image processing apparatus according to the second embodiment can obtain the same effect as the image processing apparatus according to the first embodiment with respect to the horizontal direction of the image.

  In the image processing apparatus according to the second embodiment, the changes described in the first embodiment can be added. The changes shown here are the following four changes. The first change is that the accuracy of the interpolation calculation in the interpolation calculation means 7H can be increased by making the number of pixels of the input image D1 of the interpolation calculation means 7H larger than the number of pixels of the output image D7. Alternatively, the accuracy of the interpolation calculation in the interpolation calculation means 7H can be increased by making the resolution of the input image D1 higher than that of the output image D7. The second modification is that, in the embodiment, the projection surface is composed of two discontinuous surfaces, but the projection surface may be composed of first to Nth discontinuous surfaces. A third change is that in the correction value storage unit 2H, the interval between representative pixels that hold the distortion correction value is not limited to every 64 pixels.

  Moreover, the following change can be considered as a 4th change. For example, in the first embodiment, the output image division data storage unit 9 compresses and stores the data using the output image division data. In the image processing method according to the third embodiment, data may be compressed and stored as in the output image division data shown in the first embodiment. In this case, a restoration step for restoring the original data from the compressed and saved data may be incorporated in the flow of the image processing method shown in FIG. Further, the output image D7 obtained by the above processing can be used as the image DU0 in the image display device shown in FIG.

Embodiment 3 FIG.
FIG. 22 is a flowchart of an image processing method according to the third embodiment of the present invention. The image processing method according to Embodiment 3 can be used as a part of an image display device typified by a projection display device. For example, the image processing by the image processing method according to the third embodiment may be executed in the image processing apparatus U0 of the rear projection television shown in FIG.

  FIG. 23 shows an example of an image processing apparatus that can execute the image processing method shown in FIG. The image processing apparatus shown in FIG. 23 can be used as the image processing apparatus U0. The image processing apparatus shown in FIG. 23 includes an input interface 101, a program storage memory 102, a CPU 103, a data storage memory 104, an output interface 105, an image data storage means 1, a correction value storage means 2H, output image divided data 9, and the connection thereof. A bus 106 is provided.

  An image to be processed is input to the image processing apparatus shown in FIG. The image to be processed is the input image D0. The function of the image data storage unit 1, the function of the correction value storage unit 2H, and the function of the output image division data storage unit 9 are the same as those shown in FIG. The CPU 103 executes the image processing method shown in FIG. 22 according to the program stored in the program storage memory 102. The data storage memory 104 stores various data in the course of operation. The output interface 105 outputs an output image D7. The output image D7 is an image generated as a result of the processing. Also, data necessary for the operation process is transmitted and received on the bus 106. Details of the image processing method shown in FIG. 22 will be described below.

  The image processing method shown in FIG. 22 includes a correction value reading step ST2H, an output image division data reading step ST9, and a geometric conversion step ST4. The geometric conversion step ST4 includes a reference pixel position calculation step ST3H, an image data read step ST5H, an interpolation coefficient calculation step ST6HA, and an interpolation calculation step ST7H.

  In the correction value reading step ST2H, the distortion correction value P2 of the representative pixel in the output image D7 held in the correction value storage unit 2H is read. In the output image division data reading step ST9, the output image division data D9 is read from the output image division data storage means 9.

  As shown in FIG. 24, the reference pixel position calculation step ST3H has a pixel position determination step ST3HA1, an interpolation step ST3HB1, a pixel position determination step ST3HA2, an interpolation step ST3HB2, and an interpolation step ST3HC.

  Here, the operation of the pixel position determination step ST3HA1 is the same as the operation of the pixel position determination means 3HA1. The operation of the interpolation step ST3HB1 is the same as that of the interpolation means 3HB1. The pixel position determination step ST3HA2 is the same as the operation of the pixel position determination means ST3HA2. The operation of the interpolation step ST3HB2 is the same as the operation of the interpolation means 3HB2. The operation of the interpolation step ST3HC is the same as that of the interpolation means 3HC. Thereby, the same value as the horizontal coordinate U described in the first embodiment and the vertical coordinate V described in the second embodiment is calculated for each pixel of the output image D7. Since U is a real number, its integer part is expressed as UINT and the decimal part is expressed as UDEC.

  In the image data reading step ST5H, coordinates (UINT-1, V), coordinates (UINT, V), coordinates (UINT + 1, V) among the pixels constituting the input image D0 based on the values of UINT and V. And the pixel value D0 (UINT-1, V), the pixel value D0 (UINT, V), the pixel value D0 (UINT + 1, V), and the pixel value D0 (UNIT + 2) of the pixel at the position represented by the coordinates (UINT + 2, V). , V) is read from the image data storage means 1 and output.

  In this manner, the image data reading step ST5H reads the pixel at the position indicated by the coordinates (V, UINT) and the pixels in the vicinity thereof in accordance with the value of UINT and the value of V. Neighboring pixels are, for example, a plurality of pixels composed of left and right pixels of a pixel at a position indicated by coordinates (V, UINT). Note that when a pixel in the input image D0 does not exist at the position represented by the above coordinates, a value representing black is output as the pixel value. A value representing black is, for example, zero.

  In addition, the interpolation coefficient calculation step ST6HA obtains the interpolation coefficient C6A based on the data UDEC by the same method as the interpolation coefficient calculation means 6HA in the first embodiment.

  In the interpolation calculation step ST7H, the same calculation as that of the interpolation calculation means 7H is performed on the pixel at the position represented by the coordinates (X, Y) in the output image D7. The calculation performed by the interpolation calculation step ST7H is performed using filter processing using the interpolation coefficient C6A and the pixel value of the input image D0.

  Thereby, the pixel value D7 (X, Y) for the pixel at the position represented by the coordinates (X, Y) is obtained. The output image D7 is obtained by performing the above processing for all the pixels.

  The image processing method according to the third embodiment can perform the same processing as the image processing apparatus according to the first embodiment. For this reason, the image processing method according to the third embodiment can obtain the same effects as those of the image processing apparatus according to the first embodiment. It is also possible to add the same changes as the image processing apparatus according to the first embodiment. The changes shown here are the following four changes. The first change is that the accuracy of the interpolation calculation in the interpolation calculation means 7H can be increased by making the number of pixels of the input image D1 of the interpolation calculation means 7H larger than the number of pixels of the output image D7. Alternatively, the accuracy of the interpolation calculation in the interpolation calculation means 7H can be increased by making the resolution of the input image D1 higher than that of the output image D7. The second modification is that, in the embodiment, the projection surface is composed of two discontinuous surfaces, but the projection surface may be composed of first to Nth discontinuous surfaces. A third change is that in the correction value storage unit 2H, the interval between representative pixels that hold the distortion correction value is not limited to every 64 pixels.

  Moreover, the following change can be considered as a 4th change. For example, in the first embodiment, the output image division data storage unit 9 compresses and stores the data using the output image division data. In the image processing method according to the third embodiment, data may be compressed and stored as in the output image division data shown in the first embodiment. In this case, a restoration step for restoring the original data from the compressed and saved data may be incorporated in the flow of the image processing method shown in FIG. Further, the output image D7 obtained by the above processing can be used as the image DU0 in the image display device shown in FIG.

Embodiment 4 FIG.
FIG. 25 is a flowchart of an image processing method according to the fourth embodiment of the present invention. The image processing method according to the fourth embodiment can be used as a part of an image display device typified by a projection display device. For example, the image processing by the image processing method according to the fourth embodiment may be executed in the image processing apparatus U0 of the rear projection television shown in FIG.

  Note that the image processing apparatus for carrying out the image processing method according to the fourth embodiment may be provided with the correction value storage means 2V instead of the correction value storage means 2H in the image processing apparatus shown in FIG. The same components as those in FIG. 23 are denoted by the same reference numerals, and the description thereof is omitted. 23 are the input interface 101, the program storage memory 102, the CPU 103, the data storage memory 104, the output interface 105, the image data storage means 1, the output image divided data 9, and a bus connecting them. 106.

  Hereinafter, the image processing method shown in FIG. 25 will be further described.

  The image processing method shown in FIG. 25 includes a correction value reading step ST2V, an output image divided data reading step ST9, and a geometric conversion step ST4. The geometric conversion step ST4 includes a reference pixel position calculation step ST3V, an image data reading step ST5V, an interpolation coefficient calculation step ST6VA, and an interpolation calculation step ST7V.

  In the correction value reading step ST2V, the distortion correction value P2 held in the correction value storage unit 2V is read for the representative pixel in the output image D7. In the output image division data reading step ST9, the output image division data is read from the output image division data storage means 9.

  As shown in FIG. 26, the reference pixel position calculation step ST3V includes a pixel position determination step ST3VA1, an interpolation step ST3VB1, a pixel position determination step ST3VA2, an interpolation step ST3VB2, and an interpolation step ST3VC. Here, the operation of the pixel position determination step ST3VA1 is the same as the operation of the pixel position determination means 3VA1. The operation of the interpolation step ST3VB1 is the same as that of the interpolation means 3VB1. The pixel position determination step ST3VA2 is the same as the operation of the pixel position determination means ST3VA2. The operation of the interpolation step ST3VB2 is the same as that of the interpolation means 3VB2. The operation of the interpolation step ST3VC is the same as that of the interpolation means 3VC. Thereby, the same value as the horizontal coordinate U and the vertical coordinate V described in the second embodiment is calculated for each pixel of the output image D7. Since V is a real number, its integer part is expressed as VINT, and its decimal part is expressed as VDEC.

  In the image data reading step ST5V, coordinates (U, VINT-1), coordinates (U, VINT), coordinates (U, VINT + 1) among the pixels constituting the input image D0 based on the values of U and VINT. And a pixel value D0 (U, VINT-1), a pixel value D0 (U, VINT + 1), a pixel value D0 (U, VINT + 1), and a pixel value D0 (U) at a position represented by coordinates (U, VINT + 2). , VINT + 2) is read from the image data storage means 1 and output.

  In this way, the image data reading step ST5V reads the pixel at the position indicated by the coordinates (U, VINT) and the pixels in the vicinity thereof according to the value of U and the value of VINT. Neighboring pixels are, for example, a plurality of pixels composed of upper and lower pixels at a position indicated by coordinates (U, VINT). Note that when a pixel in the input image D0 does not exist at the position represented by the above coordinates, a value representing black is output as the pixel value. The value representing black is, for example, zero.

  Further, in the interpolation coefficient calculation step ST6VA, the interpolation coefficient C6A is obtained by the same method as the interpolation coefficient calculation means 6VA of the second embodiment based on the data VDEC.

  The interpolation calculation step ST7V performs the same calculation as the interpolation calculation means 7V on the pixel at the position represented by the coordinates (X, Y) in the output image D7. The calculation performed by the interpolation calculation step ST7V is performed using filter processing using the interpolation coefficient C6A and the pixel value of the input image D0.

  Thereby, the pixel value D7 (X, Y) for the pixel at the position represented by the coordinates (X, Y) is obtained. The output image D7 is obtained by performing the above processing for all the pixels.

  The image processing method according to the fourth embodiment can perform the same processing as that of the image processing apparatus according to the second embodiment, so that the same effect as that of the image processing apparatus according to the second embodiment can be obtained. It is also possible to add the same changes as the image processing apparatus according to the first embodiment. The changes shown here are the following four changes. The first change is that the accuracy of the interpolation calculation in the interpolation calculation means 7H can be increased by making the number of pixels of the input image D1 of the interpolation calculation means 7H larger than the number of pixels of the output image D7. Alternatively, the accuracy of the interpolation calculation in the interpolation calculation means 7H can be increased by making the resolution of the input image D1 higher than that of the output image D7. The second modification is that, in the embodiment, the projection surface is composed of two discontinuous surfaces, but the projection surface may be composed of first to Nth discontinuous surfaces. A third change is that in the correction value storage unit 2H, the interval between representative pixels that hold the distortion correction value is not limited to every 64 pixels.

  Moreover, the following change can be considered as a 4th change. For example, in the first embodiment, the output image division data storage unit 9 compresses and stores the data using the output image division data. In the image processing method according to the third embodiment, data may be compressed and stored as in the output image division data shown in the first embodiment. In this case, a restoration step for restoring the original data from the compressed and saved data may be incorporated in the flow of the image processing method shown in FIG. In addition, the output image D7 obtained by the above processing can be used as the image DU0 in the image display apparatus shown in FIG.

Embodiment 5 FIG.

  FIG. 27 is a block diagram showing an image display device according to the fifth embodiment of the present invention. The image display device shown in FIG. 27 includes an image processing device U0H, an image processing device U0V, an image projection unit U1, and a projection plane U2. The image projection unit U1 has a light source U1A and an optical system U1B. The optical system U1B includes a mirror U1B1 and a lens U1B2.

  The rear projection television shown in FIG. 27 projects an image corresponding to the input image DIN on the projection plane U2 according to the following procedure. First, the image processing device U0H outputs the image DU0H generated by the procedure described in the first or third embodiment with respect to the input image DIN. The procedure shown here is the correction value reading step ST2H, the output image divided data reading step ST9, the reference pixel position calculating step ST3H, the image data reading step ST5H, the interpolation coefficient calculating step ST6HA, and the interpolation calculating step ST7H shown in FIG. It is the procedure shown in.

  Next, the image processing device U0V outputs the image DU0V generated by the procedure described in the second embodiment or the fourth embodiment with respect to the image DU0H. The procedure shown here is the correction value reading step ST2V, the output image divided data reading step ST9, the reference pixel position calculating step ST3V, the image data reading step ST5V, the interpolation coefficient calculating step ST6VA and the interpolation calculating step ST7V shown in FIG. It is the procedure shown in.

  The light source U1A outputs light corresponding to the image DU0V. The light output from the light source U1A is projected on the projection plane U2 via the optical system U1B. The optical system U1B includes a mirror U1B1 and a lens U1B2.

  The image display device according to the fifth embodiment corrects the distortion of the optical system U1B in the horizontal direction in the image processing device U0H. The image display device according to the fifth embodiment corrects distortion of the optical system U1B in the vertical direction in the image processing device U0V. Accordingly, even if the optical system U1B has optical distortion in both the horizontal direction and the vertical direction, the image display apparatus according to Embodiment 5 can display an image without distortion on the projection plane U2.

  The order of arrangement of the image processing device U0H and the image processing device U0V may be changed. That is, the image processing device U0V may perform geometric conversion on the input image DIN in the same procedure as that described in the second embodiment or the fourth embodiment. The procedure shown here is the correction value reading step ST2V, the output image divided data reading step ST9, the reference pixel position calculating step ST3V, the image data reading step ST5V, the interpolation coefficient calculating step ST6VA and the interpolation calculating step ST7V shown in FIG. It is the procedure shown in.

  Then, the image processing device U0H may perform geometric conversion on the image output from the image processing device U0V by the same procedure as that described in the first embodiment or the third embodiment. The procedure shown here is the correction value reading step ST2H, the output image divided data reading step ST9, the reference pixel position calculating step ST3H, the image data reading step ST5H, the interpolation coefficient calculating step ST6HA, and the interpolation calculating step ST7H shown in FIG. It is the procedure shown in.

  When the terms “first direction” and “second direction” are used, the fifth embodiment can be generalized and described as follows. That is, the geometric transformation may be performed in the first direction using the image processing apparatus of the first embodiment or the image processing method of the third embodiment. Then, the geometric conversion may be performed in the second direction using the image processing apparatus of the third embodiment or the image processing method of the fourth embodiment. Further, the order of performing the geometric transformation may be reversed between the first direction and the second direction. In this way, it is possible to correct the distortion that the optical system U1B has with respect to the first direction and the second direction.

  In addition, although embodiment of this invention was described as mentioned above, this invention is not limited to these embodiment.

  1 image data storage means, 2H, 2V correction value storage means, 3H, 3V reference pixel position calculation means, 4 geometric conversion means, 5H, 5V image data read means, 6HA, 6VA interpolation coefficient calculation means, 7H, 7V interpolation calculation Calculation means, 8 control means, 9 output image division data storage means, D0 input image, D1 pixel value, D7 output image, C6A interpolation coefficient, P2 distortion correction value, P3 reference pixel position data, P5, P3INT, P3DEC data, D9 Output image segmentation data.

Claims (5)

When N is a natural number, the projection plane is divided into first to Nth areas, and the projection plane is discontinuous at the boundary of the projection plane divided into each area. An output image that is a value indicating which of the first to Nth areas of the projection plane each pixel constituting the output image corresponds to a projection plane that is continuous within the range of the divided projection planes. Output image divided data storage means for holding the divided data;
Correction value storage means for holding coordinate conversion data for geometrically converting at least a part of the input image;
Geometric conversion means for performing the geometric conversion corresponding to the projection plane divided into the regions based on the output image division data and the coordinate conversion data for each projection plane divided into the regions;
The correction value storage means stores information on the position of the pixel to be referred to in the input image when the geometric conversion is performed for the representative pixel selected from the pixels constituting the output image. Hold as
The geometric conversion means calculates a reference pixel position for each pixel of the output image by interpolating the coordinate conversion data for a pixel position to be referred to in the input image when performing the geometric conversion. Having means,
The reference pixel position calculation means, when J is a natural number from 1 to N, for pixels included in the J-th area among the pixels of the output image, representative pixels included in the J-th area An image processing apparatus that performs the interpolation using only the coordinate conversion data given for. The correction value storage means holds the compressed output image divided data as an output image divided data compressed value,
2. The image processing apparatus according to claim 1, wherein the geometric conversion means further comprises output image divided data decoding means for decoding the output image divided data from the output image divided data compression value. A projection plane composed of projection planes divided into first to Nth areas, where N is a natural number;
The image processing apparatus according to claim 1 or 2,
An image display device comprising: an optical system that projects the output image onto the projection surface. The image display apparatus according to claim 3, wherein the input image is an image having a higher resolution than the output image. When N is a natural number, the output image divided data is read out from the output image divided data storage means for holding the output image divided data for dividing the output image into the first to Nth areas. Steps,
A correction value for holding information on a reference pixel position in the input image as coordinate conversion data for geometrically converting at least a part of the input image for a representative pixel selected from pixels constituting the output image A correction value reading step for reading the data for coordinate conversion from the storage means;
For each pixel of the output image, when J is a natural number from 1 to N when interpolating the coordinate conversion data, pixels included in the Jth area among the pixels of the output image For the above, the reference pixel position is calculated by using only the coordinate conversion data given for the representative pixel included in the Jth region, and the reference image position is calculated based on the output image division data and the coordinate conversion data. An image processing method comprising: a geometric conversion step for performing geometric conversion.

Images