JP6814484B2 - Image processing equipment, image processing method and image processing program - Google Patents

Description

The present invention relates to an image processing technique.

Interframe prediction coding is one of the important techniques for compressing moving images. In inter-frame prediction coding, the image of the target frame is predicted based on the coded frame (hereinafter referred to as "reference frame") at a time different from the frame to be encoded (hereinafter referred to as "target frame"), and the target is targeted. Encode the difference between the actual image of the frame and the predicted image. The amount of data in the frame can be reduced by encoding the difference between the target frame and the coded frame.

In the inter-frame prediction coding, the following processes (1) to (3) are performed in block units in which a frame is divided into a plurality of frames.
(1) A block (hereinafter referred to as "reference block") similar to a block in the target frame (hereinafter referred to as "target block") is searched from within the reference frame using an image matching technique.
(2) Obtain the difference image between the target block and the reference block obtained by the search.
(3) Obtain a motion vector indicating the movement from the reference block obtained by the search to the target block.

When decoding the frame of the moving image compressed in this way, the image of the target block can be restored from the reference block, the difference image, and the motion vector.

As described above, in the above (1), the image matching technique is used when searching for a reference block similar to the target block. Various methods such as feature-based matching and area-based matching are known as image matching techniques. The movement of an object in a moving image includes rotation in addition to movement and expansion and contraction (scale change). In (1) above, it is required to calculate the relative angle between the target block and the reference block in order to determine that the blocks in which the rotating object exists in the moving image are similar to each other. Further, as described above, the movement from the reference block to the target block calculated in (3) above includes movement, expansion / contraction, and rotation. Therefore, it is necessary to calculate the relative angle between the target block and the reference block.

Patent Document 1 discloses a technique for calculating motion in moving image compression. The coordinates of the three points in the region of the previous frame are obtained using the three points in the known region of the current frame on the image plane and the three translation vectors searched for these three points. Furthermore, the horizontal and vertical components of each vector are calculated from the position coordinates of the six points in the regions of the current frame and the previous frame, and the horizontal and vertical rotation angles are calculated from the x and y components of those vectors. Six affine conversion parameters of expansion / contraction magnification and translation amount are calculated.

In addition, Non-Patent Document 1 discloses a technique related to scale change and rotation of an image. A histogram about the gradient of the pixels included in the image block is created, and the peak that is the maximum value of the histogram is assigned as the orientation of the key points. Then, the image is rotated in the orientation direction of the key point.

Japanese Unexamined Patent Publication No. 9-98424

Gradient-based feature extraction-SIFT and HOG- Hironobu Fujiyoshi Information Processing Society of Japan Research Report CVIM 160, pp. 211-224, September, 2007.

In the technique disclosed in Patent Document 1, the process of searching for a translation vector is complicated. In the technique disclosed in Non-Patent Document 1, the process of creating a histogram using the gradient intensity and the gradient direction of an image is complicated. Therefore, there is a problem that the amount of calculation for obtaining the relative enlargement / reduction ratio between the target block and the reference block increases. Enlargement or reduction may be referred to as "enlargement / reduction" below. In addition, the ratio of enlargement or reduction may be referred to as "scale ratio" below.

An object of the present invention is to provide a technique for calculating the ratio of expansion or contraction between regions with a small amount of calculation.

The image processing apparatus according to one embodiment corresponds to a reference area determination unit that determines a reference area that is a reference for enlargement or reduction of the target area, a plurality of sub-target areas in the target area, and each of the sub-target areas. Scale calculation that determines the enlargement or reduction ratio based on the eigenvalues of the principal components obtained by principal component analysis of a plurality of vectors based on the relative positional relationship with the plurality of subreference regions in the reference region. It has a part and.

According to the present invention, the ratio of expansion or contraction of the target area to the reference area can be calculated with a small amount of calculation.

It is a block diagram which shows the functional structure of an image processing apparatus. It is a block diagram which shows the hardware structure of an image processing apparatus. It is a flowchart of the whole processing by an image processing apparatus. It is a figure which shows an example of the target image. It is a figure which shows an example of a reference image. It is a flowchart of the 1st scaling calculation processing. It is a figure which shows the state that the sub-target area was created in the target area. It is a figure which shows the state that the sub-reference area is determined in the reference area. It is a figure which shows an example of the sub-target area normalized with respect to a sub-reference area. It is a figure which shows the eigenvector of the principal component of a plurality of vectors obtained in step S203. It is the flowchart of the 2nd scaling calculation processing. It is a figure which shows the state that the sub-reference area is created in the reference area. It is a figure which shows the state that the sub-target area was determined in the target area. It is a figure which shows the eigenvector of the principal component of a plurality of vectors obtained in step S303. It is a flowchart of scaling decision processing.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

In the present embodiment, an image coding device that performs differential coding of an image is illustrated as an example of a device that calculates the enlargement / reduction ratio of an image. In the difference coding, the difference between the image of the still image frame to be encoded (target image) and the image predicted from the image of the still image frame used for coding (reference image) (predicted image) is encoded. It is a coding method to be used. The reference image is, for example, an already encoded image of the previous frame. If a predicted image as close as possible to the target image is generated from the reference image, the difference between the target image and the predicted image becomes small, and as a result, the amount of data after encoding can be reduced. For example, if a predicted image as close as possible to the target image is obtained, the difference is almost eliminated, and the amount of data after encoding becomes small. On the contrary, if the prediction is unpredictable, for example, the difference between the target image and the reference image is encoded as it is, and the amount of data after encoding becomes large.

FIG. 1 is a block diagram showing a functional configuration of an image processing device. FIG. 2 is a block diagram showing a hardware configuration of the image processing device.

Referring to FIG. 1, the image processing device 1 includes a control unit 10 and a storage unit 20. The storage unit 20 stores the frame information 21 and the difference code information 22. The frame information 21 is image data of each frame constituting the moving image. The difference code information 22 is image data of each frame encoded by the difference coding.

The control unit 10 encodes the image of the frame using the frame information 21 stored in the storage unit 20, and records the encoded image data in the storage unit 20 as the difference code information 22.

Referring to FIG. 2, the hardware of the image processing apparatus 1 is a computer apparatus 400 as an example. The computer device 400 includes a control circuit 401, a storage device 402, a reading device 403, a recording medium 404, a communication interface 405, an input / output interface 406, an input device 407, and a display device 408. .. The control circuit 401, the storage device 402, the reading device 403, the communication interface 405, the input / output interface 406, and the display device 408 are connected to each other by the bus 410. The recording medium 404 is connected to the reading device 403. The input device 407 is connected to the input / output interface 406. The communication interface 405 is also connected to the network 409.

The control circuit 401 controls the entire computer device 400. The control circuit 401 is, for example, a processor. The control circuit 401 operates as the control unit 10 shown in FIG. 1 by executing an image processing program (not shown) recorded in the storage device 402.

The storage device 402 stores an image processing program that causes the control circuit 401 to function as the control unit 10. In addition, the storage device 402 stores various data for the control circuit 401 to operate. Further, the storage device 402 functions as the storage unit 20 in FIG.

The reading device 403 reads data from the removable recording medium 404 and writes the data to the recording medium 404.

The recording medium 404 is a non-temporary recording medium such as an SD (Secure Digital) memory card, an FD (Floppy Disk), a CD (Compact Disk), a DVD (Digital Versaille Disk), a BD (Blu-ray Disc: registered trademark), and a flash memory. It is a target recording medium. The recording medium 404 stores, for example, data given to the computer device 400, data generated by the computer device 400, and data acquired by the computer device 400. Further, the recording medium 404 may store the information in the storage unit 20 shown in FIG.

The communication interface 405 connects the computer device 400 and another device (not shown) in a communicable manner via the network 409.

The input / output interface 406 is an interface for connecting to various input / output devices such as the input device 407. The input / output interface 406 outputs the signal input from the input device 407 to the control circuit 401 via the bus 410. Further, the input / output interface 406 outputs the signal output from the control circuit 401 to the input / output device via the bus 410. The input device 407 is an example of an input / output device.

The input device 407 is an input device such as a keyboard and a mouse. The input device 407 accepts an information input operation by the user and outputs a signal corresponding to the input information to the input / output interface 406. For example, when the user inputs an instruction to start differential coding of video to the input device 407, the input device 407 outputs a signal indicating the instruction to the input / output interface 406.

The display device 408 displays various information on the screen under the control of the control circuit 401.

The network 409 connects the computer device 400 and other devices in a communicable manner.

Hereinafter, the details of the control unit 10 of FIG. 1 will be described.

Referring to FIG. 1, the control unit 10 includes a reference area determination unit 11, a scale calculation unit 12, a prediction image generation unit 13, and a difference coding unit 14.

The reference area determination unit 11 acquires image data of the target image to be encoded from the frame information 21 of the storage unit 20, and uses the target in the reference image for encoding the block area (target area) in the target image. A region similar to the region is determined as a reference region. "Similarity" means that the value of the index indicating the degree of similarity is large. Hereinafter, an index showing the degree of similarity is also referred to as a degree of similarity. The method for evaluating the similarity between images is not particularly limited, but the evaluation can be performed by, for example, the sum of squared errors of pixel values. For example, a region whose similarity with the target region exceeds a predetermined threshold value may be used as a reference region. Alternatively, a region having a high degree of similarity to the target region as compared with other regions in the reference image may be used as the reference region. When determining the reference area, the reference area determination unit 11 sequentially slides the candidates of the reference area in the reference image and calculates the similarity between the candidate area and the target area by brute force to obtain the target area. Search for areas with high similarity. Since the image of the target area is predicted by enlarging or reducing the reference area in the creation of the prediction image described later, the reference area serves as a reference for enlarging or reducing the target area.

The scaling calculation unit 12 performs principal component analysis of a plurality of vectors based on the relative positional relationship between the plurality of sub-target areas in the target area and the plurality of sub-reference areas in the reference area corresponding to each sub-target area. Based on the eigenvalues of the principal components obtained thereby, the ratio of enlargement or reduction of the target area to the reference area is determined.

According to the present embodiment, it is not necessary to perform complicated processing such as searching for a translation vector or creating a histogram using the gradient intensity and gradient direction of an image, and the expansion or contraction of the target area with respect to the reference area is not required. The ratio can be calculated with a small amount of calculation.

The prediction image generation unit 13 generates a prediction image in which the target image is predicted from the reference image by using an image in which the reference area is enlarged or reduced at a ratio determined by the scaling calculation unit 12.

The difference coding unit 14 encodes the difference between the predicted image generated by the predicted image generation unit 13 and the target image. The code data in which the difference is encoded is recorded in the storage unit 20 as the difference code information 22.

According to the present embodiment, the ratio is calculated with a small amount of calculation, and the predicted image of the target area is generated using the image obtained by enlarging or reducing the reference area with the ratio. Therefore, good difference coding can be performed with a small amount of calculation. It can be carried out.

The reference area determination unit 11 uses a plurality of areas in the reference image that can overlap with each other as candidate areas, and calculates the degree of similarity between the rotation candidate area obtained by rotating the candidate area by a predetermined rotation angle and the target area. , The reference area may be determined from a plurality of candidate areas based on the similarity. Since the reference area is searched using the similarity between the rotated area and the target area in the reference image, a good reference area can be determined, and as a result, good difference coding can be performed. it can. At this time, for example, a plurality of rotation angles are determined in advance, rotation candidate regions are created by rotating each candidate region by each rotation angle, and the similarity between the plurality of rotation candidate regions and the target region is brute-forced. The original candidate area of the rotation candidate area having the highest degree of similarity may be used as the reference area. Further, it can be said that the rotation angle of the rotation candidate region at this time is the relative rotation angle between the target region and the reference region.

FIG. 3 is a flowchart of overall processing by the image processing apparatus.

Referring to FIG. 3, first, in step S101, the reference area determination unit 11 determines a region in the reference image similar to the target region in the target image as a reference region.

FIG. 4 is a diagram showing an example of the target image. FIG. 5 is a diagram showing an example of a reference image. Here, it is assumed that a region having a high degree of similarity to the target region 110 in the target image 100 shown in FIG. 4 is searched from the reference image 200 shown in FIG. In this example, the reference area determination unit 11 sequentially slides the candidate area 211 which is a candidate of the reference area in the reference image 200, and rotates the candidate area 211 by a predetermined rotation angle to make it a rotation candidate area, and the target area 110 The candidate region 211, which is the source of the rotation candidate region having the highest similarity, is set as the reference region 210.

Next, in step S102, the scaling calculation unit 12 executes the first scaling calculation process. The first scaling operation is an operation related to enlargement or reduction when the reference area is viewed from the target area.

FIG. 6 is a flowchart of the first scaling calculation process.

Referring to FIG. 6, first, in step S201, the scaling calculation unit 12 creates a plurality of sub-target areas in the target area. FIG. 7 is a diagram showing how a sub-target area is created in the target area. As shown in FIG. 7, the scaling calculation unit 12 creates a plurality of sub-target areas (first sub-target areas) 121 at equal intervals on the circumference centered on the center of the target area 110. Here, eight sub-target areas 121 numbered from 1 to 8 are created.

Next, in step S202, the scaling calculation unit 12 searches for a region (sub-reference region) similar to each of the plurality of sub-target regions 121 in the reference region. FIG. 8 is a diagram showing how the sub-reference area is determined in the reference area. The scaling calculation unit 12 searches the reference area 210 for a region similar to each sub-target region 121 shown in FIG. 7, and searches for a region having a high degree of similarity in the sub-reference region (first) corresponding to the sub-target region 121. Sub-reference area) 221. As a specific example, the scaling calculation unit 12 sets the target area 110 and the reference area 210 about the center of the reference area 210 from the same position in the reference area 210 as the position where the sub target area 121 is cut out in the target area 110. While sliding the candidate area near the position rotated by the relative rotation angle of, the similarity between the candidate area and the sub-target area 121 is evaluated each time, and the highest degree of similarity is evaluated. The high candidate region may be defined as the subreference region 221.

After the sub-reference areas 221 corresponding to all the sub-target areas 121 are determined, then in step S203, the scaling calculation unit 12 has coordinates indicating the positions of the plurality of sub-target areas 121 with the center of the target area 110 as the origin. A plurality of vectors (first vector) obtained by normalizing the coordinate vectors indicating the positions of the plurality of sub-reference regions 221 with the center of the reference region 210 as the origin so that the magnitude of the vector becomes 1. The set of vectors obtained here shows the distribution of the enlargement or reduction ratio of the target area with respect to the reference area in each direction. FIG. 9 is a diagram showing an example of a normalized sub-target region. FIG. 9 shows a sub-target area 122 in which the distance from the origin O is increased or decreased with respect to the corresponding sub-reference area 221 as a result of normalization.

Next, in step S204, the scaling calculation unit 12 analyzes the plurality of vectors obtained in step S203 for principal component analysis to obtain the principal component (principal component A). The eigenvector of the principal component obtained here indicates the direction of expansion or contraction, and the eigenvalue indicates the ratio of expansion or contraction. Let the eigenvalue of the first principal component be α'and the eigenvalue of the second principal component be β'. Let the eigenvector of the first principal component be Vα'and the eigenvector of the second principal component be Vβ'. FIG. 10 is a diagram showing eigenvectors of the principal components of the plurality of vectors obtained in step S203. With reference to FIG. 10, the eigenvector of the first principal component is Vα'and the eigenvector of the second principal component is Vβ'.

Returning to FIG. 3, in step S103, the scaling calculation unit 12 executes the second scaling calculation process. The second scaling calculation process is a calculation process relating to scaling when the target area is viewed from the reference area.

FIG. 11 is a flowchart of the second scaling calculation process.

Referring to FIG. 11, first, in step S301, the scaling calculation unit 12 creates a plurality of sub-reference areas in the reference area. FIG. 12 is a diagram showing how a subreference area is created in the reference area. As shown in FIG. 12, the scaling calculation unit 12 creates a plurality of sub-reference regions (second sub-reference regions) 222 on the circumference centered on the center of the reference region 210 at equal intervals. Here, eight sub-reference areas 222 numbered from 1 to 8 are created.

Next, in step S302, the scaling calculation unit 12 searches for a region (sub-target region) similar to the plurality of sub-reference regions 222 in the target region. FIG. 13 is a diagram showing how a sub-target area is determined within the target area. The scaling calculation unit 12 searches for a region similar to each sub-reference region 222 shown in FIG. 12 in the target region 110, and searches for a region having a high degree of similarity in the sub-target region (second) corresponding to the sub-reference region 222. Sub target area) 122. As a specific example, the scaling calculation unit 12 sets the reference area 210 and the target area 110 about the center of the target area 110 from the same position in the target area 110 as the position where the sub-reference area 222 is cut out in the reference area 210. While sliding the candidate area near the position rotated by the relative rotation angle of, the similarity between the candidate area and the sub-reference area 222 was evaluated each time, and the highest degree of similarity was evaluated. The high candidate area may be defined as the sub-target area 122.

After the sub-target areas 122 corresponding to all the sub-reference areas 222 are determined, then in step S303, the scaling calculation unit 12 has coordinates indicating the positions of the plurality of sub-reference areas 222 with the center of the reference area 210 as the origin. A plurality of vectors (second vectors), which are coordinate vectors indicating the positions of the plurality of sub-target areas 122 with the center of the target area 110 as the origin, are obtained, which are normalized so that the magnitude of the vector is 1. The set of vectors obtained here shows the distribution of the enlargement or reduction ratio of the target area with respect to the reference area in each direction.

Next, in step S304, the scaling calculation unit 12 analyzes the plurality of vectors obtained in step S303 by principal component analysis to obtain the principal component (main component B). The eigenvector of the principal component obtained here indicates the direction of expansion or contraction, and the eigenvalue indicates the ratio of expansion or contraction. Let α be the eigenvalue of the first principal component and β be the eigenvalue of the second principal component. Let Vα be the eigenvector of the first principal component and Vβ be the eigenvector of the second principal component. FIG. 14 is a diagram showing eigenvectors of the principal components of the plurality of vectors obtained in step S303. With reference to FIG. 14, the eigenvector of the first principal component is Vα, and the eigenvector of the second principal component is Vβ.

The order in which the first scaling calculation process and the second scaling calculation process described above are executed is not particularly limited. The second scaling calculation process may be executed before the first scaling calculation process.

Returning to FIG. 3, in step S104, the scaling calculation unit 12 executes the scaling determination process. The scaling determination process is a process of determining the direction and ratio of scaling with respect to the reference area of the target area based on the processing result of the first scaling calculation process and the processing result of the second scaling calculation process.

FIG. 15 is a flowchart of the scaling determination process.

With reference to FIG. 15, the scaling calculation unit 12 determines in step S401 whether or not αα ′ ≦ 1 and ββ ′ <1. If the determination in step S401 is YES, the scaling calculation unit 12 reduces in step S402 by the ratio indicated by the eigenvalue α in the direction indicated by the eigenvector Vα and by the ratio indicated by the eigenvalue β in the direction indicated by the eigenvector Vβ. Judge that

If the determination in step S401 is NO, the scaling calculation unit 12 determines in step S403 whether or not αα ′> 1 and ββ ′ ≧ 1. If the determination in step S403 is YES, the scaling calculation unit 12 expands in the direction indicated by the eigenvector Vα'at the ratio indicated by the eigenvalue α'in step S404, and the eigenvalue β'indicates in the direction indicated by the eigenvector Vβ'. Judge that it is expanding at a rate.

If the determination in step S403 is NO, the scaling calculation unit 12 determines in step S405 that the angle formed by αα'> 1 and ββ'<1 and the eigenvector Vα and the eigenvector Vα'is smaller than a predetermined angle (for example, 45 °). Judge whether or not. If the determination in step S405 is YES, the scaling calculation unit 12 recalculates scaling in step S406. In the recalculation, the scaling calculation unit 12 performs principal component analysis on the sum set of the first vector near the eigenvector Vα'and the second vector near the eigenvector Vβ, and expands or expands the eigenvalues of the obtained principal components. The reduction ratio and the eigenvectors are the direction of expansion or contraction. The vector near the eigenvector Vα'means a vector whose direction is close to the eigenvector Vα', that is, a vector whose direction is oriented within a predetermined range with the direction of the eigenvector Vα'. Further, in this example, the condition that the angle formed by the eigenvector Vα and the eigenvector Vα'is smaller than a predetermined angle (for example, 45 °) is that the eigenvector Vα and the eigenvector Vα'should be in originally close directions. , Excludes cases where these angles are large.

If the determination in step S405 is NO, the scaling calculation unit 12 determines in step S407 whether or not αα ′ = 1 and ββ ′ = 1. If the determination in step S407 is YES, the scaling calculation unit 12 determines in step S408 that the target area is neither expanded nor contracted with respect to the reference area. If the determination in step S407 is NO, the scaling calculation unit 12 determines in step S409 that it is unclear whether the target area is expanded or contracted with respect to the reference area.

Returning to FIG. 3, in step S105, the prediction image generation unit 13 generates a prediction image in which the target image is predicted from the reference image. At that time, the prediction image generation unit 13 uses an image in which the reference area is enlarged or reduced at a ratio determined by the scaling calculation unit 12 as the area corresponding to the target area 110 of the prediction image. Therefore, the predicted image generation unit 13 generates a motion vector indicating the amount of movement from the reference region to the target region, and a difference image between the image predicted from the image in the reference region and the image in the target region. The generated data is included in the predicted image data. An image of a reference region having the same magnification may be used for the region corresponding to the target region where it is determined whether the scaling calculation unit 12 is expanding or contracting.

Subsequently, in step S106, the difference coding unit 14 encodes the target image using the difference image between the predicted image and the target image generated by the prediction image generation unit 13, and obtains the code data obtained by encoding the difference. It is recorded in the storage unit 20 as the difference code information 22.

As described above, in the image processing apparatus 1 of the present embodiment, the reference area determination unit 11 determines a reference area as a reference for enlargement or reduction of the target area. Then, the scaling calculation unit 12 mainly uses a plurality of vectors based on the relative positional relationship between the plurality of sub-target areas in the target area and the plurality of sub-reference areas in the reference area corresponding to each of the sub-target areas. The ratio of enlargement or reduction is determined based on the eigenvalues of the principal components obtained by component analysis. By doing so, it is not necessary to perform complicated processing such as searching for a translation vector or creating a histogram using the gradient intensity and gradient direction of the image, and the expansion or contraction of the target area with respect to the reference area is not required. The ratio can be calculated with a small amount of calculation.

Further, the scaling calculation unit 12 analyzes a plurality of vectors for principal component analysis, calculates the eigenvalues and eigenvectors of the principal components, and determines the ratio of the target area to the reference area in the direction determined based on the eigenvectors. Calculate that it is enlarged or reduced with. By doing so, not only the enlargement / reduction ratio but also the enlargement / reduction direction can be obtained, so that it is possible to obtain how much the enlargement / reduction is in which direction.

Further, the scaling calculation unit 12 calculates the first principal component and the second principal component, and enlarges or reduces the first principal component and the second principal component in a direction determined based on the eigenvector of the first principal component at a ratio determined based on the eigenvalue of the first principal component. , It is calculated that the enlargement or reduction is performed at a ratio determined based on the eigenvalue of the second principal component in the direction determined based on the eigenvector of the second principal component. By doing so, the enlargement or reduction can be obtained for each of the two orthogonal axes on the image plane, so that the enlargement or reduction of the image can be well represented.

Further, the scaling calculation unit 12 defines a plurality of first sub-target areas on a predetermined circumference included in the target area. Further, the scaling calculation unit 12 determines a plurality of first sub-reference areas corresponding to each of the plurality of first sub-target areas in the reference area based on the similarity with each of the plurality of first sub-target areas. .. Further, the scaling calculation unit 12 determines a plurality of first vectors based on the positions of the plurality of first sub-reference regions in the reference region. Then, the scaling calculation unit 12 calculates the principal component A by performing principal component analysis on the plurality of first vectors.

In addition, the scaling calculation unit 12 defines a plurality of second sub-reference regions on a predetermined circumference included in the reference region. Further, the scaling calculation unit 12 determines a plurality of second sub-target areas corresponding to each of the plurality of second sub-reference areas in the target area based on the similarity with each of the plurality of second sub-reference areas. .. Further, the scaling calculation unit 12 determines a plurality of second vectors based on the positions of the plurality of second sub-target regions in the target region. Then, the scaling calculation unit 12 calculates the principal component B by performing principal component analysis on the plurality of second vectors.

Further, the scaling calculation unit 12 determines the scaling ratio based on the principal component A and the principal component B. By doing so, the enlargement / reduction ratio is calculated using the principal components obtained from both the principal component analysis when the reference area is viewed from the target area and when the target area is viewed from the reference area. The enlargement or reduction of the image can be well represented.

Further, the scaling calculation unit 12 has a plurality of first subs with the center of the reference region as the origin so that the size of the coordinate vector indicating the position of the plurality of first sub target regions is 1 with the center of the target region as the origin. A vector obtained by normalizing the coordinate vector indicating the position of the reference region is defined as a plurality of first vectors. Further, the scaling calculation unit 12 is normalized so that the size of the coordinate vector indicating the position of the plurality of second sub-reference regions is 1 with the center of the reference region as the origin, and the center of the target region is the origin. The coordinate vectors indicating the positions of the second sub-target areas of the above are defined as a plurality of second vectors.

Further, the scaling calculation unit 12 includes the eigenvalue α'and the eigenvector Vα' of the first principal component of the principal component A, the eigenvalues β'and the eigenvector Vβ' of the second principal component of the principal component A, and the first principal component B. The eigenvalue α and the eigenvector Vα of the principal component and the eigenvalue β and the eigenvector Vβ of the second principal component of the principal component B are calculated.

Then, if αα ′ ≦ 1, ββ ′ <1, the scaling calculation unit 12 reduces in the direction indicated by the eigenvector Vα at a ratio based on the eigenvalue α, and is based on the eigenvalue β in the direction indicated by the eigenvector Vβ. It is assumed that the ratio is shrinking. Further, if αα ′> 1, ββ ′ ≧ 1, the scaling calculation unit 12 expands in the direction indicated by the eigenvector Vα ′ at a ratio based on the eigenvalue α ′, and the eigenvalue calculation unit 12 expands in the direction indicated by the eigenvector Vβ ′. It is assumed that the ratio is based on β'. Further, if the angle formed by αα'> 1, ββ'<1, the eigenvector Vα and the eigenvector Vα'is smaller than the predetermined angle, the scaling calculation unit 12 is oriented in the direction of the eigenvector Vα'and within the predetermined range. Principal component analysis is performed on the sum set of one vector and the direction of the eigenvector Vβ and the second vector whose direction is oriented within a predetermined range, the eigenvalue of the obtained principal component is used as the scaling factor, and the eigenvector is defined as the scaling direction. And. Further, the scaling calculation unit 12 assumes that the target region and the reference region have the same magnification if αα ′ = 1 and ββ ′ = 1.

Further, in the present embodiment, the target area is an area included in the target image to be encoded, and the reference area is an area included in the reference image used for encoding the target image. Then, in the image processing device 1, the prediction image generation unit 13 generates a prediction image that predicts the target image by using an image in which the reference area is enlarged or reduced at the above ratio. The difference coding unit 14 encodes the difference between the predicted image and the target image. The reference area determination unit 11 uses a plurality of overlapping regions in the reference image as candidate regions, calculates the degree of similarity between the rotation candidate region obtained by rotating the candidate region by a predetermined rotation angle and the target region, and calculates the similarity thereof. The reference area is determined from the candidate areas based on the similarity. By doing so, the ratio is calculated with a small amount of calculation, and the predicted image of the target area is generated using the image in which the reference area is enlarged or reduced at that ratio, so that good difference coding can be performed with a small amount of calculation. can do. In addition, since the reference area is searched using the similarity between the rotated area in the reference image and the target area, a good reference area can be determined, and as a result, good difference coding is performed. be able to.

The embodiments of the present invention described above are examples for the purpose of explaining the present invention, and the scope of the present invention is not limited to those embodiments. One of ordinary skill in the art can practice the present invention in various other aspects without departing from the scope of the present invention. The devices of each of the above embodiments have been illustrated as being realized by executing a software program on a processor, but the present invention is not limited thereto. A part or all the functions of the control unit 10 may be realized by hardware. Further, a part or all the functions of the control unit 10 may be realized by a dedicated processor.

1 ... Image processing device, 10 ... Control unit, 11 ... Reference area determination unit, 12 ... Scale calculation unit, 13 ... Prediction image generation unit, 14 ... Difference coding unit, 20 ... Storage unit, 21 ... Frame information, 22 ... Difference code information, 100 ... target image, 110 ... target area, 121 ... sub target area, 122 ... sub target area, 200 ... reference image, 210 ... reference area, 211 ... candidate area, 221 ... sub reference area, 222 ... sub Reference area, 400 ... Computer device, 401 ... Control circuit, 402 ... Storage device, 403 ... Reading device, 404 ... Recording medium, 405 ... Communication interface, 406 ... Input / output interface, 407 ... Input device, 408 ... Display device, 409 ... network, 410 ... bus

Claims (9)

A reference area determination unit that determines a reference area that serves as a reference for expanding or contracting the target area,
By principal component analysis of a plurality of vectors based on the relative positional relationship between the plurality of sub-target regions in the target region and the plurality of sub-reference regions in the reference region corresponding to each of the sub-target regions. A scaling calculation unit that determines the scaling ratio based on the eigenvalues of the obtained principal components, and
An image processing device having. The scaling calculation unit analyzes the plurality of vectors with principal components to calculate the eigenvalues and eigenvectors of the principal components, and the eigenvalues in a direction in which the target region is determined with respect to the reference region based on the eigenvectors. Calculates that it is expanding or contracting at a rate determined based on
The image processing apparatus according to claim 1. The scaling calculation unit calculates the first principal component and the second principal component, and enlarges or reduces the first principal component and the second principal component in a direction determined based on the eigenvector of the first principal component at a ratio determined based on the eigenvalue of the first principal component. , Calculates that the enlargement or reduction is performed at a ratio determined based on the eigenvalue of the second principal component in the direction determined based on the eigenvector of the second principal component.
The image processing apparatus according to claim 2. The scaling calculation unit
A plurality of first sub-target areas are defined on a predetermined circumference in the target area, and the plurality of first sub-targets are set in the reference area based on the degree of similarity with each of the plurality of first sub-target areas. defining a plurality of first sub-reference region corresponding to each region, defining a first vector of multiple based on the position of the plurality of first sub-reference area in the reference area, the main the plurality of first vector Principal component A is calculated by component analysis,
A plurality of second sub-reference regions are defined in the reference region on a predetermined circumference, and the plurality of second sub-reference regions are defined in the target region based on the degree of similarity with each of the plurality of second sub-reference regions. defining a plurality of second sub-target region corresponding to each region, defining a second vector of multiple based on the position of the plurality of second sub-target region in the target area, the main the plurality of second vectors Principal component B is calculated by component analysis,
The enlargement or reduction ratio is determined based on the main component A and the main component B.
The image processing apparatus according to claim 1. The scaling calculation unit
The positions of the plurality of first sub-reference regions with the center of the reference region as the origin so that the size of the coordinate vector indicating the positions of the plurality of first sub-target regions is 1 with the center of the target region as the origin. The vector obtained by normalizing the coordinate vector indicating the above is defined as the plurality of first vectors.
The plurality of second sub-objects are normalized so that the size of the coordinate vector indicating the position of the plurality of second sub-reference regions is 1 with the center of the reference region as the origin. The coordinate vector indicating the position of the region is defined as the plurality of second vectors.
The image processing apparatus according to claim 4. The scaling calculation unit
The eigenvalues α ′ and eigenvectors Vα ′ of the first principal component of the principal component A, the eigenvalues β ′ and the eigenvectors Vβ ′ of the second principal component of the principal component A, and the eigenvalues α of the first principal component of the principal component B. And the eigenvector Vα, and the eigenvalue β and the eigenvector Vβ of the second principal component of the principal component B are calculated.
If αα ′ ≦ 1, ββ ′ <1, it is assumed that the eigenvector Vα is reduced in the direction indicated by the eigenvalue α and the eigenvector Vβ is reduced in the direction indicated by the eigenvalue β.
If αα ′> 1, ββ ′ ≧ 1, the eigenvector Vα ′ expands in the direction indicated by the eigenvalue α ′, and the eigenvector Vβ ′ expands in the direction indicated by the eigenvalue β ′. And
αα'> 1, ββ'<1, If the angle between the eigenvector Vα and the eigenvector Vα'is smaller than the predetermined angle, the direction of the eigenvector Vα'and the first vector whose direction is within the predetermined range and the eigenvector Vβ Principal component analysis is performed on the sum set of the direction of and the second vector whose direction is within a predetermined range, the eigenvalues of the obtained principal components are set as the scaling ratio, and the eigenvectors are set as the scaling direction.
If αα ′ = 1 and ββ ′ = 1, it is assumed that the target region and the reference region have the same magnification.
The image processing apparatus according to claim 4. The target area is a region included in the target image to be encoded, and the reference region is a region included in the reference image used for encoding the target image.
The image processing device is
A prediction image generation unit that generates a prediction image that predicts the target image by using an image obtained by enlarging or reducing the reference area at the ratio.
It further has a difference coding unit that encodes the difference between the predicted image and the target image.
The reference area determination unit calculates the degree of similarity between the rotation candidate area obtained by rotating the candidate area by a predetermined rotation angle and the target area, using a plurality of regions in the reference image that can overlap each other as candidate areas. Then, the reference region is determined from the candidate regions based on the similarity.
The image processing apparatus according to claim 1. The reference area determination unit determines the reference area that serves as a reference for expanding or contracting the target area .
The scaling calculation unit obtains by principal component analysis of a plurality of vectors based on the relative positional relationship between a plurality of sub-target regions in the target region and a plurality of sub-reference regions in the reference region corresponding to each other. The enlargement or reduction ratio is determined based on the eigenvalues of the principal components.
Image processing method. The reference area determination unit determines the reference area that serves as a reference for expanding or contracting the target area .
The scaling calculation unit obtains by principal component analysis of a plurality of vectors based on the relative positional relationship between a plurality of sub-target regions in the target region and a plurality of sub-reference regions in the reference region corresponding to each other. The enlargement or reduction ratio is determined based on the eigenvalues of the principal components.
An image processing program characterized by causing a computer to perform processing.

Images