JP5338741B2 - Image processing apparatus, image processing program, information processing apparatus, and information processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To derive target coordinates in a voting space with high accuracy and light operation loads in voting processing which is used for a PSA method and Hough transformation or the like. <P>SOLUTION: At a mid-stage of the voting processing for sequentially integrating voting values in an intersection area crossing each of a plurality of planes or lines for each of a plurality of voting areas formed by discretizing voting spaces, an image processing processor 13 performs control so that an excluded area which is excluded from the object for the voting processing is specified based on an integrated value of the voting values by executing an excluded area specification module 134, and the voting processing is executed for only the remaining voting area obtained by excluding the excluded area from all the voting areas by executing a voting area restriction module 135. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to an image processing apparatus and an image processing device that perform processing for integrating voting values in regions intersecting with a plurality of lines and deriving target coordinates based on the integrated values for each of a plurality of voting regions in a voting space. The present invention relates to a processing program, an information processing apparatus, and an information processing program.

  Corresponding point search processing using a plurality of images is an important image processing technique in various fields such as moving image tracking, three-dimensional image measurement, and computer vision. Corresponding point search processing derives the coordinates of corresponding points in the second image corresponding to the point of interest specified in the first image based on two images in time series or two images constituting a stereo image. It is processing to do. The corresponding point is a point in the second image where an image having a high correlation with the image of the target point in the first image exists. Recently, in the corresponding point search process, it is required to derive the coordinates of the corresponding point with high accuracy.

  Conventionally, there is a method of phase difference analysis (Phase-difference Spectrum Analysis) as a method of high-accuracy corresponding point search processing. Hereinafter, this method is referred to as a PSA method. The PSA method is shown in, for example, Non-Patent Document 1, Non-Patent Document 2, Patent Document 1, and Patent Document 2.

  Hereinafter, the PSA method will be described with reference to FIGS. In the PSA method, the coordinates (dx0, dy0) of the deviation of the coordinates (x02, y02) of the corresponding point in the second image with respect to the coordinates (x01, y01) of the target point in the first image are derived by the procedure shown below. It is a technique. Hereinafter, the coordinates of the deviation are referred to as corresponding point deviation coordinates. Note that (dx0, dy0) = (x02-x01, y02-y01).

  In the PSA method, a first phase image and a second phase image, which are phase images in the spatial frequency domain corresponding to the first image and the second image, are calculated. These phase images are obtained by performing a two-dimensional discrete Fourier transform on each of the first image and the second image.

  Next, for each frequency component of both phase images, a phase difference Δθ (u, v) corresponding to the frequency component (u, v) between the first phase image and the second phase image is calculated. U and v are coordinate values in the frequency space, and are parameters for specifying the frequency component.

  In the PSA method, a voting space 100 indicating a possible range of the corresponding point deviation coordinates (dx0, dy0) is set. The voting space 100 is discretized by being divided into a plurality of unit regions at a predetermined spatial period. The resolution (accuracy) for deriving the corresponding point deviation coordinates (dx0, dy0) is determined according to the spatial period.

  FIG. 10 is a schematic diagram illustrating an example of the voting space 100. In the example shown in FIG. 17, the voting space 100 is discretized into 10 sections in each of the ΔX axis direction and the ΔY axis direction. A plurality of unit areas that are segmented (discretized) in a predetermined spatial period in the voting space 100 are referred to as voting areas 20. Setting the voting space 100 means that the image processing apparatus secures a data area corresponding to the array of voting areas 20 in the memory. That is, each voting area 20 corresponds to each element data in the data area secured in the memory. The size of the voting space 100 and the spatial period of the voting area 20 are determined in advance according to the application target.

  In the PSA method, the frequency component (u, v), the phase difference Δθ (u, v) based on the frequency component, and the coordinate (dx, dy) in the voting space are changed into a predetermined expression as variables. , v) and a phase difference Δθ (u, v) calculated based on the frequency component thereof, a straight line (primary expression) in the voting space 100 is derived. The linear expression thus derived is generally called a voting function. The frequency component (u, v) and the phase difference Δθ (u, v) corresponding to the frequency component (u, v) are changed to the frequency component (u, v), the phase difference Δθ (u, v) and the coordinates (dx, A straight line in the voting space 100 obtained by applying to a predetermined expression having dy) as a variable (parameter) is a candidate for the corresponding point deviation coordinate (dx0, dy0) determined according to the frequency component (u, v). Represents the position. Hereinafter, this straight line is referred to as a voting straight line.

  FIG. 11 is a schematic diagram showing an example of the voting space 100 including one voting line 32 corresponding to one certain frequency component (u, v). By applying the frequency component (u, v) and the phase difference Δθ (u, v) to the voting function, a linear expression representing the voting line 32 that is a straight line in the coordinate system of the voting space 100 is obtained. In the PSA method, the image processing apparatus specifies the voting area 20 that intersects the voting line 32 plotted in the voting space 100. Hereinafter, the voting area 20 specified in this way is referred to as an intersection area 20a. In FIG. 11, the intersection region 20a is shaded with a sand pattern.

  Further, in the PSA method, for each frequency component (u, v), the derivation of the voting line 32 and the specification of the intersecting region 20a intersecting with the voting line 32 are sequentially repeated. Further, every time the intersection area 20a is specified, a process of sequentially adding a predetermined vote value to the intersection area 20a is executed. The vote value is normally “1”, but a value that changes by being weighted according to a predetermined criterion may be adopted as the vote value.

  FIG. 12 is a schematic diagram illustrating an example of a state of the voting space 100 including a plurality of voting straight lines 32 corresponding to a plurality of frequency components (u, v). In FIG. 12, the crossing region 20a that intersects with only one voting line 32 is shaded with sand, and the intersecting region 20a that intersects with a plurality of voting lines 32 is shaded or blackened. ing. The state shown in FIG. 12 is obtained by repeating the derivation of the voting line 32 and the identification of the intersecting region 20a for each of the plurality of frequency components (u, v).

  In the PSA method, for all frequency components (u, v), when the identification of the intersecting region 20a and the integration of the vote values are completed, the coordinates of the peak positions of the integrated values of the vote values are the corresponding point deviation coordinates ( dx0, dy0). Further, the coordinates (x02, y02) of the corresponding point in the second image are calculated based on the coordinates (x01, y01) of the attention point and the corresponding point deviation coordinates (dx0, dy0) in the first image.

  The coordinates of the peak position of the integrated value of the voting value are, for example, the coordinates of the intersecting region 20a (voting region 20) where the integrated value of the voting value is the maximum, or a set of intersecting regions 20a using the integrated value of the voting value as a weighting factor. The coordinates of the center of gravity position. FIG. 12 shows a state where the intersection area 20a where the integrated value of the vote value is the maximum is specified as the peak position 230 (black portion) of the integrated value of the vote value.

  A voting process for accumulating voting values in a region where a plurality of voting regions, which are discrete partial regions in the voting space, and a plurality of straight lines intersect is also performed in the well-known Hough transform. The Hough transform is a technique for extracting a figure such as a straight line, a circle, or an ellipse from an image, and is described in detail in, for example, Patent Document 3.

  For example, when a circle is extracted from an image by Hough transform, an edge image is generated by edge detection processing on the original image. Further, a voting space corresponding to each pixel in the edge image is set. This voting space is a three-dimensional finite space in which the values of three parameters of the coordinates (a, b) of the center position for specifying the circle and the radius r of the circle are coordinate values.

  Further, among each of a plurality of voting areas, which are partial areas obtained by discretizing the voting space (parameter space), intersects with a conical surface in the voting space corresponding to a circle centered on one pixel in the edge image. A voting process is performed in which a voting value is added in association with a voting area that intersects with the conical surface. The voting process is sequentially executed for each of the plurality of conical surfaces corresponding to each pixel of the edge image, whereby the voting values are integrated in the voting area intersecting with each of the plurality of conical surfaces. Then, the coordinates of the peak position of the integrated value of voting values in the voting space are derived. The coordinates of the peak position are information for specifying the position and radius of a circle existing in the edge image.

  As described above, the voting process is executed in both the PSA method and the Hough transform. In the voting process, a finite voting space having a plurality of parameter values as coordinate values is set, and information for specifying a plurality of faces or lines in the voting space is given. That is, in the PSA method, a plurality of straight lines corresponding to each frequency component (u, v) are given, and in the Hough transform for extracting a circle, a plurality of conical surfaces corresponding to each pixel of the edge image are given.

  Further, in the voting process, for each of a plurality of voting areas that are partial areas in which the voting space is discretized, a voting area that intersects one face or line is specified, and the voting value is associated with the specified voting area. Are sequentially executed for each of a plurality of given surfaces or lines. As a result, the voting values are integrated in the voting area intersecting the given surface or straight line. Then, the peak coordinates of the integrated value of the vote value are derived as target coordinates in the vote space. In the PSA method, the target coordinates are corresponding point deviation coordinates, and in the Hough transform for extracting a circle, the coordinates are corresponding to the circle existing in the edge image.

JP 2001-175864 A JP 2003-099787 A

Isamu KOUZAKI, Tomonori KANEKO, Minoru ITO, "Precise and reliable image shift detection by a new phase-difference spectrum analysis", IEICE TRANSACTIONS on Information and Systems Academic Journal 2004, E87D / 1 58-65 Isamu Kozuki and 4 others, “Highly accurate image shift detection method from phase difference”, Image Recognition and Understanding Symposium (MIRU 2002), published July 2002, II-379-II-384

  However, in the conventional voting process, it is necessary to determine whether or not the voting values should be accumulated for all the voting areas and all the faces or lines in the voting space. Also, by reducing the number of voting area sections in the voting space, the calculation load of the voting process is reduced, but the calculation accuracy of the target coordinates in the voting space deteriorates. Therefore, the conventional voting process has a problem that it is difficult to achieve both high accuracy and high speed in deriving the target coordinates.

  An object of the present invention is to be able to derive a target coordinate in a voting space with high accuracy and a light calculation load in voting processing used for the PSA method and the Hough transform.

The image processing apparatus according to the present invention is based on the first phase image and the second phase image that are phase images in the spatial frequency domain corresponding to each of two images (first image and second image). An apparatus for executing a corresponding point search process for deriving the coordinates of a corresponding point in the second image corresponding to a point of interest specified in an image, and includes the following components.
(1) The first component is a phase difference calculation unit that calculates a phase difference corresponding to a frequency component between the first phase image and the second phase image.
(2) The second component applies the phase difference corresponding to the frequency component and the frequency component to a predetermined formula, whereby the second image with respect to the coordinates of the attention point in the first image. Is a straight line deriving unit for deriving a straight line in a finite voting space indicating a possible range of the corresponding point deviation coordinates which are the coordinates of the deviation of the coordinates of the corresponding points.
(3) The third component specifies the voting area intersecting one straight line for each of a plurality of voting areas, which are partial areas in which the voting space is discretized, and the specified voting A voting processing unit that performs a voting process for accumulating the voting values by sequentially executing a process of adding voting values in association with a region for each of the straight lines corresponding to the plurality of frequency components.
(4) The fourth component is excluded from the object of the voting process based on the integrated value of the voting value in the middle of the voting process for a part of the plurality of straight lines. An exclusion area specifying unit that specifies an exclusion area that is the voting area.
(5) The fifth component controls the voting process to be executed only for the remaining voting areas excluding the excluded area from all the voting areas after the excluded area is specified. It is a voting restriction section.
(6) A sixth component is a target coordinate deriving unit that derives the corresponding point deviation coordinates based on the integrated value of the vote values and the coordinates of the vote area where the vote values are integrated.

In addition, the image processing apparatus according to the present invention further includes each component shown below, and the exclusion region specifying unit specifies the exclusion region based on a determination result of the level determination unit shown below. It is possible.
(7) The seventh component is for determination that sequentially selects a partial area for determination consisting of a set of the voting areas that occupy a part and a series of ranges in the voting space from all the voting areas. It is a partial area selection part.
(8) In the eighth component, a representative value of the integrated value of the voting values corresponding to the voting area in the determination partial area is preset for each selected determination partial area. It is a level determination unit that determines which of a plurality of levels showing a relative magnitude relationship with respect to a threshold value corresponds to.

  Further, it is conceivable that the level determining unit determines the level of the representative value as at least one of a relatively large high level and a relatively small low level based on the threshold value. In this case, the exclusion area specifying unit obtains the high-level determination result among all the voting areas when the determination partial area from which the high-level determination result is obtained exists. The remaining voting area excluding the voting area within a predetermined area including the determination partial area is specified as the exclusion area.

  Further, the level determination unit determines the level of the representative value as at least one of a relatively large high level, a relatively small low level, and a medium level therebetween based on the threshold value. It is also possible. In this case, the exclusion area specifying unit includes the determination partial area from which the high-level determination result is obtained and the determination partial area from which the medium-level determination result is obtained does not exist. In addition, among all the voting areas, the remaining voting areas excluding the voting areas within the range including the determination partial area where the high-level determination result is obtained are specified as the exclusion areas.

  In the case where the level determination unit determines that the representative value level is at least one of a high level and a low level, the exclusion region specifying unit obtains the determination result of the high level. It is also conceivable that the determination partial area from which the low-level determination result is obtained is specified as the exclusion area.

  Further, when the level determination unit determines that the representative value level is at least one of a high level, a low level, and an intermediate level between them, the exclusion area specifying unit obtains the determination result of the high level. It is also conceivable to specify the determination partial region from which the low-level determination result is obtained as the exclusion region when the determined determination partial region exists.

  In addition, the image processing apparatus according to the present invention may further include a threshold setting unit that sets the threshold according to the number of the straight lines for which the voting process has been completed in the intermediate stage. On the other hand, it is also conceivable that the threshold value is a predetermined fixed value.

  Further, it is conceivable that the determination partial area is each of the voting areas, and the representative value is an integrated value of the voting values in each of the voting areas.

It is also conceivable that the image processing apparatus according to the present invention further includes the following constituent elements.
(9) A ninth component is a correlation for calculating an index of a correlation degree for each frequency component between a first frequency image corresponding to the first image and a second frequency image corresponding to the second image. It is a degree calculation part. The index of the correlation is, for example, the power for each frequency component in the cross spectrum between the first frequency image and the second frequency image.
(10) The tenth component is an order setting unit that sets the order of the straight lines applied to the voting process in order from the one corresponding to the frequency component having the high correlation degree based on the correlation degree index. is there.

  In the image processing apparatus according to the present invention, the excluded area specifying unit may execute processing for specifying the excluded area in a plurality of predetermined intermediate stages.

The present invention is not limited to image processing, and may be configured as an information processing apparatus that performs voting processing. That is, the information processing apparatus according to the present invention includes the following constituent elements.
(1) The first component is a partial region in which the voting space is discretized when information for specifying a plurality of faces or lines in a finite voting space having a plurality of parameter values as coordinate values is given. The voting area that intersects one face or the line for each of the plurality of voting areas is a process of adding a voting value in association with the specified voting area. A voting processing unit that performs voting processing for accumulating the voting values by sequentially executing each line.
(2) The second component may be a target of the voting process based on the integrated value of the voting values in the middle of the voting process for a part of the plurality of surfaces or the lines. An exclusion area specifying unit that specifies an exclusion area that is the voting area to be excluded.
(3) The third component controls so that the voting process is executed only for the remaining voting areas excluding the excluded areas from all the voting areas after the excluded areas are specified. It is a voting restriction section.
(4) A fourth component is a target coordinate deriving unit that derives a target coordinate in the voting space based on the integrated value of the voting value and the coordinates of the voting area where the voting value is integrated. .

  Further, the present invention is configured as an image processing program or an information processing program for causing a computer (processor) to execute processing executed by each component included in the image processing apparatus or information processing apparatus according to the present invention described above. It is also conceivable.

  Also in the present invention, as in the conventional voting process, voting values are accumulated for a specific voting area that intersects a plane or a straight line determined according to a frequency component, etc., among a plurality of voting areas. Target coordinates are derived according to the distribution of the integrated values. Here, the target coordinates are, for example, the coordinates of the deviation between the coordinates of the point of interest and the coordinates of the corresponding point. In the present invention, the voting area that is the target of the voting process is limited to a part of the area after the middle stage of the voting process. For example, in the middle stage of the voting process, the remaining voting area obtained by removing the area where the voting value integrated value is in the high level and the adjacent area, or the voting value integrated value is in the low level. Certain areas are excluded from the voting process. Therefore, even when the voting space is discretized with high spatial resolution and a large number of voting areas are set, the voting process for the non-important voting areas is omitted after the intermediate stage, and the calculation load is kept low. Therefore, according to the present invention, when the target coordinates in the voting space are derived by the voting process, the target coordinates can be derived with high accuracy and a relatively light calculation load.

  By the way, when the voting space is divided into a large number of small voting areas, it is difficult for a situation where intersections where planes or straight lines intersect to concentrate on one voting area in the middle of the voting process. For this reason, when the status of the integrated value of voting values is determined for each voting area, it is not possible to correctly determine the distribution of intersections and specify an appropriate excluded area. Therefore, in the present invention, for each determination partial area consisting of a set of voting areas that occupy a part and a series of ranges in the voting space, the exclusion area is specified based on the level determination of the representative value of the integrated value of the voting value It is preferred that this is done. As a result, regardless of the size of the voting area, it is possible to specify an appropriate excluded area that reflects the distribution of intersections in the voting space.

  In the image processing apparatus according to the present invention, it is conceivable that the order of straight lines applied to the voting process is set in order from the one corresponding to the frequency component having a high degree of correlation between the two frequency images. Thereby, in a relatively early stage, the distinction between the area where the intersections are concentrated and the area where the intersections hardly exist is clarified, and the exclusion area can be appropriately specified at a relatively early stage. .

1 is a schematic block diagram of an image processing apparatus according to an embodiment of the present invention. It is a flowchart showing the procedure of the corresponding point search process by the image processing apparatus which concerns on embodiment of this invention. It is a flowchart showing the procedure of the voting area | region restriction | limiting process by the image processing apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows the 1st example of the process which selects the partial area for determination from voting space in a voting area restriction | limiting process. It is a schematic diagram which shows the 2nd example of the process which selects the partial area for determination from voting space in a voting area restriction | limiting process. It is a schematic diagram which shows an example of the voting space divided into two areas of a high level and a low level by the 1st level determination process based on the integrated value of a vote value. It is a schematic diagram which shows an example of the voting space divided into two areas of a high level and a low level by the 2nd level determination process based on the integrated value of a vote value. It is a schematic diagram which shows an example of the voting space divided into three area | regions of a high level, a medium level, and a low level by the 3rd level determination process based on the integrated value of a vote value. It is a schematic diagram which shows an example of the voting space where the exclusion area | region was set based on the result of the 2nd level determination process. It is a schematic diagram which shows an example of voting space. It is a schematic diagram which shows an example of the voting space containing one voting line. It is a schematic diagram which shows an example of the voting space containing a some voting line.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. The following embodiments are examples embodying the present invention, and do not limit the technical scope of the present invention. In the following description, unless otherwise specified, the term “image” is used in the same meaning as image data.

  First, the configuration of the image processing apparatus 10 according to the embodiment of the present invention will be described with reference to the block diagram shown in FIG. As shown in FIG. 1, the image processing apparatus 10 includes an image input interface 11, a frame memory 12, an image processor 13, a ROM (Read Only Memory) (14), and a data output interface 15.

  The image input interface 11 is a circuit board that inputs an image obtained by imaging from one or two cameras that capture a stereo image and records the image in the frame memory 12. When one camera is used, for example, the relative positional relationship between the imaging target and the camera changes, and two images obtained in time series are recorded in the frame memory 12. On the other hand, when two cameras are used, the two cameras are arranged so as to capture the same imaging object from different directions, and two images constituting a stereo image are recorded in the frame memory 12. . Hereinafter, one of the two images recorded in the frame memory 12 is referred to as a first image and the other is referred to as a second image.

  The frame memory 12 is a high-speed memory in which images are recorded. An image recorded in the frame memory 12 by the image input interface 11 is read or processed by the image processor 13.

  The image processor 13 includes a processor such as a DSP (Digital Signal Processor) or an MPU (Micro Processor Unit). The image processor 13 executes a corresponding point search process based on a stereo image by executing an image processing program stored in advance in the ROM (11).

  The image processing program includes program modules such as a frequency data processing module 131, a straight line derivation module 132, a voting processing module 133, an excluded area specification 134, a voting area restriction module 135, and a corresponding point coordinate calculation calculation module 136. The image processor 13 functions as an aggregate of a plurality of hardware that executes processes defined by the program modules 131 to 136 by developing and executing the program modules 131 to 136 in the main memory.

  The ROM (14) is a memory that stores an image processing program executed by the image processor 13 and various data referred to by the image processor 13.

  The data output interface 15 is a circuit board that outputs image processing result data delivered from the image processing processor 13 to an external device of the image processing apparatus 10.

  Next, the corresponding point search process executed by the image processing apparatus 10 according to the embodiment of the present invention will be described with reference to the flowchart shown in FIG. In the flowchart, S1 to S12 represent identification codes of processing procedures.

  Before the corresponding point search process shown in FIG. 2 is started, the image input interface 11 records each data of the first image and the second image in the frame memory 12 in advance. Further, the corresponding point search process is executed for each one or a plurality of attention points (coordinates of the attention points) in the first image designated in advance. The information on the point of interest (coordinates of the point of interest) may be stored in advance in the ROM (14) or input from an external device through a data input interface (not shown), for example.

<Step S1>
In the corresponding point search process by the image processing apparatus 10, the image processor 13 sets the voting space 100 as shown in FIG. 10, that is, an array of a plurality of voting areas 11 (S1). More specifically, the image processor 13 secures a memory area in the frame memory 12 in which data corresponding to each voting area 20 of the voting space 100 is arranged. The processing in step S1 is realized, for example, when the image processing processor 13 executes the voting processing module 133.

  As described above, the voting space 100 is an area indicating a possible range of the corresponding point deviation coordinates (dx0, dy0). The corresponding point deviation coordinates (dx0, dy0) are the coordinates of the deviation of the coordinates (x20, y20) of the corresponding points in the second image with respect to the coordinates (x10, y10) of the target point in the first image. Accordingly, one of the coordinate axes of the voting space 100 is a seat axis for the deviation dx of each coordinate value in the X-axis direction in the second image with respect to the coordinate value x0 in the X-axis direction of the target point in the first image, and the other is This is a sitting axis of deviation dy of each coordinate value in the Y-axis direction in the second image with respect to the coordinate value y0 in the Y-axis direction of the target point in the first image. In FIG. 10, the coordinate axis of the coordinate value dx is denoted by ΔX, and the coordinate axis of the coordinate value dy is denoted by ΔY.

  The image processing apparatus 10 applies information about the range (upper and lower boundary values) of the voting space 100 for specifying the voting space 100 and the number of voting areas 20 (number of sections) in each of the two coordinate axes. It is given in advance according to the object. For example, the information may be stored in advance in the ROM (14) or may be input from an external device through a data input interface (not shown).

<Step S2>
Further, the image processor 13 performs a well-known discrete Fourier transform process on each data of the first image and the second image, thereby each of the first image in the frequency domain and the second image in the frequency domain. Data is calculated (S2). That is, the image processor 13 calculates a first frequency image corresponding to the first image and a second frequency image corresponding to the second image by discrete Fourier transform processing. Hereinafter, the first frequency image is referred to as F1 (u, v), and the second frequency image is referred to as F2 (u, v). Here, (u, v) is the frequency component of the image, that is, the coordinates in the two-dimensional coordinate system of the frequency domain. The process of step S2 is realized by the image processor 13 executing the frequency data processing module 131.

<Step S3>
Further, the image processor 13 sets the selection order of the frequency component (u, v) (S3). The selection order of the frequency component (u, v) is the frequency component (u, v) applied to the processing when the processing from step S4 to step S11 is executed for each frequency component (u, v). ) Order.

  The selection order of the frequency components (u, v) is, for example, a predetermined order or a large cross spectrum between the first frequency image F1 (u, v) and the second frequency image F2 (u, v). The order may be considered. The cross spectrum represents the level of correlation between the first frequency image F1 (u, v) and the second frequency image F2 (u, v) in each frequency component (u, v). Therefore, setting the selection order of the frequency components (u, v) based on the size of the cross spectrum is between the first frequency image F1 (u, v) and the second frequency image F2 (u, v). Are sequentially selected from the frequency components (u, v) that increase the correlation of. Since the calculation method of the cross spectrum is well known, the description thereof is omitted here.

  For example, the image processor 13 calculates the cross spectrum between the first frequency image F1 (u, v) and the second frequency image F2 (u, v) by executing the frequency data processing module 131. The cross spectrum is calculated as a set of power for each frequency component (u, v). Further, the image processor 13 executes the voting processing module 132 to compare the power for each frequency component (u, v) in the cross spectrum, and in order from the higher frequency component (u, v) to the frequency component. Set the selection order of (u, v). Thereby, the voting line corresponding to the frequency component (u, v) selected according to the selection order is sequentially selected as the voting line applied to the voting process.

  The image processor 13 that executes the frequency data processing module 131 to calculate the cross spectrum is an example of a correlation degree calculation unit. Further, the image processor 13 that executes the voting processing module 132 to set the selection order of the frequency components (u, v) is an example of an order setting unit.

<Step S4>
Then, the image processor 13 sequentially selects the frequency components (u, v) according to the order set in step S3 (S4). The image processor 13 executes the following processing from step S5 to step S11 for the selected frequency component (u, v), thereby performing processing from step S5 to step S10 until a predetermined end condition is satisfied. repeat. The processing in step S4 is realized by the image processing processor 13 executing the voting processing module 133.

<Step S5>
For each selection of the frequency component (u, v), the image processor 13 calculates the phase difference Δθ (u, v) between the first frequency image and the second frequency image at the selected frequency component (u, v). Calculate (S5). The process of step S5 is realized by the image processor 13 executing the frequency data processing module 131.

  For the amplitude component Cr (u, v) of the first frequency image F1 (u, v) and the second frequency image F2 (u, v), the frequency component (u, v) is applied to the equation (1). Is calculated by The expression (1) is an expression for calculating a cross spectrum. The phase difference Δθ (u, v) between the first frequency image F1 (u, v) and the second frequency image F2 (u, v) applies the frequency component (u, v) to equation (2). Is calculated by

<Step S6>
Next, the image processor 13 selects the frequency selected in the equation (3) using the frequency component (u, v), the phase difference Δθ (u, v), and the coordinates (dx, dy) in the voting space as variables. By applying the phase difference Δθ (u, v) calculated based on the component (u, v) and its frequency component, the coefficient of the equation (primary equation) of the voting line 31 that is a straight line in the voting space 100 is calculated. (S6). In Equation (3), M and N are the sizes (number of pixels) of the two images in the X-axis direction and Y-axis direction, respectively, and are known constants.

  As shown in Non-Patent Document 1, etc., this equation (3) is calculated between the parameters of the frequency component (u, v), the phase difference Δθ (u, v), and the coordinates (dx, dy) in the voting space 100. Is a relational expression that is established in advance, and is a predetermined expression. The processing in step S6 is realized by the image processing processor 13 executing the straight line derivation module 132.

<Step S7>
Further, the image processor 13 specifies, for each of the plurality of voting areas 20 in the voting space 100, a voting area 20 that intersects with one voting line 31 corresponding to the selected frequency component (u, v). (S7). The processing in step S7 is realized by the image processing processor 13 executing the voting processing module 133.

  As described above, the voting area 20 that intersects the voting straight line 31 is referred to as an intersecting area 20a. The processing in step S7 is realized by the image processing processor 13 executing the voting processing module 133. One voting line 31 and the intersecting region 20a intersecting therewith are illustrated in FIG.

<Step S8>
Further, every time an intersection area 20a intersecting with one voting line 32 is specified, the image processor 13 uses a predetermined vote value for data associated with the intersection area 20a (voting area 20). Are added (S8). That is, the image processor 13 adds a predetermined voting value to the data stored in each data area in the frame memory 12 corresponding to each of the voting areas 20 defined as the intersection areas 20a. The initial value of each data area in the frame memory 12 corresponding to the voting space 100 is “0”.

  It is conceivable that the voting value is the same value for each of all the voting areas 20 specified as the intersection area 20a. In addition, it is conceivable to employ different voting values weighted according to the power (correlation degree) of the cross spectrum. In this case, a relatively large vote value is added in the intersection region 20a that intersects the voting line 31 corresponding to the frequency component (u, v) having a large cross spectrum power. Also, a relatively small vote value is added to the intersection region 20a that intersects the voting line 31 corresponding to the frequency component (u, v) having a small cross spectrum power.

<Steps S9 and S10>
The image processor 13 determines whether or not a predetermined process transition condition is satisfied each time the processes from step S4 to step S8 are executed (S9). If the image processor 13 determines that the process transition condition is satisfied, the image processor 13 executes a voting area restriction process (S10) described later. On the other hand, if the image processor 13 determines that the process transition condition is not satisfied, the image processor 13 skips the voting area restriction process (S10).

  For example, the first process transition condition is a condition that the status of the voting process is a point in time when the voting process (S6 to S8) for the predetermined first number J1 of voting straight lines 31 is completed. In addition, the second process transition condition is that the voting process is performed for the voting straight line 31 of the second number J2 determined in advance after the establishment of the first process transition condition (S6 to S8). It is a condition that it is a periodic timing every time is completed. The voting area restriction process (S10) is executed every time one of the first process transition condition and the second process transition condition is satisfied. Thereby, the voting area restriction process (S10) is executed at a plurality of intermediate stages where the voting process for a part of the plurality of voting straight lines 31 is completed.

<Step S11>
The image processor 13 determines whether or not a predetermined end condition is satisfied each time the processing from step S4 to step S9 is executed (S11). Then, the image processor 13 selects the frequency component (u, v) (S4) and the phase difference Δθ (u, v) for each selected frequency component (u, v) until the termination condition is satisfied. Calculation (S5), specification of the intersecting area 20a intersecting the voting line 31 corresponding to the phase difference Δθ (u, v) (S6, S7), and addition of vote values for the intersecting area 22 (S8) Run sequentially.

  That is, the image processor 13 executes the voting processing module 133 to identify the intersecting region 20a that intersects with one voting line for each of the plurality of voting regions 20 (S7). Further, the image processor 13 executes the voting processing module 133 to perform a process of adding voting values in association with the intersecting region 20a (S8), and voting corresponding to each of the plurality of frequency components (u, v). Each straight line 31 is sequentially executed (S4 to S8). Thereby, vote values are integrated with respect to the intersection area 20a. The plurality of voting straight lines 31 and the intersecting region 20a intersecting with them are illustrated in FIG.

  The end condition is, for example, that an integrated value of voting values in each voting area 20 satisfies a predetermined condition. The condition is, for example, that the integrated values P1 and P2 of the vote values satisfy the following expression (4). In Equation (4), H is the number of all frequency components (u, v), h is the number of frequency components (u, v) for which voting processing (voting value accumulation) has been completed, and Umax Is the maximum voting value per voting process, and P1 and P2 are the maximum integrated values of voting values at the time when the voting process is performed for h frequency components (u, v) And the second largest value. Note that, when the voting value is not weighted according to the position of the voting area 20, Umax is a voting value added for each voting area 20 in one voting, and is normally “1”.

  Satisfying the expression (4) means that the voting area 20 whose voting value integrated value is P1 at the present time is finally determined to have the maximum voting value integrated value. Therefore, when the corresponding point deviation coordinates (dx0, dy0) are derived based on the position of the voting area 20 where the integrated value of the voting value finally becomes the maximum, it is effective to use the expression (4) as an end condition. .

  In addition, other conditions may be considered as the end condition based on the integrated value of the vote value in each vote area 20. For example, the maximum integrated value P1 of the voting value is equal to or greater than a predetermined threshold, the maximum integrated value P1 of the voting value, and the second largest value P2 of the integrated voting value It is conceivable that the difference (P1-P2) is greater than or equal to a predetermined threshold value, or that both conditions are satisfied. It is also conceivable that the end condition is that the processes in steps S4 to S8 have been performed for a predetermined number of frequency components (u, v).

  Here, the details of the voting area restriction process (S10) will be described with reference to the flowchart shown in FIG. In the flowchart, S21 to S25 represent identification codes of processing procedures.

<Step S21>
In the voting area limiting process, first, the image processor 13 selects a partial area 20b for determination that is a set of voting areas 20 that occupy a part and a series of ranges in the voting space 100 from all the voting areas 20. Select sequentially (S21: determination partial region selection process). Here, the set of voting areas 20 means that one or more voting areas 20 are included. The determination partial area 20b is a unit that is a target of a level determination process (S22) described later. The processing in step S21 is realized by the image processor 13 executing the excluded area specifying module 134.

  4 and 5 are schematic diagrams illustrating a first example and a second example of processing for selecting the determination partial region 20b from the voting space 100 in the voting region restriction processing.

  The first example shown in FIG. 4 is an example in which individual voting areas 20 are sequentially selected as determination voting areas 21 in the determination partial area selection processing (S21), that is, the voting area 20 and the determination partial area 20b. Is an example of one-to-one correspondence. In the second example shown in FIG. 5, in the determination partial area selection process (S21), nine voting areas 20 occupying a range of three columns in the ΔX-axis direction and a range of three rows in the ΔY-axis direction. In this example, the determination voting area 21 is sequentially selected. Here, the rows and columns in the voting space 100 mean the rows and columns of the voting area 20.

  In the example of FIG. 5, for example, the determination voting area 21 occupying a range of 3 columns × 3 rows is sequentially selected from positions shifted by one voting area 20 minutes in the ΔX-axis direction or ΔY-axis direction. In the second example shown in FIG. 5, the determination voting area 21 is an area of 3 columns × 3 rows, but an area of 2 columns × 2 rows, an area of 2 columns × 3 rows, or 5 An area selected as the voting area for determination 21 such as an area of columns × 5 rows is arbitrarily determined according to the application target of the image processing apparatus 10.

<Step S22>
Next, for each selected determination partial area 20b, the image processor 13 sets a threshold value in which a representative value of the integrated values of voting values corresponding to the voting areas 20 in the determination partial area 20b is set in advance. It is determined how much level it is based on (S22: level determination process). The determination result is recorded in the main memory of the image processor 13 in association with the identification number of each voting area 20 included in the determination partial area 20b. This level determination process is realized by the image processor 13 executing the excluded area specifying module 134.

  As shown in FIG. 5, when the determination partial area 20b includes a plurality of voting areas 20, the representative value is, for example, the maximum value of a plurality of integrated values corresponding to the voting areas 20 in the determination partial area 20b, or For example, an average value. As shown in FIG. 4, when the determination partial area 20b is each voting area 20, the representative value is the integrated value of the voting value itself.

  In the level determination process of step S22, for example, the image processor 13 compares the representative value with one or a plurality of threshold values, so that the representative value is relatively high “high level” and relatively small “ It is conceivable to determine which level (rank) corresponds to “low level”. For example, when the representative value is SV and one threshold is SL1, the image processor 13 determines that the representative value SV corresponds to “high level” when SV ≧ SL1, and SV < When it is SL1, it is determined that the representative value SV corresponds to “low level”. Further, when the two threshold values are SL1, SL2 (SL1> SL2), the image processor 13 determines that the representative value SV corresponds to “high level” when SV ≧ SL1, When SV <SL2, it is determined that the representative value SV corresponds to “low level”.

  In addition, the image processor 13 compares the representative value with each of the plurality of threshold values, and the representative value has a relatively large “high level” and a relatively small “low level”. It is also conceivable to determine which of the three levels (ranks), “medium level”, corresponds to. For example, when the two threshold values are SL1, SL2 (SL1> SL2), the image processor 13 determines that the representative value SV corresponds to “high level” when SV ≧ SL1, When SV <SL2, it is determined that the representative value SV corresponds to “low level”, and at other times, the representative value SV is determined to correspond to “medium level”. It is also conceivable to determine “high level”, “low level”, and “medium level” using three or more threshold values.

<Step S23>
The image processor 13 determines whether or not the processing of step S22 has been completed for all the determination partial areas 20b (S23), and selects the determination partial areas 20b for all the determination partial areas 20b (S21). The level determination process (S22) for the determination partial area 20b is sequentially executed. The determination process in step S23 is realized by the image processor 13 executing the excluded area specifying module 134.

  By the way, in the voting area 20 belonging to the plurality of determination partial areas 20b, it is considered that a different level determination is performed for each corresponding determination partial area 20b. For such a voting area 20, it may be determined in advance whether to give priority to the determination result on the “high level” side or to give priority to the determination result on the “low level” side.

  In the following description, the determination partial area 20b (a set of voting areas 20) from which the determination results of “high level”, “low level”, and “medium level” are obtained are referred to as a high level area 21 and a low level, respectively. These are referred to as level region 22 and middle level region 23.

  FIG. 6 is a schematic diagram showing an example of the voting space 100 divided into two regions, a high level region 21 and a low level region 22, by the first level determination process based on the integrated value of voting values. In FIG. 6, the high level region 21 is indicated by a blank, and the low level region 22 is shaded with a sand pattern.

  In the example shown in FIG. 6, the determination partial area 20 b is set to one-to-one for each voting area 20, and the determination partial area 20 b (voting area) that intersects one or more voting straight lines 31 is high. This is an example in which the determination partial region 20 b that does not intersect the level region 21 and the voting line 31 is determined to be the low level region 22.

  That is, in the example shown in FIG. 6, when the vote value added at one time in all the vote areas 20 is “1”, the determination partial area 20 b (the accumulated value of the vote values is “1” or more). This is an example in which the determination partial area 20b (voting area) in which the voting area) is the high level area 21 and the integrated voting value is less than “1” is determined as the low level area 22.

  FIG. 7 is a schematic diagram showing an example of the voting space 100 divided into two regions, a high level region 21 and a low level region 22, by a second level determination process based on the integrated value of voting values. Also in FIG. 7, the high level region 21 is shown as blank, and the low level region 22 is shaded with a sand pattern.

  In the example shown in FIG. 7, the determination partial area 20 b is set to one-to-one for each voting area 20, and the determination partial area 20 b (voting area) intersecting with two or more voting straight lines 31 is high. This is an example in which the determination partial region 20 b that does not intersect the level region 21 and the voting line 31 is determined to be the low level region 22.

  That is, in the example shown in FIG. 7, when the vote value added at one time in all the vote areas 20 is “1”, the determination partial area 20 b (the accumulated value of the vote values is “2” or more) This is an example in which the determination partial area 20 b (voting area) in which the voting area) is the high level area 21 and the integrated value of the voting value is less than “2” is determined as the low level area 22. For convenience of explanation, the threshold value SL1 in the examples of FIGS. 6 and 7 is set to “1” or “2”, but it is assumed that a larger value is usually set as the actual threshold value SL1. The

  FIG. 8 shows a voting space 100 divided into three areas of a high level area 21, a low level area 22, and an intermediate level area 23, which is a level between them, by a third level determination process based on an integrated value of voting values. It is a schematic diagram which shows an example. In FIG. 8, the high level region 21 is shown as blank, the low level region 22 is sanded, and the middle level region 23 is shaded.

  In the example shown in FIG. 8, the determination partial area 20 b is set to one-to-one for each voting area 20, and the determination partial area 20 b (voting area) intersecting with two or more voting straight lines 31 is high. An example in which the determination partial region 20 b that is determined as the level region 21, does not intersect the voting straight line 31, is determined as the low level region 22, and the determination partial region 20 b that intersects one voting straight line 31 is determined as the middle level region 23 It is.

  That is, in the example shown in FIG. 8, when the vote value added at one time in all the vote areas 20 is “1”, the determination partial area 20 b (the accumulated value of the vote values is “2” or more) Voting area) is determined as the high-level area 21, the determination partial area 20 b whose voting value integrated value is less than “1” is determined as the low-level area 22, and the other determination partial areas 20 b are the intermediate-level area 23. It is an example determined as. For convenience of explanation, the threshold value SL1 in the example of FIG. 8 is “2”, but it is assumed that a larger value is usually set as the actual threshold value SL1.

<Step S24>
When the level determination processing (S22) for all the determination partial areas 20b is completed, the image processing processor 13 is excluded from the subsequent voting processing (S7 and S8) based on the level determination result. Exclusion area specifying processing for specifying the exclusion area 24 that is the voting area 20 is executed (S24). The determination process in step S23 is realized by the image processor 13 executing the excluded area specifying module 134. Hereinafter, a specific example of the excluded area specifying process will be described.

<First Example of Exclusion Area Identification Process>
The first example of the excluded area specifying process (S24) is an example in which at least the high level area 21 and the low level area 22 are determined based on one or more threshold values. In this example, the image processor 13 requires the high-level area 21 to exist, and the remaining voting areas excluding the voting areas 20 within the range including the high-level area 21 among all the voting areas 20. 20 is specified as the exclusion region 24.

  In the first example, the following two methods are conceivable as a method of specifying the exclusion region 24. The first method is a method of specifying all the remaining voting areas 20 except the voting area 20 in the high level area 21 as the exclusion area 24. When the level determination result shown in FIG. 6 or FIG. 7 is obtained, in the first method in the first example, all of the low level regions 22 that are shaded with sand pattern are used as the exclusion regions 24. Identified.

  When the level determination result shown in FIG. 8 is obtained, in the first method in the first example, the low-level region 22 that is shaded with a sand pattern and the shaded hatching are attached. All of the intermediate level areas 23 are specified as the exclusion areas 24.

  On the other hand, the second method in the first example is a method in which the remaining voting area 20 excluding the voting area 20 included in the range obtained by enlarging the high-level area 21 by a predetermined size is specified as the excluded area 24. It is. In the second method in the first example, among the voting areas 20 other than the high level area 21, only the remaining voting areas 20 excluding a part of the voting areas 20 close to the high level area 21 are specified as the excluded areas 24. Is done.

  FIG. 9 shows the voting space 100 in which the exclusion area 24 is set by the second method in the first example when the level determination result (the result of the second level determination process) shown in FIG. 7 is obtained. It is a schematic diagram which shows an example. More specifically, in the example shown in FIG. 9, when the level determination result shown in FIG. 7 is obtained, the voting included in the range in which the high level region 21 is expanded by one column and one row is included. In this example, the remaining voting area 20 excluding the area 20 is specified as the exclusion area 24. Note that the width of enlargement is not limited to the width of one column and one row, but may be the width of a plurality of columns and the width of a plurality of rows.

<Second Example of Exclusion Area Identification Process>
The second example of the excluded area specifying process (S24) is an example in which at least the high level area 21, the low level area 22, and the middle level area 23 are determined based on two or more threshold values. is there. In this example, the image processor 13 is within the range including the high level area 21 among all the voting areas 20 on condition that the high level area 21 exists and the intermediate level area 23 does not exist. The remaining voting area 20 excluding the voting area 20 is specified as the exclusion area 24.

  Also in the second example, the method for specifying the exclusion region 24 includes the first method and the second method similar to those in the first example. That is, the first method is a method of specifying all the remaining voting areas 20 except the voting area 20 in the high level area 21 as the exclusion area 24.

  On the other hand, the second method is a method in which the remaining voting area 20 excluding the voting area 20 included in the range in which the high level area 21 is enlarged by a predetermined size is specified as the exclusion area 24. In this case, the remaining area of the low level area 22 excluding a part of the voting area 20 close to the high level area 21 is specified as the exclusion area 24.

  In the second example, as a result of the voting processing (S <b> 8) being integrated in a very small part of the high-level area 21, only a small part of the high-level area 21 and the other voting areas 20 are obtained. An example in which a region excluding the high-level region 21 or a region excluding a range obtained by enlarging the high-level region 21 is specified as the excluded region 24 when the difference between the vote values is large. It is.

<Third example of exclusion area specifying process>
The third example of the excluded area specifying process (S24) is an example in which the high level area 21 and the low level area 22 are determined based on one threshold SL1 as in the first example. It is. In this example, the image processor 13 specifies the voting area 20 included in the low level area 22 among all the voting areas 20 as the exclusion area 24 on the condition that the high level area 21 exists. Accordingly, all of the voting areas 20 included in the low level area 22 are specified as the excluded areas 24.

  When the level determination result shown in FIG. 6 or FIG. 7 is obtained, in the third example, the low level region 22 that is shaded with a sand pattern is specified as the excluded region 24. Further, when the level determination result shown in FIG. 8 is obtained, in this third example, the low level region 22 that is shaded with a sand pattern is specified as the excluded region 24.

<Fourth Example of Exclusion Area Identification Process>
The fourth example of the excluded area specifying process (S24) is an example in which at least the high level area 21, the low level area 22, and the middle level area 23 are determined based on two or more threshold values. is there. In this example, the image processor 13 specifies the low-level area 22 as an excluded area 24 out of all the voting areas 20 on the condition that the high-level area 21 exists and the middle-level area 23 does not exist. To do.

  In the fourth example, as a result of the voting value accumulation (S8) by the voting process being concentrated in a very small part of the high level area 21, a very small part of the high level area 21 and the other voting areas 20 are obtained. This is an example in which the low-level region 22 is specified as the exclusion region 24 when the difference between the integrated voting values is large.

<Step S25>
When the specification of the exclusion area 24 (S25) ends, the image processor 13 thereafter executes a process of excluding the exclusion area 24 from the target of the voting process (S7 and S8) (S25). That is, the process of step S25 is a process of controlling the voting process module 133 so that the voting process (S7 and S8) is executed only for the remaining voting areas 20 excluding the excluded areas 24 from all the voting areas 20. It is. The process in step S25 is realized by the image processor 13 executing the voting area restriction module 135. Note that the image processor 13 does not restore the voting area 20 once excluded from the voting target as the voting target again.

  For example, in steps S7 and S8, the image processor 13 refers to a preset list of identification numbers of the voting area 20, and performs voting processing for the voting areas 20 corresponding to the identification numbers included in the list. It is conceivable to execute. In this case, the image processor 13 may delete the identification number corresponding to the excluded area 24 from the list in step S25.

  In steps S7 and S8, the image processor 13 refers to flag information set in advance corresponding to each voting area 20, and performs voting processing only for the voting area 20 whose flag information is "ON". Can be considered. In this case, the image processor 13 may set the flag information corresponding to the excluded area 24 to “OFF” in step S25.

  By executing the voting area restriction process (S21 to S25) shown above, the voting area 20 specified as the exclusion area 24 accumulates the voting values in the subsequent voting processes (S7 and S8). Excluded from the target. Hereinafter, FIG. 2 will be referred to again.

<Step S12>
If the end condition is satisfied when the processes in steps S4 to S10 described above are performed, the image processor 13 determines that the vote value is accumulated based on the accumulated value of the vote values in each of the vote areas 20. Corresponding point deviation coordinates (dx0, dy0) corresponding to the coordinates of the region 20 are calculated (S12). More specifically, the image processor 13 calculates the coordinates of the peak position of the integrated value of the vote value in the vote space 100 as the corresponding point deviation coordinates (dx0, dy0). The process of step S12 is realized by the image processor 13 executing the corresponding point coordinate calculation calculation module 136. The corresponding point deviation coordinates (dx0, dy0) are an example of target coordinates.

  For example, the image processor 13 calculates the coordinates of the center of the voting area 20 where the integrated value of the voting values when the end condition is satisfied is the corresponding point deviation coordinates (dx0, dy0). Alternatively, the image processor 13 uses the integrated value of the voting values when the end condition is satisfied as a weighting coefficient, and the coordinates of the centroid position of the voting area 20 where the voting values are integrated as the corresponding point deviation coordinates (dx0, dy0). calculate.

  In the image processing apparatus 10, the exclusion area 24 that is a part of the voting space 100 is excluded from the voting process target after the voting process. Therefore, the integrated value of the voting value in the exclusion area 24 cannot be compared with the integrated value of the voting value in the other voting areas 20. Therefore, it is desirable to exclude the excluded area 24 from the target used for calculating the corresponding point deviation coordinates (dx0, dy0).

  In the process of step S12, the image processor 13 outputs the corresponding point deviation coordinate (dx0, dy0) information calculated as described above to the external device through the data output interface 15. Alternatively, the image processor 13 determines the coordinates (x02, y02) of the corresponding point in the second image based on the coordinates (x01, y01) of the target point in the first image and the corresponding point deviation coordinates (dx0, dy0). The information of the coordinates (x02, y02) is calculated and output to the external device through the data output interface 15. The process described above is the corresponding point search process in the image processing apparatus 10.

  According to the corresponding point search process of the image processing apparatus 10, as in the conventional voting process, the intersection area 20 a intersecting the voting straight line 31 determined according to the frequency component (u, v) among the plurality of voting areas 20. The voting values are integrated against each other, and the target coordinates are derived according to the distribution of the integrated voting values. Here, the target coordinates are the corresponding point deviation coordinates (dx, dy).

  Then, according to the corresponding point search process of the image processing apparatus 10, the voting area 20 that is the target of the voting process is limited to a part of the area after the voting process (S10). For example, in the middle of the voting process, the remaining voting area 20 or the low-level area 22 from which the high-level area 21 and its neighboring areas are removed is excluded from the voting process.

  Therefore, even when the voting space 100 is discretized with a high spatial resolution and a large number of voting areas 20 are set, the voting process for the unimportant voting areas 20 is omitted after the intermediate stage, and the calculation load of the image processor 13 is reduced. It can be kept low. Therefore, according to the image processing apparatus 10, when the target coordinates in the voting space 100 are derived by the voting process, the target coordinates can be derived with high accuracy and a relatively light calculation load.

  In addition, when the voting space 100 is divided into a large number of small voting areas 20, a situation occurs in which the intersection where the voting straight lines 31 intersect is particularly concentrated on one voting area 20 in the middle of the voting process. Hateful. For this reason, when the status of the integrated value of the vote value is determined for each individual vote area 20, it is not possible to correctly determine the distribution of the intersection and specify the appropriate excluded area 24.

  In the image processing apparatus 10, an exclusion based on the level determination of the representative value of the integrated value of the vote value is made for each determination partial area 20 b including a set of the vote areas 20 that occupy a part and a series of ranges in the voting space 100. The area 24 is specified (S21 to S24). Thereby, regardless of the size of the voting area 20, it is possible to specify an appropriate exclusion area 24 that reflects the distribution of intersections in the voting space 100.

  In the image processing apparatus 10, the order of the voting straight lines 31 applied to the voting process, that is, the selection order of the frequency components (u, v) is two frequency images F1 (u, v) and F2 (u, v). Are set in the order corresponding to the power of the cross spectrum representing the degree of correlation between the two (S3). As a result, the distinction between the region where the intersections are concentrated and the region where the intersections hardly exist is clarified at a relatively early stage, and the exclusion region 24 can be appropriately identified at a relatively early stage. Become.

  The embodiment described above is an example in which the present invention is applied to the corresponding point search processing based on the PSA method, but the present invention is an intersection region where a plurality of faces or lines intersect for each voting region in the voting space. The present invention is applicable to all voting processes that specify 20a and add (accumulate) voting values in association with the intersection area 20a.

  For example, when extracting a circle from an image by Hough transform, a voting space corresponding to each pixel in the edge image is set. This voting space is a three-dimensional finite space in which the values of three parameters of the coordinates (a, b) of the center position for specifying the circle and the radius r of the circle are coordinate values.

  Furthermore, the image processing apparatus that performs the Hough transform corresponds to a circle centered on one pixel in the edge image among each of a plurality of voting areas that are partial areas in which the voting space (parameter space) is discretized. The intersection area that intersects the conical surface in the voting space is specified.

  Further, the image processing apparatus that performs the Hough transform performs a voting process that adds a voting value in association with a voting area that intersects the conical surface. The voting process is sequentially executed for each of the plurality of conical surfaces corresponding to each pixel of the edge image, whereby the voting values are integrated in the voting area intersecting with each of the plurality of conical surfaces. Then, the coordinates of the peak position of the integrated value of voting values in the voting space are derived.

  When the present invention is applied to the Hough transform, the image processing apparatus that performs the Hough transform may select a part of the voting area in the voting space in the middle stage of the voting process in the same manner as the processes shown in Steps S21 to S25. It is specified as an excluded area, and the specified excluded area is excluded from the objects of subsequent voting processes. By doing so, the same operation and effect as the above-described embodiment can be obtained also in the Hough transform.

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 11 Image input interface 12 Frame memory 13 Image processor 14 ROM
DESCRIPTION OF SYMBOLS 15 Data output interface 20 Voting area 20a Intersection area 20b Judgment partial area 21 High level area 22 Low level area 23 Middle level area 31 Voting straight line 131 Frequency data processing module 132 Straight line deriving module 133 Voting processing module 134 Exclusion area specification module 135 Voting Area restriction module 136 Corresponding point coordinate calculation module S1 to S25 Processing procedure identification code

Claims (15)

Based on the first phase image and the second phase image, which are phase images in the spatial frequency domain corresponding to the first image and the second image, respectively, the second corresponding to the attention point specified in the first image An image processing apparatus that executes a corresponding point search process for deriving the coordinates of corresponding points in an image,
A phase difference calculator that calculates a phase difference according to a frequency component between the first phase image and the second phase image;
By applying the frequency component and the phase difference corresponding to the frequency component to a predetermined formula, the coordinates of the deviation of the coordinates of the corresponding point in the second image with respect to the coordinates of the target point in the first image A straight line deriving unit for deriving a straight line in a finite voting space indicating a possible range of the corresponding point deviation coordinates,
For each of a plurality of voting areas, which are partial areas in which the voting space is discretized, the voting area that intersects one straight line is specified, and a voting value is added in association with the specified voting area A voting processing unit that performs voting processing for accumulating the voting values by sequentially executing processing for each of the straight lines corresponding to each of a plurality of frequency components;
In the middle of completing the voting process for a part of the plurality of straight lines, an excluded area that is the voting area to be excluded from the voting process target is specified based on the integrated value of the voting values. An exclusion area identification part;
After the exclusion area is specified, a voting restriction unit that controls the voting process to be performed only on the remaining voting area except the exclusion area from all the voting areas; and
An image processing apparatus comprising: a target coordinate deriving unit that derives the corresponding point deviation coordinates based on the integrated value of the vote values and the coordinates of the vote area where the vote values are integrated. A determination partial area selection unit for sequentially selecting a determination partial area consisting of a set of the voting areas that occupy a part and a series of ranges in the voting space from among all the voting areas;
For each selected partial area for determination, a representative value of the integrated values of the voting values corresponding to the voting areas in the partial area for determination is relatively large with reference to a preset threshold value. A level determination unit that determines whether the level is a high level or a relatively low level, and
The image processing apparatus according to claim 1, wherein the exclusion region specifying unit specifies the exclusion region based on a determination result of the level determination unit. The level determination unit performs the determination of the high level and the low level based on one threshold value,
The exclusion area specifying unit, when there is the determination partial area where the high-level determination result is obtained, out of all the voting areas, the determination part where the high-level determination result is obtained The image processing apparatus according to claim 2, wherein the remaining voting area excluding the voting area within a range including the area is specified as the exclusion area. The level determination unit determines whether the representative value is the high level, the low level, or an intermediate level between them based on the two threshold values,
The exclusion area specifying unit is all in the case where the determination partial area where the high-level determination result is obtained and the determination partial area where the medium-level determination result is obtained does not exist. The remaining voting areas except for the voting area within the range including the determination partial area from which the high-level determination result is obtained are specified as the exclusion area. The image processing apparatus described. The level determination unit performs the determination of the high level and the low level based on one threshold value,
The exclusion area specifying unit specifies, as the exclusion area, the determination partial area from which the low-level determination result is obtained when the determination partial area from which the high-level determination result is obtained exists. The image processing apparatus according to claim 2. The level determination unit determines whether the representative value is the high level, the low level, or an intermediate level between them based on the two threshold values,
The exclusion area specifying unit specifies, as the exclusion area, the determination partial area from which the low-level determination result is obtained when the determination partial area from which the high-level determination result is obtained exists. The image processing apparatus according to claim 2.   The image processing apparatus according to claim 2, further comprising a threshold setting unit configured to set the threshold according to the number of the straight lines for which the voting process has been completed in the intermediate stage. .   The image processing apparatus according to claim 2, wherein the threshold value is a predetermined fixed value.   The image processing device according to claim 2, wherein the determination partial area is each of the voting areas, and the representative value is an integrated value of the voting values in each of the voting areas. A degree-of-correlation calculating unit that calculates an index of a degree of correlation for each frequency component between a first frequency image corresponding to the first image and a second frequency image corresponding to the second image;
The order setting part which sets the order of the straight line applied to the voting process from the thing corresponding to the frequency component with the high degree of correlation based on the index of the degree of correlation. An image processing apparatus according to any one of the above.   The image processing apparatus according to claim 10, wherein the index of correlation is a power for each frequency component in a cross spectrum between the first frequency image and the second frequency image.   The image processing apparatus according to claim 1, wherein the exclusion area specifying unit executes a process of specifying the exclusion area in a plurality of predetermined intermediate stages. Based on the first phase image and the second phase image, which are phase images in the spatial frequency domain corresponding to the first image and the second image, respectively, the second corresponding to the attention point specified in the first image An image processing program for causing a computer to execute corresponding point search processing for deriving coordinates of corresponding points in an image,
A phase difference calculation process for calculating a phase difference according to a frequency component between the first phase image and the second phase image;
By applying the frequency component and the phase difference corresponding to the frequency component to a predetermined formula, the coordinates of the deviation of the coordinates of the corresponding point in the second image with respect to the coordinates of the target point in the first image A straight line derivation process for deriving a straight line in a finite voting space indicating the possible range of the corresponding point deviation coordinates,
For each of a plurality of voting areas, which are partial areas in which the voting space is discretized, the voting area that intersects one straight line is specified, and a voting value is added in association with the specified voting area A voting process for accumulating the voting values by sequentially executing a process for each of the straight lines corresponding to a plurality of the frequency components;
In the middle of completing the voting process for a part of the plurality of straight lines, an excluded area that is the voting area to be excluded from the voting process target is specified based on the integrated value of the voting values. Exclusion area identification processing,
After the exclusion area is specified, a voting restriction process for controlling the voting process to be executed only on the remaining voting area excluding the exclusion area from all the voting areas; and
An image processing program for causing a computer to execute a target coordinate derivation process for deriving the corresponding point deviation coordinates based on the integrated value of the voting values and the coordinates of the voting area where the voting values are integrated. When information specifying a plurality of faces or lines in a finite voting space having a plurality of parameter values as coordinate values is given, there is one for each of a plurality of voting regions which are discrete regions of the voting space. The voting area is specified by sequentially specifying the voting area that intersects the face or the line and adding a voting value in association with the specified voting area for each of the face or the line. A voting processing unit that performs voting processing for accumulating values;
An exclusion area that is the voting area that is excluded from the voting process based on the integrated value of the voting values in the middle of the voting process for a part of the plurality of faces or lines. An excluded area specifying part for specifying
After the exclusion area is specified, a voting restriction unit that controls the voting process to be performed only on the remaining voting area except the exclusion area from all the voting areas; and
An information processing apparatus comprising: a target coordinate deriving unit that derives a target coordinate in the voting space based on the integrated value of the voting value and the coordinates of the voting area in which the voting value is integrated. . When information specifying a plurality of faces or lines in a finite voting space having a plurality of parameter values as coordinate values is given, there is one for each of a plurality of voting regions which are discrete regions of the voting space. The voting area is specified by sequentially specifying the voting area that intersects the face or the line and adding a voting value in association with the specified voting area for each of the face or the line. A voting process for accumulating values,
An exclusion area that is the voting area that is excluded from the voting process based on the integrated value of the voting values in the middle of the voting process for a part of the plurality of faces or lines. An excluded area identification process for identifying
After the exclusion area is specified, a voting restriction process for controlling the voting process to be executed only on the remaining voting area excluding the exclusion area from all the voting areas; and
An information processing program for causing a computer to execute a target coordinate derivation process for deriving a target coordinate in the voting space based on the integrated value of the voting value and the coordinates of the voting area in which the voting value is integrated .

Images