JP6812731B2 - Image processing device, image processing method, image processing program - Google Patents

Description

The present invention relates to an image processing apparatus, an image processing method, and an image processing program for extracting straight lines included in an image.

For example, a concrete wall surface such as a wall surface inside a tunnel may crack due to deterioration over time. As described in Non-Patent Document 1, it has been proposed to photograph a concrete wall surface with a camera to detect the presence or absence of cracks. Cracks can be detected by extracting the edges of the captured image.

However, when the edges of the image of the wall surface inside the tunnel are extracted, almost linear edges may be extracted in addition to the edges showing non-linear cracks. The linear edge is generated by factors such as the construction method of the wall surface and the structure of the wall surface. If a straight line can be extracted from an image in which non-linear cracks and straight lines are mixed, the straight line can be removed.

As described in Patent Documents 1 and 2 and Non-Patent Document 1, the Hough transform is often used to extract a straight line included in an image.

Japanese Unexamined Patent Publication No. 2006-252400 Japanese Unexamined Patent Publication No. 2014-146326

Obata, Yamazaki, Tsubouchi, "Automation of crack inspection of indoor walls by mobile camera", Proceedings of the 13th Image Sensing Symposium, IN3-02, (CD-ROM), 2007. (http://www.jsk. tu-tokyo.ac.jp/~yamazaki/rp/SSII2007_yamazaki.pdf)

Straight lines are often horizontal or vertical, but may be slightly tilted from horizontal or vertical. Straight lines may occur intermittently. Images taken of the walls inside a tunnel may have multiple straight lines, not necessarily horizontal or vertical, throughout the frame with low contrast.

A huge amount of calculation is required to extract a plurality of straight lines existing in the entire frame by using the Hough transform. An image processing device, an image processing method, and an image processing program capable of extracting straight lines with a small amount of calculation while using the Hough transform are required.

An object of the present invention is to provide an image processing apparatus, an image processing method, and an image processing program capable of extracting a straight line included in an image with a small amount of calculation while using a Hough transform.

The present invention includes an edge extraction unit that extracts at least one edge in the horizontal direction and the vertical direction included in the input image signal, and the edge extracted by the edge extraction in a completely horizontal or vertical direction. Considering that it is a straight line, each pixel constituting the edge is Huff-transformed to obtain a first angle indicating a perfect horizontal or vertical direction and a distance from the origin of the frame of the image signal to the straight line. The Huff space storage unit that stores the data represented by the voting value obtained by voting the first number of pixels of the pixels constituting the edge in the Huff space and the actual inclination angle of the edge are calculated. The inclined straight line fitting unit for fitting the edge to the inclined straight line and calculating the number of second pixels of the edge included in the inclined straight line and the distance from the origin to the inclined straight line, and the actual The number of the second pixel is added to the voting value of the second set of the inclination angle of the above and the distance from the origin to the inclined straight line, and the number of the second pixel is calculated from the voting value of the first set. It is characterized by including a huff space correction unit that corrects the huff space by subtraction, and a straight line extraction unit that extracts data of voting values exceeding a predetermined threshold in the huff space corrected by the huff space correction unit as a straight line. To provide an image processing apparatus.

The present invention extracts at least one edge of the horizontal direction and the vertical direction included in the image signal, considers that the extracted edge is a perfect horizontal line or a vertical straight line, and determines the edge. Each of the constituent pixels is Huff-transformed to form the edge with respect to a first set of an angle indicating a perfect horizontal or vertical direction and a distance from the origin of the frame of the image signal to the straight line. Data represented by a voting value obtained by voting for the first number of pixels to be used is stored in the huff space, the actual inclination angle of the edge is calculated, the edge is fitted to an inclined straight line, and the edge is inclined. A second set of the number of second pixels of the edge included in the straight line and the distance from the origin to the inclined straight line is calculated, and the actual inclination angle and the distance from the origin to the inclined straight line are calculated. The number of the second pixels is added to the voting value of the first set, the number of the second pixels is subtracted from the voting values of the first set, the huff space is modified, and a predetermined threshold value in the modified huff space is set. Provided is an image processing method characterized by extracting data of a voting value exceeding a straight line as a straight line.

The present invention considers to the computer the step of extracting at least one of the horizontal and vertical edges contained in the image signal, and that the extracted edge is a perfect horizontal or vertical straight line. Then, each pixel constituting the edge is Hough transformed with respect to a first set of an angle indicating a perfect horizontal direction or a vertical direction and a distance from the origin of the frame of the image signal to the straight line. , The step of storing the data represented by the voting value obtained by voting the first number of pixels of the pixels constituting the edge in the Hough space, and calculating the actual inclination angle of the edge to form a straight line in which the edge is inclined. To calculate the number of second pixels of the edge included in the inclined straight line, the distance from the origin to the inclined straight line, the actual inclination angle, and the inclined straight line from the origin. The step of adding the number of the second pixels to the voting value of the second set with the distance to and subtracting the number of the second pixels from the voting value of the first set to correct the Hough space. Provided is an image processing program characterized by executing a step of extracting data of a voting value exceeding a predetermined threshold as a straight line in a modified Hough space.

According to the image processing apparatus, image processing method, and image processing program of the present invention, straight lines included in an image can be extracted with a small amount of calculation while using the Hough transform.

It is a block diagram which shows the image processing apparatus of one Embodiment. It is a figure which shows an example of the image which photographed the wall surface of the concrete including a non-linear crack line and a plurality of straight lines. It is a figure for demonstrating the Hough transform. It is a figure for demonstrating the inclined straight line fitting operation of a horizontal straight line. It is a figure for demonstrating the inclined straight line fitting operation of a vertical straight line. It is a flowchart which shows the processing by the image processing method of one Embodiment, and the procedure executed by the image processing program of one Embodiment. It is a block diagram which shows the 1st modification. It is a block diagram which shows the 2nd modification.

Hereinafter, the image processing apparatus, the image processing method, and the image processing program of one embodiment will be described with reference to the accompanying drawings. In the present embodiment, an image of a concrete wall surface such as a wall surface inside a tunnel is taken as an example of an image including a straight edge. Images containing straight edges are not limited to images taken of concrete walls.

<Image processing device>
First, the configuration and operation of the image processing apparatus of one embodiment will be described with reference to FIG. In FIG. 1, pixels forming a frame 10 of an image signal obtained by photographing a concrete wall surface as shown in FIG. 2 are sequentially input to the edge extraction unit 1. The image signal input to the edge extraction unit 1 is, for example, a still image image signal, and may be a grayscale image signal.

The frame 10 of the image signal includes a non-linear crack line 11 which is a crack generated on a wall surface, a plurality of straight lines 12a to 12c extending in a substantially horizontal direction, and a plurality of straight lines 13a to 13i extending in a substantially vertical direction. .. The straight lines 12a to 12c extending in the substantially horizontal direction are collectively referred to as the horizontal straight line 12, and the straight lines 13a to 13i extending in the substantially vertical direction are collectively referred to as the vertical straight line 13.

The horizontal straight line 12 is slightly tilted from the horizontal, and the vertical straight line 13 is slightly tilted from the vertical. The horizontal straight line 12 may include a completely horizontal straight line, or the vertical straight line 13 may include a completely vertical straight line.

In FIG. 2, the upper left corner of the frame 10 is the coordinate (0,0) which is the origin, and the pixel coordinate is (i, j). Let w be the width (number of horizontal pixels) and h be the height (number of vertical pixels) of the frame 10. In the present embodiment, the horizontal direction is 0 °, and the vertical direction rotated clockwise is 90 °.

The edge extraction unit 1 extracts edges of an image by, for example, applying a differential filter to 9 pixels of horizontal 3 pixels and vertical 3 pixels centered on the pixel of interest. As the differential filter, for example, a Sobel filter is used. The edge extraction unit 1 classifies the pixel of interest into three categories: a horizontal edge (hereinafter, horizontal edge), a pixel of interest is a vertical edge (hereinafter, vertical edge), and a pixel of interest is not an edge. Divide.

The Hough space storage unit 2 performs a Hough transform on each pixel extracted as a horizontal edge or a vertical edge, and stores the Hough-transformed data in the Hough space.

Here, a general Hough transform will be described. As shown in FIG. 3, assuming that the coordinates before the Hough transform are (x, y), the straight line L passing through the coordinates (x, y) is represented by ρ = x · cos θ + y · sin θ. θ is the angle with respect to the horizon, and ρ is the distance from the coordinate (0,0), which is the origin, to the straight line L. There are innumerable pairs of the angle θ and the distance ρ of the straight line passing through the coordinates (x, y). In a general Hough transform, a candidate for a pair of angle θ and distance ρ is voted, and one pair of angle θ and distance ρ is selected by a majority vote. Such a Hough transform requires a huge amount of calculation.

Therefore, in the present embodiment, the angles of the horizontal straight line 12 and the vertical straight line 13 shown in FIG. 2 are regarded as a perfect horizontal straight line or a vertical straight line, and the pixels constituting the horizontal straight line 12 and the vertical straight line 13 are respectively. The huff conversion is limited to 0 ° and 90 °, respectively.

As can be seen from FIG. 3, when the angle θ is 0 °, the straight line L becomes a vertical straight line, and when the angle θ is 90 °, the straight line L becomes a horizontal straight line. Therefore, the Hough space storage unit 2 performs the Hough transform by subtracting the angles 0 ° and 90 ° defined in FIG. 2 from the angle 90 ° and converting them into the angles θ used in the general Hough transform. However, for the sake of brevity, unless it is noted that the angle θ is shown in FIG. 3, the angle in the huff space will be described by the angle defined in FIG.

In the present embodiment, since the angle of the horizontal straight line 12 is limited to 0 °, the straight line including the horizontal edge at arbitrary coordinates (i, j) has an angle of 0 ° from the origin, as shown by a broken line in FIG. The distance of can be represented by j. The huff space storage unit 2 votes 1 for the huff space H [0, j]. Further, since the angle of the vertical straight line 13 is limited to 90 ° in the present embodiment, the straight line including the vertical edge at arbitrary coordinates (i, j) has an angle of 90 ° as shown by the broken line in FIG. The distance from the origin can be represented by i. The huff space storage unit 2 votes 1 for the huff space H [90, i].

The huff space storage unit 2 votes 1 for the huff space H [0, j] or H [90, i] in each pixel extracted as a horizontal edge or a vertical edge in the frame 10. As a result of voting in the frame 10, the huff space H [0, j] at the vertical position j where the horizontal straight line 12 is located and the huff space H [90, i] at the horizontal position i where the vertical straight line 13 is located in FIG. Is a large value.

The huff space storage unit 2 contains a first number of pixels forming an edge with respect to a first set of an angle indicating a perfect horizontal or vertical direction and a distance from the origin of the frame 10 to a straight line. The data represented by the voting value that voted for will be stored. In the case of the image shown in FIG. 2, there are a plurality of first sets.

The inclined straight line fitting unit 3 fits the extracted straight line so as to match the actual horizontal straight line 12 or vertical straight line 13. The inclined straight line fitting portion 3 is a strict horizontal straight line depending on whether the pixels of the horizontal edge are arranged in the horizontal direction, or the pixels of the horizontal edge are sequentially increased or decreased as the pixels of the horizontal edge are moved toward the right side of the frame 10. Determine if it exists. The inclined straight line fitting portion 3 depends on whether the pixels of the vertical edge are arranged in the vertical direction, or the pixels of the vertical edge are shifted to the right side or the left side in order toward the lower side of the frame 10. Determine if it is an exact vertical straight line.

The operation of the inclined straight line fitting portion 3 will be specifically described with reference to FIGS. 4 and 5. As shown in FIG. 4, the inclined straight line fitting unit 3 counts the number of horizontal edges on the straight line connecting the coordinates (0, j) and the coordinates (w, j + n). Here, n is, for example, ± w × 0.1. The angle range shown in FIG. 4 is about ± 6 ° with respect to the horizontal direction from arctan (0.1) = 5.7 °. Let any angle within this angle range be a positive or negative angle p.

The maximum count value of the horizontal edge is edhmax, the angle p when the maximum count value edhmax is obtained is the angle pmax, and the distance between the straight line of the angle pmax and the coordinates (0,0) is rh. The maximum count value of the horizontal edge is represented by the number of pixels.

As shown in FIG. 5, the inclined straight line fitting unit 3 counts the number of vertical edges on the straight line connecting the coordinates (i, 0) and the coordinates (i + m, h). Here, m is, for example, ± h × 0.1. The angle range shown in FIG. 5 is also an angle range of about ± 6 ° with respect to the vertical direction. Let any angle within this angle range be a positive or negative angle q.

The maximum count value of the vertical edge is edvmax, the angle q when the maximum count value edvmax is obtained is the angle qmax, and the distance between the straight line of the angle qmax and the coordinates (0,0) is rv. The maximum count value for vertical edges is expressed in pixels.

In this way, the inclined straight line fitting unit 3 calculates the actual inclination angle (pmax or qmax) of the horizontal edge and the vertical edge, and fits the horizontal edge and the vertical edge to the inclined straight line. The inclined straight line fitting unit 3 calculates the number of second pixels (edhmax or edvmax) of the horizontal edge or the vertical edge included in the inclined straight line and the distance (rh or rv) from the origin to the inclined straight line.

The huff space correction unit 4 has a horizontal straight line 12 determined by the dip straight line fitting unit 3 that the angle pmax is not 0 ° (the actual angle is not 0 °) and the angle qmax is not 90 °. Correct the huff space in the vertical straight line 13 determined to be tilted (instead of the actual angle of 90 °).

The huff space correction unit 4 adds the count value edhmax to the voting value of the huff space H [pmax, rh], and adds the count value edvmax to the voting value of the huff space H [90 + qmax, rv]. Further, the huff space correction unit 4 subtracts the count value edhmax from the voting value of the huff space H [0, j], and subtracts the count value edvmax from the voting value of the huff space H [90, i].

In this way, the huff space correction unit 4 adds the number of pixels of the second set to the vote value of the second set of the actual tilt angle and the distance from the origin to the tilted straight line, and the vote value of the first set. The second pixel count is subtracted to correct the huff space. This modifies the huff space.

The straight line extraction unit 5 extracts data in which the voting value in the huff space exceeds a predetermined threshold value thr as a straight line. As described above, according to the image processing apparatus of the present embodiment, the straight line included in the image can be extracted with a small amount of calculation while using the Hough transform.

The image signal and information indicating the horizontal straight line 12 and the vertical straight line 13 extracted by the straight line extraction unit 5 are input to the image correction unit 6. The information indicating the horizontal straight line 12 and the vertical straight line 13 can be represented by the coordinates of the pixels constituting each of the horizontal straight line 12 and the vertical straight line 13.

The image correction unit 6 erases the horizontal straight line 12 and the vertical straight line 13 or makes the horizontal straight line 12 and the vertical straight line 13 inconspicuous from the image shown in FIG. 2 based on the information indicating the horizontal straight line 12 and the vertical straight line 13. The image is modified so that the crack line 11 is visually conspicuous by processing the image. If the modified image signal is supplied to a display (not shown) to display the modified image, the inspector can easily recognize the crack line 11.

The image processing apparatus shown in FIG. 1 may be configured by hardware, may be configured by software, or may be configured by combining hardware and software.

The image processing device shown in FIG. 1 may be composed of a central processing unit (CPU) and a memory of a microcomputer, or an integrated circuit. The image processing apparatus shown in FIG. 1 may be functionally realized by executing a computer program (image processing program) stored in the memory by the CPU.

<Image processing method and image processing program>
Next, the procedure executed by the image processing program of one embodiment will be described with reference to FIG. FIG. 6 also shows processing by the image processing method of one embodiment.

In FIG. 6, when the image processing program is executed and the processing is started, the CPU captures the image signal in step S11. The CPU extracts the horizontal edge and the vertical edge in step S12. Step S11 and step S12 correspond to the processing executed by the edge extraction unit 1.

In step S21, the CPU votes 1 for the huff space H [0, j] in each pixel of the horizontal straight line 12. Step S21 corresponds to the process executed by the huff space storage unit 2.

In step S22, the CPU counts the number of horizontal edges on a straight line passing through the coordinates (0, j) with an inclination of an angle (0 + p) °. In step S23, the CPU sets the angle p when the maximum count value edhmax is obtained as the angle pmax, and calculates the distance rh between the straight line of the angle pmax and the origin. Steps S22 and S23 correspond to the processes executed by the inclined straight line fitting unit 3.

In step S24, the CPU adds the count value edhmax to the voting value of the huff space H [pmax, rh], and in step S25, subtracts the count value edhmax from the voting value of the huff space H [0, j]. .. Steps S24 and S25 correspond to the processes executed by the huff space correction unit 4.

In parallel with the above steps S21 to S25, the CPU executes steps S31 to S35. In step S31, the CPU votes 1 for the huff space [H90, i] in each pixel of the vertical straight line 13. Step S31 corresponds to the process executed by the huff space storage unit 2.

In step S32, the CPU counts the number of vertical edges on a straight line passing through the coordinates (i, 0) with an inclination of an angle (90 + q) °. In step S33, the CPU sets the angle q when the maximum count value edvmax is obtained as the angle qmax, and calculates the distance rv between the straight line of the angle qmax and the origin. Steps S32 and S33 correspond to the processes executed by the inclined straight line fitting unit 3.

In step S34, the CPU adds the count value edvmax to the voting value of the huff space H [90 + qmax, rv], and in step S35, subtracts the count value edvmax from the voting value of the huff space H [90, i]. .. Steps S34 and S35 correspond to the processes executed by the huff space correction unit 4.

In step S41, the CPU selects data satisfying the huff space H [θ, ρ]> thr as a straight line to be extracted and ends. As a result, the horizontal straight line 12 and the vertical straight line 13 of FIG. 2 are extracted. Step S41 corresponds to the process executed by the linear extraction unit 5. The angle θ and the distance ρ in step S41 are the angle of the straight line defined in FIG. 3 and the distance from the origin to the straight line.

According to the image processing method and the image processing program of the present embodiment, the same effect as the effect of the image processing apparatus of the present embodiment described above is obtained. The image processing program may be stored in a non-temporary storage medium, or may be distributed via a communication line such as the Internet.

<First modification>
In order to facilitate the correction of the image signal in the image correction unit 6, it may be configured as shown in FIG. In FIG. 7, illustrations from the huff space storage unit 2 to the huff space correction unit 4 are omitted. In FIG. 7, the binary conversion unit 7 binarizes the image signal and supplies it to the image correction unit 6.

In this way, the portions of the crack line 11, the horizontal straight line 12, and the vertical straight line 13 in FIG. 2 are represented by white, and the other portions are represented by black. The image correction unit 6 easily replaces the horizontal straight line 12 and the vertical straight line 13 from white to black based on the information indicating the horizontal straight line 12 and the vertical straight line 13. It can be erased.

According to the configuration shown in FIG. 7, if the corrected image signal is supplied to a display (not shown) to display the corrected image, only the crack line 11 is displayed in white, so that the inspector can easily recognize the crack line 11. be able to.

<Second modification>
In the present embodiment described above, the case where the horizontal straight line 12 is a straight line slightly inclined from the horizontal and the vertical straight line 13 is a straight line slightly inclined from the vertical is taken as an example. When the straight line extending in the horizontal direction is greatly inclined from the horizontal and the straight line extending in the vertical direction is greatly inclined from the vertical direction, n of the coordinates (w, j + n) and m of the coordinates (i + m, h) in the inclined straight line fitting portion 3 are increased. It must be a value, and the amount of calculation in the inclined straight line fitting unit 3 increases.

In such a case, as shown in FIG. 8, it is preferable to provide the affine transformation unit 8 in front of the edge extraction unit 1. The affine transformation unit 8 rotates the image signal so that the straight line that is greatly tilted from the horizontal is brought closer to the horizontal (or is completely horizontal), and the straight line that is greatly tilted from the vertical is brought closer to the vertical (or is completely vertical). Let me. The operation after the edge extraction unit 1 is the same as that in FIG. Note that in FIG. 8, the image correction unit 6 is not shown.

<Third variant>
The configuration may be a combination of the first modification shown in FIG. 7 and the second modification shown in FIG.

The present invention is not limited to the present embodiment described above, and various modifications can be made without departing from the gist of the present invention. In the present embodiment, it is assumed that the image signal includes both the horizontal straight line 12 and the vertical straight line 13, but only one of the horizontal straight line 12 and the vertical straight line 13 may be included. In this case, it is sufficient to execute the process of extracting only one of the horizontal straight line 12 and the vertical straight line 13. Although the image signal input to the image processing device is a still image image signal, it may be a moving image signal.

The image correction unit 6 can be omitted when the purpose is only to extract the straight lines included in the image. Of course, an image processing device, an image processing method, and an image processing program for extracting straight lines included in the image may be used for purposes other than modifying the image.

1 Edge extraction unit 2 Huff space storage unit 3 Inclined straight line fitting unit 4 Huff space correction unit 5 Straight line extraction unit 6 Image correction unit 7 Binary conversion unit 8 Affine conversion unit

Claims (4)

An edge extraction unit that extracts at least one edge in the horizontal direction and the vertical direction included in the input image signal,
The edge extracted by the edge extraction unit is regarded as a straight line in a perfect horizontal direction or a vertical direction, and each pixel constituting the edge is Hough transformed to obtain an angle indicating a perfect horizontal direction or a vertical direction. The data represented by the voting value obtained by voting the first number of pixels of the pixels constituting the edge with respect to the first set of the distance from the origin of the frame of the image signal to the straight line is stored in the Hough space. Hough space storage to store and
The actual inclination angle of the edge is calculated, the edge is fitted to the inclined straight line, and the number of second pixels of the edge included in the inclined straight line and the distance from the origin to the inclined straight line are calculated. Inclined straight line fitting to calculate and
The number of the second pixel is added to the voting value of the second set of the actual tilt angle and the distance from the origin to the tilted straight line, and the second pixel is obtained from the voting value of the first set. The Huff space correction part that corrects the Huff space by subtracting the number,
A straight line extraction unit that extracts data of voting values exceeding a predetermined threshold value in the huff space modified by the huff space correction unit as a straight line, and a straight line extraction unit.
An image processing device characterized by comprising. The image signal is an image signal indicating an image including a non-straight line and a straight line in a horizontal direction or a vertical direction.
Claim 1 is characterized by further including an image correction unit that corrects the image signal based on the straight line extracted by the straight line extraction unit so that the non-straight line included in the image can be easily visually recognized. The image processing apparatus according to. Extract at least one of the horizontal and vertical edges contained in the image signal.
The extracted edge is regarded as a straight line in the perfect horizontal or vertical direction, and each pixel constituting the edge is Hough-transformed to indicate an angle indicating the perfect horizontal or vertical direction and the image signal. Data represented by a voting value obtained by voting the first number of pixels of the pixels constituting the edge for the first set of the distance from the origin of the frame to the straight line is stored in the Hough space.
The actual inclination angle of the edge is calculated, the edge is fitted to the inclined straight line, and the number of second pixels of the edge included in the inclined straight line and the distance from the origin to the inclined straight line are calculated. Calculate and
The number of the second pixel is added to the voting value of the second set of the actual tilt angle and the distance from the origin to the tilted straight line, and the second pixel is obtained from the voting value of the first set. Subtract the numbers to fix the huff space,
An image processing method characterized by extracting as a straight line the data of voting values exceeding a predetermined threshold value in the modified huff space. On the computer
A step of extracting at least one edge of the horizontal direction and the vertical direction included in the image signal, and
The extracted edge is regarded as a straight line in the perfect horizontal or vertical direction, and each pixel constituting the edge is Hough-transformed to indicate an angle indicating the perfect horizontal or vertical direction and the image signal. With respect to the first set of the distance from the origin of the frame to the straight line, the step of storing the data represented by the voting value obtained by voting the first number of pixels of the pixels constituting the edge in the Hough space. ,
The actual inclination angle of the edge is calculated, the edge is fitted to the inclined straight line, and the number of second pixels of the edge included in the inclined straight line and the distance from the origin to the inclined straight line are calculated. Steps to calculate and
The number of the second pixel is added to the voting value of the second set of the actual tilt angle and the distance from the origin to the tilted straight line, and the second pixel is obtained from the voting value of the first set. Steps to subtract numbers and modify the huff space,
The step of extracting the data of the voting value exceeding a predetermined threshold in the modified Huff space as a straight line, and
An image processing program characterized by executing.

Images