JP4515787B2 - Method for processing images in tube - Google Patents

Description

  The present invention creates an expanded view of the tube based on the video data obtained by photographing the inner wall surface of the large and small caliber, and uses this expanded view to create an image in the tube for obtaining three-dimensional data. It relates to a processing method.

  Conventionally, the inspection of the wall surface of the pipe has been mainly performed by drawing a deformation by visual observation while a person walks around in a dark and long pipe and creating a development view based on the drawing. In this method, the object to be inspected is a large structure, and there are many long pipes. Therefore, not only a lot of manpower and time are required, but it is difficult to accurately capture all deformations. In addition, inspection of the wall surface of the pipe wall, which has a small diameter for water supply and sewerage and cannot be entered by humans, is carried out by moving a video camera or the like, but it cannot be accurately inspected.

An apparatus for processing a tube inner surface image for continuous inspection is already known (Patent Document 1). As shown in FIG. 5, a so-called fish-eye lens 61 having an angle of view (vertical angle of view) of 180 degrees or more is used as the omnidirectional sensor (or panoramic sensor) of the camera to shoot. is there. The fish-eye lens 61 is a double-sided convex lens as a whole, but has a shape cut so that the central portion of one surface (the right side surface in FIG. 5A) is concave and the central portion of the other surface is flat. None, the concave cut portion on one side is the reflecting surface 65 of the convex mirror viewed from the inside of the fisheye lens 61, and the flat cut portion on the other side is the transmitting surface 66 from which the light is refracted and emitted. None, the outer peripheral surface on the reflecting surface 65 side of the convex mirror constitutes an incident surface 68 on which light is incident, and the outer peripheral surface on the transmitting surface 66 side constitutes a reflecting surface 67 on which light is reflected.
For example, as shown in FIG. 5B, the fisheye lens 61 configured in this way is visible with a maximum shooting angle αmax of 115 degrees and a minimum shooting angle αmin of up to 45 degrees with respect to the optical axis S of the video camera 35. It is assumed that the range is configured.

The image data when the center line optical axis S of the fisheye lens 61 and the center line of the tube 10 are set to coincide with each other will be described.
As shown in FIG. 5C, the image data (x, y) of the shooting angle α within the visible range of the wall surface 11 is a circle whose center is the fisheye lens 61 and whose radius is proportional to the shooting angle α.

As shown in FIG. 6, when the video camera 35 is set in the center of the tube 10 and toward the optical axis S of the center line, the position of the wall surface 11 of the tube 10 and the coordinate data of the development view Explain the correspondence.
As shown in FIG. 7A, the image of one frame when the maximum shooting angle αmax = 90 degrees and the minimum shooting angle αmin = 45 degrees projected by the video camera 35 is a donut-shaped wall image 75. The central portion (shaded portion) becomes a non-projection portion 73, and among these, the wall surface image 75 is converted into a coordinate point (m, n) on the development view corresponding to the coordinate point (x, y) on the actual image data by the expansion processing. When a development view is created from the obtained coordinate data, the development view is developed in the longitudinal direction of the tube rod 10 as shown in FIG.
6 and 7 (a) and 7 (b), assuming that the mesh or lattice pattern in the optical axis direction and the circumferential direction is a typical crack, 14a in FIG. 14b in FIG. 7A shows a crack in the image projected by the video camera 35, and 14c in FIG. 7B shows a crack in the developed view.

  The fisheye lens 61 is an area where the substantially central portion is hardly used as a reflecting mirror. Therefore, as shown in FIG. 8A, the central portion of the reflection surface 65 is removed to form a transmission portion 70. Then, the linear video signal e from the front of the tube 10 can be taken in, and the video signal e is originally located at the center non-projection part 73 as shown in FIG. 8B. Can be displayed as an image f. Therefore, it can be used as the forward monitoring image 74 of the tube 10.

It is desirable that the non-projection unit 73 projects the monitoring image 74 as much as possible on the whole. For this purpose, a zoom lens 71 is interposed between the relay lens 62 and the image sensor 63. Then, in FIG. 8A, the video signal e is enlarged and projected like the video signal F (FIG. 8C).
If the area of the video signal e for forward monitoring is too narrow, the wide-angle lens 72 is interposed in front of the fisheye lens 61, but the video signal e is not enlarged as shown in FIG. The area is expanded.

A development drawing creation order by the processing device for the inner surface of the tubular culm in Patent Document 1 will be described.
The video camera 35 travels and photographs every predetermined distance, and the image signal of the wall surface 11 of the tube rod 10 is input for each frame, and the image data is developed. Since the fisheye lens 61 is used as an omnidirectional sensor to photograph the wall surface 11 of the tube rod 10, correction of the tube rod depth direction, center point correction, and inclination correction are performed in order to accurately create a development view. Since the image data expansion process is performed for each frame as shown in FIG. 4A, the more the image after the expansion process is located at the back of the tube, the less the information becomes and the distorted image becomes. In the developed view shown in FIG. 4A, 57 is a joint, 58 is a crack, 59 is a water surface, and 60 is dirt. The developed image data for each frame is stored in the developed data storage unit.

After the development process, in order to remove the distorted portion, as shown in FIG. 4B, the development images for a plurality of frames are synthesized. At the time of synthesis, since the center point of the tube 10 may be different for each image, data matching is performed.
As shown in FIG. 4C, the combined developed view information is stored in a developed view storage unit, displayed on a monitor as needed, and printed out by a printer.

The problem to be solved is that, as shown in FIG. 1, when a mounting tube 13 protruding inside is attached to the tube rod 10, the three-dimensional data is shown in a side development view with an angle of view of approximately 90 degrees. It is that it cannot be asked.
That is, in FIG. 4, after the development process, in order to remove the distorted portion, as shown in FIG. 4B, the development images for a plurality of frames are synthesized. Also, only a so-called laterally developed image directly beside the video camera 35 is shown. For example, in FIG. 4C, when the mounting tube 13 protruding in one laterally developed image is shown, is the protruding mounting tube 13 protruding beyond the wall surface 11 of the tube rod 10? It is impossible from these images to obtain three-dimensional data, such as how much or not, if there is no protrusion.

  More specifically, as shown in FIG. 1, when the protruding mounting tube 13 protrudes from the wall surface 11 of the tube rod 10, the necessity for repair is determined depending on how much the protruding amount y is. . Specifically, when y is the protruding amount of the protruding mounting tube 13 and r is the outer diameter (or inner diameter) of the protruding mounting tube 13, an emergency measure is taken when y ≧ (r / 2). If (r / 2)> y ≧ (r / 10), it is necessary to take measures within a few years, and if (r / 10)> y, take immediate measures. Judgment criteria that are deemed unnecessary are provided.

  An object of the present invention is to provide a method capable of capturing three-dimensional data of an object photographed using a developed image.

  The present invention is not limited to a lateral view with a substantially right angle of view from the side image extraction unit of the pipe in a single run by an omnidirectional sensor, but also a forward view with an inclined angle of view within a range of about 30 to 70 degrees. Can be captured at the same time, so that the corresponding development of the side image and the development of the front image are overlaid after matching the distance so that these corresponding points match. It calculates 3D data.

According to the first aspect of the present invention, the side image and the front image are obtained from the side image extraction unit including the extraction point of the three-dimensional data obtained by photographing the wall surface of the pipe with a video camera mainly composed of an omnidirectional sensor. A step of expanding the side image to obtain a developed image of the side image, a step of expanding the front image to obtain a developed image of the front image, and use when creating the developed image of the front image The same feature points of the step of adjusting the reference distance of the developed image of the side image based on the angle of view and the radius of the tube, and the developed image of the side image and the developed image of the front image after the reference distance adjusting step And a step of calculating three-dimensional data from the positional deviation of the same feature points that are associated with each other, so that a side image and a front image are obtained from the side image extraction unit by the omnidirectional sensor. To perform complex video processing and complex operations by taking advantage of Ku can simply calculate the three-dimensional data only to the processing of Kanmizo the image.
Therefore, the present invention is a three-dimensional image using a development developed by photographing the inner wall surface of a large / small diameter pipe, such as an upper / sewage pipe, a traffic tunnel such as a railway / roadway / sidewalk, and a cable buried pipe. It can be used in a method for processing images in a tube to obtain data.

From the side surface drawing extractor obtain lateral image and the front image process takes use angle lateral image containing the extracted points of the three-dimensional data in the side surface image extracting unit in a position of substantially 90 degrees in Kanmizo And a step of taking a front image including the extraction point of the three-dimensional data in the side image extraction unit in the side wall at a position where the use angle of view is 30 to 70 degrees. As for the relationship between the side view with an angle of approximately 90 degrees and the forward image with a use angle of view of 30 to 70 degrees, three-dimensional data can be obtained by solving a very simple arithmetic expression.

When the step of performing the standards distance alignment, which is a straight line connecting the three-dimensional extraction point data and the video camera in the front image, the formed angle between the optical axis of the video camera theta, a radius of Kanmizo and R, Since the reference distance deviation x between the side image and the front image is obtained by calculating x = R / tan θ, the reference distance can be adjusted by a simple trigonometric function.

The step of calculating the three- dimensional data is the step of calculating the three-dimensional data y, where d is the amount of mutual displacement when the same points of the side image and the front image are overlapped. Since d = tan θ is calculated, three-dimensional data can be calculated by a simple triangular function.

The omnidirectional sensor consists of a main reflector that faces away from the incident direction to the video camera and a sub-reflector that faces the incident direction to the video camera at a predetermined distance from the main reflector. The mirror and the sub-reflecting mirror are covered and combined with a semi-spherical transparent transparent cover, and when the side image extraction part of the wall surface enters the main reflecting mirror through this transparent cover, the reflection by the main reflecting mirror is performed. Since the light is reflected by the sub-reflecting mirror and made incident on the video camera, the visible range of the maximum shooting angle and the minimum shooting angle with respect to the optical axis S of the omnidirectional sensor is extremely wide. 3D data can be obtained by taking both a side view with a substantially right angle of view and a front view with an inclined angle of view from the side image extraction unit of the tube rod in a single run.

According to the present invention, a side image and a front image are obtained from a side image extraction unit including an extraction point of three-dimensional data as a protruding amount from a wall surface photographed by a video camera mainly composed of an omnidirectional sensor. A step of obtaining the developed image of the side image by developing the side image, a step of obtaining the developed image of the front image by developing the front image, and a step of creating the developed image of the front image. The same features of the step of adjusting the reference distance of the developed image of the side image based on the angle of view used and the radius of the tube rod, and the developed image of the side image and the developed image of the front image after the reference distance adjusting step a step of performing a mapping of points, Ri Do and a step of calculating the three-dimensional data from the positional deviation of the same feature point correspondence, to obtain a lateral image and the front image from the side surface image extraction unit, the tube Use side image including 3D data extraction point in side image extraction unit Is taken at a position of approximately 90 degrees, and the front image including the extraction point of the three-dimensional data in the side image extraction unit in the side wall is captured and photographed at a position where the use angle of view is 30 to 70 degrees. The step of adjusting the reference distance comprises the step of defining the angle of view between the straight line connecting the extraction point of the three-dimensional data in the front image and the video camera and the optical axis of the video camera, and the radius of the tube When R is set, the reference distance deviation x between the side image and the front image is obtained by calculating x = R / tan θ, and the step of associating includes the same reference distance deviation x. The step of calculating the three-dimensional data by making adjustments so as to match the feature points and calculating the three-dimensional data is the mutual position when the same point of the developed image of the side image and the developed image of the front image are superimposed. When the deviation amount is d, the obtained three-dimensional data y is It is characterized by calculating y = d × tan θ .

An embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a horizontal sectional view passing through the optical axis S of the video camera 35 as the central axis of the cylindrical tube rod 10. The central axis S is provided with a video camera 35 that travels along the central axis S at a predetermined speed. Further, it is assumed that a normal mounting tube 12 that opens to the wall surface 11 from both sides and a protruding mounting tube 13 are attached to the tube rod 10. The normal mounting tube 12 is mounted in a normal state in which the opening tip is substantially coincident with the wall surface 11 of the tube tub 10, and the protruding mounting tube 13 has an opening tip inside the wall surface 11 by a distance y. It shall be attached in an abnormal state protruding into

The video camera 35 has a built-in CCD camera, and as shown in FIG. 2, an omnidirectional sensor 61 having a hole at the center facing the objective lens at the tip of the video camera 35 is attached to the front. ing. The omnidirectional sensor 61 includes a main reflecting mirror 15 facing forward and a sub-reflecting mirror 16 facing a predetermined distance from the main reflecting mirror 15, and the main reflecting mirror 15 and the sub-reflecting mirror 16. Are covered with a semi-spherical transparent transparent cover 18 and coupled. When the side image extraction unit 20 of the wall surface 11 enters the main reflecting mirror 15 through the transparent cover 18, the reflected light from the main reflecting mirror 15 is reflected by the sub-reflecting mirror 16 and enters the video camera 35. Further, a transmission part 70 is formed in the center of the sub-reflection mirror 16, and a concave lens 17 is attached to the transmission part 70, and the front surface of the wall surface 11 from the inside of the hood 19 on the back surface of the sub-reflection mirror 16. The image extraction unit 21 enters the concave lens 17 and further enters the video camera 35.
The omnidirectional sensor 61 is configured such that, for example, the maximum photographing angle αmax is about 115 degrees and the minimum photographing angle αmin is about 30 degrees with respect to the optical axis S of the video camera 35 in the visible range.

Next, a method for calculating three-dimensional data will be described.
As shown in FIG. 1, the video camera 35 is set toward the central axis S at the center inside the tube casing 10.
(1) Creating a development view from a side image Since point A in FIG. 1 is the angle of view θ≈90 degrees of the side image, the image taken by the video camera 35 is as shown in FIG. In addition, the side image 12a1 of the normal mounting tube 12 and the side image 13a1 of the protruding mounting tube 13 are both displayed as a perfect circle having a diameter r ′. 6 and 7, from this image, the coordinate point (m, n) on the development view corresponding to the coordinate point (x, y) on the actual image data is obtained by development processing, and the obtained coordinate data As shown in FIG. 3 (a-2), the side view of the normal mounting pipe 12a2 and the side view of the protruding mounting pipe 13a2 are created. It is displayed as a perfect circle with a diameter r ′.

(2) Creating a development view from the front image Point B in FIG. 1 is a tilt angle within a range such as an angle of view θ = 30 to 70 degrees of the front image. As shown in FIG. 3 (b-1), the normal mounting tube 12 and the protruding mounting tube 13 have an elliptical shape with a short diameter of p (= rtan θ) and a long diameter of r, and the protruding mounting tube 13 Is displayed as a forward photographed image 12b1 of a normal mounting tube and a forward photographed image 13b1 of a protruding mounting tube, where the length of the protruding portion is y. 6 and 7, from this image, the coordinate point (m, n) on the development view corresponding to the coordinate point (x, y) on the actual image data is obtained by development processing, and the obtained coordinate data When the developed view 22b is created from FIG. 3, as shown in FIG. 3 (b-2), the normal mounting tube 12 and the protruding end of the protruding mounting tube 13 have a perfect circle with a diameter r, and the protruding mounting tube 13 The length of the protruding portion is d and displayed as a front expanded image 12b2 of a normal mounting tube and a front expanded image 13b2 of a protruding mounting tube.

(3) Reference distance adjustment of two types of development views Due to the incident angle of the image, there is a deviation between the reference distances of the side image development image 22a and the front image development drawing 22b. For this reason, it is necessary to adjust the reference distance of the side development view based on the relationship between the angle of view θ used when creating the development view 22b of the front image and the position of the tube rod 10.
When the use angle of view of the developed image 23a of the front image is θ and the radius of the tube 10 is R, the reference distance x between the developed image 22a of the side image and the developed image 22b of the front image is expressed by the following equation.
x = R / tan θ
On the basis of the adjustment of the reference distance x, the lateral image development image 22a and the front image development image 22b are both adjusted so that r ′ = r.

(4) Correspondence from two types of development views Using a development image 22a of a side image, a characteristic point is obtained. It is assumed that this feature point is a circular double line representing an outer diameter and an inner diameter displayed as a side development image 13a2 of the mounting pipe protruding. Using this feature point, a point having the same feature, that is, a circular double line is searched from the front development image 13b2 and the side development image 13a2 after the reference distance correction, as shown in FIG. The circular double line of the side developed image 13a2 and the circular double line of the front developed image 13b2 are overlapped for association.

(5) Calculation of the protrusion amount As shown in FIG. 3C, the protrusion amount y is calculated from the positional deviation when the coincident points common to the laterally developed image 13a2 and the forwardly developed image 13b2 are overlapped.
Here, assuming that the radius of the tube rod 10 is R, the creation angle of view of the side development image 22a is 90 degrees, the creation angle of view of the front development drawing 22b is θ, and the displacement amount on these development drawings is d. Is calculated by calculating the following equation.
y = (R × d) / x = d × tan θ

  FIG. 3 (d) shows an example when the common coincidence points of the side developed image 12a2 of the normal mounting tube and the front developed image 12b2 of the normal mounting tube are overlapped. There is no d and the protruding amount y = 0.

It is a cross-sectional top view which shows one Example of the processing method of the image in a tube tube by this invention. It is sectional drawing of the omnidirectional sensor 61 used for the processing method of the image in a tube tube by this invention. It is explanatory drawing explaining the order of the process of the processing method of the tube inner image by this invention. The order of creating a development view from the inner surface image of the tube is shown, (a) is a development view for one frame, (b) is a development view for a plurality of frames, and (c) is a view after adding damage information. FIG. It is for demonstrating the video camera 35 of the apparatus which processes a tube inner surface image, (a) is explanatory drawing of the state which attached the fisheye lens 61 as an omnidirectional sensor to the front-end | tip of the video camera 35, (b) is. The maximum shooting angle αmax, the minimum shooting angle αmin, and the explanatory view of the visible range, (c) is a projection view of the image displayed on the video camera 35. FIG. 3 is a perspective view of a state in which the video camera 35 is set on the tube rod 10. (A) is the actual image data figure before the expansion | deployment of the real tube ridge 10, (b) is an expanded view after expansion | deployment. The other example of the video camera 35 in the apparatus which processes a tube inner surface image is shown, (a) is explanatory drawing for projecting the front monitoring image of a fisheye lens, (b) is a wall image on the display screen 69 75 is an explanatory diagram showing a state in which the forward monitoring image 74 is projected, (c) is an explanatory diagram in which the forward monitoring image 74 is enlarged, and (d) is an explanatory diagram in which the area of the forward monitoring image 74 is enlarged. It is.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Tube rod, 11 ... Wall surface, 12 ... Normal installation pipe, 12a1 ... Side view image of normal installation pipe, 12a2 ... Side development image of normal installation pipe, 12b1 ... Front shot image of normal installation pipe , 12b2 ... Normal front expanded image of the mounting tube, 13 ... Projected mounting tube, 13a1 ... Side view image of the projected mounting tube, 13a2 ... Side developed image of the projected mounting tube, 13b1 ... Projected mounting tube Front shot image, 13b2 ... Front developed image of protruding mounting tube, 14 ... Crack, 14a ... Crack in actual tube, 14b ... Crack in image projected by video camera, 14c ... Crack in developed view, 15 ... Main reflector , 16 ... sub-reflector, 17 ... concave lens, 18 ... transparent cover, 19 ... hood, 20 ... side view extractor, 20a ... side view, 20b ... front view, 21 ... front view extractor,
22a ... developed image of side image, 22b ... developed image of front image, 35 ... video camera, 57 ... seam, 58 ... crack, 59 ... water surface, 60 ... dirt, 61 ... omnidirectional sensor (two-leaf hyperboloid mirror or (Fisheye lens), 62 ... relay lens, 63 ... image sensor, 64 ... illumination lamp, 65 ... reflection surface, 66 ... transmission surface, 67 ... reflection surface, 68 ... incidence surface, 69 ... screen, 70 ... transmission portion, 71 ... zoom Lens: 72 ... Wide-angle lens, 73 ... Non-projection part, 74 ... Monitoring image, 75 ... Wall image.

Claims (2)

Obtaining a side image and a front image from a side image extraction unit including an extraction point of three-dimensional data as a protruding amount from a wall surface photographed by a video camera mainly composed of an omnidirectional sensor,
Expanding the side image to obtain a developed side image;
Expanding the front image to obtain a developed image of the front image;
A step of adjusting a reference distance of the development image of the side image based on a use angle of view when creating the development image of the front image and the radius of the tube;
A step of associating the same feature points of the development image of the side image and the development image of the front image after the reference distance adjustment step;
Ri Do and a step of calculating the three-dimensional data from the positional deviation of the same feature point correspondence,
The step of obtaining the side image and the front image from the side image extraction unit is to capture the side image including the extraction point of the three-dimensional data in the side image extraction unit in the tube at a position where the use angle of view is approximately 90 degrees. And a step of capturing the front image including the extraction point of the three-dimensional data in the side image extraction unit in the tube tube at a position where the use angle of view is 30 to 70 degrees.
The step of performing the reference distance adjustment is such that the angle of view formed by the straight line connecting the extraction point of the three-dimensional data in the front image and the video camera and the optical axis of the video camera is θ, and the radius of the tube rod is R. The reference distance deviation x between the side image and the front image is obtained by calculating x = R / tan θ,
The step of associating is performed by adjusting the reference distance deviation x so that the same feature points coincide with each other,
The step of calculating the three-dimensional data includes the step of calculating the three-dimensional data y, where d is the amount of misalignment when the same points of the side image and the front image are overlapped. A method for processing an image in a tube, wherein y = d × tan θ is calculated . The omnidirectional sensor consists of a main reflector that faces away from the incident direction to the video camera and a sub-reflector that faces the incident direction to the video camera at a predetermined distance from the main reflector. The mirror and the sub-reflecting mirror are covered and combined with a semi-spherical transparent transparent cover, and when the side image extraction part of the wall surface enters the main reflecting mirror through this transparent cover, the reflection by the main reflecting mirror is performed. 2. The method of processing a tube image according to claim 1, wherein the light is reflected by a sub-reflecting mirror and is incident on a video camera.

Images