JP6822919B2 - In-pipe facility inspection system - Google Patents

Description

The present invention relates to an inspection system in a pipeline facility, and in particular, moves in a structure extending in the longitudinal direction, for example, in a pipeline facility such as a sewage pipe, acquires an image, and displays the state of the inner surface of the pipeline facility. Regarding the inspection system in the pipeline facility to grasp.

In recent years, the problem of aging drainage pipes has become more serious. As aging progresses, abnormalities such as cracks and breakage occur. As a result, earth and sand enter the pipe from the outside of the pipe, creating a gap in the ground, which causes an accident in which the road above the sewage pipe collapses. Therefore, there is a need for a technique for efficiently grasping abnormalities such as cracks and breakage in drainage pipes. In response to such needs, a technique has been proposed in which a moving body is moved into a sewer pipe and the state inside the pipe is monitored.

The state inside the pipe is monitored by an image, but it is necessary to know how far the image taken by the photographing device provided in the moving body is in the horizontal direction from the manhole. On the other hand, for example, as in Patent Document 1, a technique for measuring the moving distance of a moving body has been proposed.
In Patent Document 1, an omnidirectional camera having a wide-angle lens is mounted on a moving body that moves in the pipeline, and by moving the moving body in the pipeline, the inner surface of the pipeline is photographed by the omnidirectional camera. It is disclosed that an image taken by an azimuth camera is converted into a flat image by an image processing means and displayed on a display monitoring board on the ground. Then, in Patent Document 1, the position measurement of the moving body equipped with the omnidirectional camera is performed by converting the rotation speed of the guide roller that winds the coaxial cable (used for power supply and transmission of the image signal) into a distance. It is stated that it will be done.

Japanese Unexamined Patent Publication No. 2004-333285

However, in the method described in Patent Document 1 in which the rotation speed of a guide roller for winding a coaxial cable whose one end is connected to a moving body is converted into a moving distance, even if the tension of the cable is controlled, the kale There is a possibility that sagging may occur, and the travel distance converted thereby may include an error. Further, in Patent Document 1, no consideration is given to a mobile body driven by wireless communication and a battery (battery).
Therefore, the present invention is a pipe capable of acquiring the moving distance of a moving body from image data imaged for inspection in a pipeline facility extending in the longitudinal direction without requiring a cable or a moving distance sensor in the horizontal direction. Provide an in-road facility inspection system.

In order to solve the above problems, the pipeline in-facility inspection system according to the present invention is equipped with an image pickup device and controls a moving body that moves in the pipeline facility extending in the longitudinal direction and the movement of the moving body. The image processing device includes a control terminal and an image processing device that processes image data captured by the image pickup device, and the image processing device captures images at two different points in the longitudinal direction in the pipeline facility. The area extraction unit that acquires the first image data and the second image data and extracts a specific area from the first image data or the second image data, and the extracted specific area are the other images. A search unit that searches for a corresponding region in the data, a position difference extraction unit that obtains a position deviation between specific regions in the first image data and the second image data, and a position difference extraction unit that obtains the deviation of the extracted positions. An inner surface distance calculation unit that calculates the distance from the inner surface of the pipeline facility to the image pickup device, and
It is characterized by having a moving distance calculation unit that obtains a distance between the two points based on the viewing angle of the specific region and the distance from the inner surface to the imaging device.

According to the present invention, a pipe capable of acquiring the moving distance of a moving body from image data imaged for inspection in a pipeline facility extending in the longitudinal direction without requiring a cable or a moving distance sensor in the horizontal direction. It becomes possible to provide an in-road facility inspection system.
Issues, configurations and effects other than those described above will be clarified by the description of the following embodiments.

It is an overall schematic block diagram of the inspection system in a pipeline facility of Example 1 which concerns on one Example of this invention. It is a functional block diagram of the image processing apparatus which concerns on Example 1. FIG. It is a block diagram of the main part of the image processing apparatus which concerns on Example 1. FIG. It is a schematic diagram of the direct view image taken by the image pickup apparatus shown in FIG. It is a schematic diagram which shows an example of the position of the 1st region and the 2nd region extracted from the captured image data. It is a schematic diagram which shows the positional relationship of the image pickup apparatus which images the inner surface of the structure (pipeline facility) extending in the longitudinal direction from two different points. It is a schematic diagram which shows that the 1st region extracted from the captured image data is at a different position in a 1st image data and a 2nd image data. It is a schematic diagram which shows that the 2nd region extracted from the captured image data is at a different position in a 1st image data and a 2nd image data. It is a schematic diagram which shows that the positions of the 1st region and the 2nd region extracted from the photographed image data are at different positions in the 1st image data and the 2nd image data, and each has a different deviation. It is a schematic diagram which shows the optical geometric relationship in the 1st image data and the 2nd image data. It is a schematic diagram which shows the positional relationship between the 2nd region included in the 2nd image data and the image center. It is a schematic diagram which shows the optical geometric relation of the angle from the axial direction with respect to the 2nd region included in the 2nd image data. It is a schematic diagram which showed the optical characteristic of an image pickup apparatus. It is a schematic diagram which shows the optical geometric relation of the angle from the axial direction with respect to the 2nd region included in the 2nd image data. This is an example of a screen display example of the display unit. This is another example of the screen display example of the display unit. It is a functional block diagram of the image processing apparatus which constitutes the inspection system in a pipeline facility of Example 2 which concerns on another Example of this invention. It is a schematic diagram of the direct view image taken by the image pickup apparatus. It is a schematic diagram which shows an example of the position of the 1st region extracted from the captured image data. It is a schematic diagram which shows that the 1st region extracted from the captured image data is at a different position in a 1st image data and a 2nd image data. It is a schematic diagram which shows that the position of the 1st region extracted from the captured image data is at a different position in the 1st image data and the 2nd image data, and has a deviation between them. It is a schematic diagram which shows the optical geometric relationship in the 1st image data and the 2nd image data.

In the present specification, the structure extending in the longitudinal direction, that is, the pipeline facility to be inspected by the inspection system in the pipeline facility according to the present invention is, for example, a sewer pipe, a gas pipe, a tunnel, a collecting pipe, and a content. There are water pipes, steam pipes, oil pipes, sewage pipes, drainage culverts, manholes, chimneys, etc. that have been removed and replaced with gas, and are not limited to any of these. Further, the structure (pipeline facility) extending in the longitudinal direction does not have to have a circular cross section, and may have a rectangular cross section.
Further, as a moving body equipped with an imaging device for imaging the inside of the pipeline facility and moving in the pipeline facility, for example, a flying body such as a drone, a vehicle-shaped moving body having wheels or crawlers, a ship, or the like is included. It is not limited to any of these.
In the following, as an example, it is assumed that a sewer pipe is used as a pipeline facility and a flying object such as a drone is used as a moving body, and an embodiment of the present invention will be described with reference to the drawings.

<Structure of inspection system in pipeline facility>
FIG. 1 is an overall schematic configuration diagram of an inspection system in a pipeline facility according to an embodiment of the present invention. As shown in FIG. 1, the pipeline facility inspection system 1 is composed of a mobile body 2 (flying body such as a drone) equipped with an image pickup device 5, a mobile body controller (control terminal) 3, and an electronic terminal 4. To. In the state shown in FIG. 1, the moving body 2 passes through the opening (manhole) 6 on the left side and flies through the structure (pipeline facility) 60 extending in the longitudinal direction toward the opening (manhole) 6 on the right side. It shows the state of doing.
The worker operates the mobile controller (control terminal) 3 and sewage 52 in the structure (pipeline facility) 60 (hereinafter, may be referred to as a sewer pipe) extending in the longitudinal direction. Fly without touching. For example, image data (image data from the image pickup device 5 mounted on the moving body 2 for inspection / investigation (inspection of cracks and breakage) in the structure (pipeline facility) 60 (sewer pipe) extending in the longitudinal direction. The moving body 2 can be moved by a desired moving path by the worker confirming the (direct view image captured in the longitudinal direction in the pipe) on the display unit of the moving body controller (control terminal) 3. ..

In FIG. 1, as shown by the black-painted arrows, the moving body 2 invades the sewer pipe 60 through the opening (manhole) 6 on the left side and flies in the sewer pipe to the opening (manhole) 6 side on the right side. While monitoring the inner surface of the sewer pipe 60, it returns to the outside of the sewer pipe 60 through the opening (manhole) 6 on the right side. Instead of this, as shown by the dotted arrow, the moving body 2 invades the sewer pipe 60 through the opening (manhole) 6 on the left side, and the opening (manhole) 6 on the right side enters the sewer pipe 60. While flying to the side, monitor the inner surface of the sewer pipe 60 (inspect for cracks and damage), turn it back when it reaches the vicinity of the opening (manhole) 6 on the right side or just below the opening (manhole) 6 on the right side, and turn it back to the left side. The moving body controller (control terminal) 3 may be used to return to the outside of the structure (pipeline facility) 60 (sewer pipe) extending in the longitudinal direction through the opening (manhole) 6 of the above.

In this way, while moving in the structure (pipeline facility) 60 (sewer pipe) extending in the longitudinal direction, the image data captured by the image pickup device 5 mounted on the moving body 2 is electronically transmitted by wireless communication. It is transmitted to the terminal 4. The electronic terminal 4 includes an image processing device 7 (not shown), which will be described in detail later, and the image processing device 7 (not shown) executes various processes on the received image data to move at least the moving body 2. Find the distance.

In addition, in order to realize more stable wireless communication between the mobile body 2 and the mobile body controller (control terminal) 3 and wireless communication between the mobile body 2 and the electronic terminal 4, the left and right openings ( It is desirable to install a transmit / receive antenna (not shown) so as to pass through the manhole 6 and extend into the sewer pipe 60. Further, since the moving body 2 uses a battery as a power source, it is desirable to mount a voltage sensor or the like for monitoring the remaining amount of the battery on the moving body 2.

<Configuration of image processing device>
FIG. 2 is a functional block diagram of the image processing apparatus according to the present embodiment, and FIG. 3 is a block diagram of a main part of the image processing apparatus according to the present embodiment.
As shown in FIG. 2, the image processing device 7 captures image data captured in a structure (pipeline facility) 60 (sewer pipe) extending in the longitudinal direction by the image pickup device 5 transmitted from the moving body 2. Communication I / F 31 that can be received, area extraction unit 14 that extracts a specific area (details will be described later) in the received image data, search unit (matching unit) 20, position difference calculation unit 26, left and right inner surface distance calculation unit 32, a viewing angle calculation unit 36, a moving distance calculation unit 38, an integration unit 40, a storage unit 33, and a display control unit 34 are provided, and these are connected to each other by an internal bus 112. Further, the image processing device 7 has a display unit 42 connected to the display control unit 34. The area extraction unit 14, the search unit (matching unit) 20, the position difference calculation unit 26, the left and right inner surface distance calculation unit 32, the viewing angle calculation unit 36, the movement distance calculation unit 38, the integration unit 40, and the display control unit 34 are, for example, , A processor such as a CPU, a ROM for storing the program, and a storage device such as a RAM for temporarily storing data in the process of executing the program read from the ROM.

The storage unit 33 is an image captured in the structure (pipeline facility) 60 (sewer pipe) extending in the longitudinal direction by the image pickup device 5 transmitted from the moving body 2 received via the communication I / F 31. The data, the movement distance integrated value calculated by the integrating unit 40 described in detail later, and the date and time when the image data was captured are linked and stored in a predetermined storage area. Further, the storage unit 33 stores in advance a pipeline network diagram of the structure (pipeline facility) 60 extending in the longitudinal direction to be inspected in a predetermined storage area.

As shown in FIG. 3, the information input to the image processing device 7 is the first image data 12 and the second image data 10. The first image data 12 and the second image data 10 are direct-view images taken in the longitudinal direction in the sewer pipe 60, and a schematic diagram thereof is shown in FIG. FIG. 4 is a schematic view of a direct-view image captured by the image pickup apparatus 5 shown in FIG. Note that FIG. 4 also shows the seam 54 of the sewage 52 and the sewer pipe 60 for the sake of clarity.

[Area extraction unit]
As shown in FIG. 3, the second image data 10 is input to the region extraction unit 14. The region extraction unit 14 extracts two regions, a first region 16 of the second image data 10 and a second region 18 of the second image data 10, from the second image data 10 as specific regions. .. It is desirable that the first region 16 of the second image data 10 and the second region 18 of the second image data 10 are symmetrical with respect to the center of the image and are located as far apart from each other as possible. For example, as shown in FIG. 5, which is a schematic diagram showing an example of the positions of the first region and the second region extracted from the captured image data, the vicinity of the left end of the second image data 10 is the second image. It is desirable to use the vicinity of the right end of the second image data 10 as the first region 16 of the data and as the second region 18 of the second image data. When the liquid is flowing downward like the sewer pipe 60 and the surface features change in a short time, the vicinity of the lower end of the image data should not be used as the first region and the second region. desirable. It is desirable that the first region 16 of the second image data 10 and the second region 18 of the second image data 10 are both located near the edges of the second image data 10, but the lens is distorted. It does not necessarily have to be the edge when there is a large amount of data, or when there is blurring or out of focus. The size of the area 16 of the first area 16 of the second image data 10 and the area 18 of the second area 18 of the second image data 10 is not limited, but if it is desired to speed up the subsequent processing, set it small. Is desirable.

Hereinafter, a case where the first region 16 of the second image data 10 is set near the left end of the image and the second region 18 of the second image data 10 is set in the small region near the right end of the image will be described.

[Search section (matching section)]
As shown in FIG. 3, the search unit (matching unit) 20 inputs the second image data 10 having the first region 16 extracted by the region extraction unit 14 via the internal bus 112 (FIG. 2). To do. The first image data 12 received via the communication I / F 31 (FIG. 2) is transferred to the search unit (matching unit) 20 via the internal bus 112. Here, it is assumed that the first image data 12 captures a wider area than the imaging range of the second image data 10. That is, the imaging position of the first image data 12 is located behind the imaging position of the second image data 10. When an image is taken by the image pickup device 5 mounted on the moving body 2, it is common to take an image data on the traveling direction side of the moving body 2, so that the first image data 12 is the first image data 12. It will be imaged at a time earlier than the image data 10 of 2. FIG. 6 shows the relationship between the imaging positions of the first image data 12 and the second image data 10. FIG. 6 is a schematic view showing the positional relationship of an image pickup device that images the inner surface of a structure (pipeline facility) 60 (sewer pipe) extending in the longitudinal direction from two different points. In FIG. 6, two image pickup devices 56 and an image pickup device 58 are drawn for convenience, but in practice, as shown by the white arrows, before and after one image pickup device is moved by the moving body 2. It shows the positional relationship. The image pickup device 56 at the position where the first image data 12 is imaged is located on the left side, that is, behind the image pickup device 58 at the position where the second image data 10 is imaged (hereinafter, the first image data). It may be referred to as a position 56 in which the image 12 is imaged and a position 58 in which the second image data is imaged).

The search unit (matching unit) 20 searches for a region in which the first region 16 of the second image data 10 best matches among the first image data 12. In this case, the entire area of the first image data 12 may be searched, but if the difference between the imaging times of the first image data 12 and the second image data 10 is short, the second image data It is preferable to search the first image data 12 at the coordinates near the existence of the first region 16 of 10.

Specifically, if the inner surface of the structure (pipeline facility) 60 extending in the longitudinal direction to be inspected has characters or stains having a characteristic shape adhering to the inner surface, these are found. It can be a feature quantity. Therefore, the search unit (matching unit) 20 extracts the contours of characters or stains existing in the first region 16 of the second image data 10 by known image processing, and similarly, the first image data. The contours of characters or stains existing in 12 are extracted by known image processing. Then, the search unit (matching unit) 20 uses the pattern of the outline of characters or stains existing in the first region 16 of the extracted second image data 10 as a template, and similarly extracts the outlines of characters or stains. Then, the pattern matching process is executed on the first image data 12. As a result, among the first image data 12, the region where the first region 16 of the second image data 10 best matches is extracted (searched) as a specific region.
If the inner surface of the structure (pipeline facility) 60 extending in the longitudinal direction to be inspected does not have the above-mentioned stains or characters adhering to the inner surface which can be a feature amount, the search is performed. The unit (matching unit) 20 obtains, for example, the area ratio of the second image data 10 and the first image data 12 of the image of the seam 54, and based on the obtained area ratio, the second image data 12 in the first image data 12. The configuration may be such that the coordinates (pixels) corresponding to the first region 16 in the image data 10 are determined and extracted (searched) as a specific region. Instead of the area ratio, the ratio of the lengths in the circumferential direction in the image of the seam 54 is obtained, and based on the obtained ratio of the lengths, in the first region 16 in the second image data 10 in the first image data 12. Corresponding coordinates (pixels) may be determined and extracted (searched) as a specific area.

FIG. 7 is a schematic diagram showing that the first region extracted from the captured image data is located at different positions in the first image data 12 and the second image data 10, and is a schematic diagram showing an example of the search result. It is a figure. As shown in the lower figure of FIG. 7, the search unit (matching unit) 20 indicates a region 62 in which the extracted first region 16 corresponds to the first image data 12. The position of the first region 16 of the second image data 10 shown in the upper part of FIG. 7 and the extracted first area 16 shown in the lower figure of FIG. 7 correspond to the corresponding area 62 in the first image data 12. Correspondence information 22 (FIG. 3) of the first region composed of positions is transferred to the position difference calculation unit 26 and the viewing angle calculation unit 36 via the internal bus 112.

The search unit (matching unit) 20 also executes the same processing as described above for the second region 18 of the second image data 10. FIG. 8 is a schematic diagram showing that the second region extracted from the captured image data is located at different positions in the first image data 12 and the second image data 10, and is a schematic diagram showing an example of the search result. It is a figure. As shown in the lower figure of FIG. 8, the search unit (matching unit) 20 indicates a region 64 in which the second region 18 of the extracted second image data 10 corresponds to the first image data 12. .. Similar to the case of the first region described above, the position of the second region 18 of the extracted second image data 10 shown in the upper figure of FIG. 8 and the extracted second region 18 shown in the lower figure of FIG. The correspondence information 24 (FIG. 3) of the second region consisting of the positions of the corresponding regions 64 in the first image data 12 of the above is sent to the position difference calculation unit 26 and the viewing angle calculation unit 36 via the internal bus 112. Transferred.

[Position difference calculation unit]
The position difference calculation unit 26 calculates how much the first region 16 of the second image data 10 is deviated from the first image data 12. This deviation is defined as a deviation in a direction parallel to the line connecting the first region 16 of the second image data 10 and the second region 18 of the second image data 10 (horizontal direction). The information of this deviation is transferred to the left and right inner surface distance calculation unit 32 via the internal bus 112 as the deviation 28 of the position of the first region.
Similarly, the position difference calculation unit 26 calculates how much the second region 18 of the second image data 10 is deviated from the first image data 12. This deviation is defined as a deviation in the line-wise direction connecting the second region 16 of the second image data 10 and the second region 18 of the second image data 10. The information of this deviation is transferred to the left and right inner surface distance calculation unit 32 via the internal bus 112 as the deviation 30 of the position of the second region.
[Left and right inner surface distance calculation unit]
FIG. 9 shows that the positions of the first region and the second region extracted from the captured image data are different positions in the first image data 12 and the second image data 10, and each has a different deviation. It is a schematic diagram. An example is shown in which the deviation 28 of the position of the first region is larger than the deviation 30 of the position of the second region.
The left and right inner surface distance calculation unit 32 takes an image from the inner surface of the structure (pipeline facility) 60 extending in the longitudinal direction based on the values of the deviation 28 of the position of the first region and the deviation 30 of the position of the second region. Calculate the distance to the device. FIG. 10 is a schematic diagram showing an optical geometric relationship between the first image data 12 and the second image data 10. The first region 16 of the second image data 10 and the second region 18 of the second image data 10 are projected onto the equivalent image plane 66 on which the first image data 12 is projected by the focal length 76 of the image pickup apparatus 5. Is also reflected. The reflected positions are the first region 16 of the second image data 10 and the second region 18 of the second image data 10 on the equivalent image plane 68 on which the second image data 10 is projected. The projected position is deviated by the deviation 28 of the position of the first region and the deviation 30 of the position of the second region, respectively. This is a state in which the moving body 2 on which the image pickup apparatus 5 is mounted is displaced in the longitudinal direction by a predetermined amount from the central portion in the horizontal direction in the structure (pipeline facility) 60 extending in the longitudinal direction. This is due to going straight along the longitudinal direction in the structure (pipeline facility) 60.

The ratio of the deviation 28 of the position of the first region and the deviation 30 of the position of the second region is a coefficient related to the viewing angle of the image pickup apparatus 5 based on the geometrical positional relationship, and the first image data 12 is imaged. It can be expressed by the following equation (1) using the difference 70 between the position 56 and the position 58 where the second image data is captured, the distance 72 from the left inner surface, and the distance 74 from the right inner surface. However, dL: deviation of the position of the first region 28, dR: deviation of the position of the second region 30, L: distance 72 from the inner surface on the left side, R: distance 74 from the inner surface on the right side, m: first image The difference 70 between the position 56 where the data 12 is captured and the position 58 where the second image data 10 is captured, a: a coefficient related to the viewing angle of the imaging device.
dL: dR = a ・ m + R: a ・ m + L ・ ・ ・ (1)
Here, the difference 70 between the position 56 where the first image data 12 is imaged and the position 58 where the second image data 10 is imaged is sufficiently larger than the distance 72 from the left inner surface and the distance 74 from the right inner surface. If it is small, a is at most 1.0 when the viewing angle is 90 ° (in the case of a wide-angle camera, the viewing angle is generally 90 ° and the coefficient related to the viewing angle in equation (1) is 1.0). , Am, which is a minute term on the right side of equation (1), can be ignored.
dL: dR = R: L ... (2)
Equation (2) holds. When the image pickup device 5 is a device for capturing moving image data, it is usual that 50 to 60 frames of image data can be captured per second, and the position 56 in which the first image data 12 is captured and the second image data The difference 70 from the position 58 in which 10 is imaged can be set sufficiently small. In other words, even if the moving speed of the moving body 2 is taken into consideration, 50 to 60 frames of image data are acquired per second. Therefore, if the frame difference is taken, the position 56 where the first image data 12 is captured is taken. The difference 70 between the image and the position 58 where the second image data 10 is captured is a considerably small value. That is, by using the above equation (2), the structure (pipeline facility) 60 extending in the longitudinal direction is based on the values of the deviation 28 of the position of the first region and the deviation 30 of the position of the second region. The ratio of the distance from the left and right inner surfaces of the image pickup device 5 to the image pickup apparatus 5 can be calculated.

As shown in FIG. 3, the cross-sectional dimension 50 of the inner surface of the structure (pipeline facility) 60 extending in the longitudinal direction is also input to the left-right inner surface distance calculation unit 32, and extends in the longitudinal direction described above. By using the ratio of the distances from the left and right inner surfaces of the structure (pipeline facility) 60 to the image pickup apparatus 5, the distance 72 from the left inner surface and the distance 74 from the right inner surface are calculated. The cross-sectional dimension 50 of the inner surface of the structure (pipeline facility) 60 extending in the longitudinal direction is stored in advance in a predetermined storage area of the storage unit 33 (FIG. 2).
When the cross-sectional shape of the structure (pipeline facility) 60 extending in the longitudinal direction is a circular pipe, the diameter C is defined as the cross-sectional dimension 50 of the inner surface of the structure (pipeline facility) 60 extending in the longitudinal direction. Given, the distance can be calculated by the following equations (3) and (4). Here, L: the distance to the left inner surface 72, R: the distance to the right inner surface 74, dL: the deviation of the position of the first region 28, and dR: the deviation of the position of the second region 30.
L = C × (dR ÷ (dL + dR)) ・ ・ ・ (3)
R = C × (dL ÷ (dL + dR)) ・ ・ ・ (4)
When the cross-sectional shape is rectangular, the distance 72 to the inner surface on the left side and the distance 74 to the inner surface on the right side can be calculated by multiplying the width W of the rectangle instead of the diameter C by the following equations (5) and (6). Can be calculated by.
L = W × (dR ÷ (dL + dR)) ・ ・ ・ (5)
R = W × (dL ÷ (dL + dR)) ・ ・ ・ (6)
[Viewing angle calculation unit]
The viewing angle calculation unit 36 obtains a half viewing angle 48, which will be described in detail later. As shown in FIG. 3, the search unit (matching unit) 20 transfers the correspondence information 22 in the first region and the correspondence information 24 in the second region to the viewing angle calculation unit 36 via the internal bus 112. At the same time, the distance 72 from the left inner surface and the distance 74 from the right inner surface are transferred from the left and right inner surface distance calculation unit 32. First, the second region 18 of the second image data 10 on the right side included in the correspondence information 24 of the second region will be described.
FIG. 11 is a schematic diagram showing the positional relationship between the second region 18 and the image center 78 included in the second image data 10. First, as shown in FIG. 11, the viewing angle calculation unit 36 has the number of pixels in which the second region 18 of the second image data 10 is deviated from the image center 78 of the second image data 10, that is, the image center. The number of pixels 80 from 78 is calculated. This can be easily obtained from the image data.
FIG. 12 shows a schematic diagram showing the optical geometric relationship of the angle from the axial direction (optical axis) with respect to the second region 18 included in the second image data 10 in this case. When the image pickup device 58 at the position where the second image data 10 is imaged faces the axial direction of the structure (pipeline facility) 60 extending in the longitudinal direction, the second region of the second image data 10 Reference numeral 18 denotes a semi-viewing angle 48 from the axial direction (optical axis). The half viewing angle 48 (the angle formed by only one side of the viewing angle with respect to the optical axis) can be obtained by the following procedure based on the number of pixels 80 from the image center 78.

FIG. 13 is a schematic view showing the optical characteristics of the image pickup apparatus. The horizontal axis is the number of pixels from the center of the image, and the vertical axis is the semi-viewing angle (angle from the axial direction). When the number of pixels deviating from the image center 78 is 0, the half viewing angle is 0. As the number of pixels from the image center 78 increases, the angle (semi-viewing angle) from the axial direction (optical axis) also increases. The shape of this optical characteristic differs depending on the imaging device 5, but since it is a characteristic unique to the imaging device 5, it can be obtained in advance based on the specifications of the imaging device 5 or by a simple simulated imaging test. By using the mathematical formula or table of this optical characteristic, the half viewing angle 48 (angle from the axial direction) can be obtained from the image data. The optical characteristics of the image pickup apparatus shown in FIG. 13 are stored in advance in a predetermined storage area of the storage unit 33 (FIG. 2).
The viewing angle calculation unit 36 transfers the obtained half viewing angle 48 to the moving distance calculation unit 38 via the internal bus 112.

[Movement distance calculation unit]
The moving distance calculation unit 38 has a half viewing angle 48 obtained by the viewing angle calculation unit 36, a distance 72 from the left inner surface obtained by the left and right inner surface distance calculation unit 32, and a right inner surface via the internal bus 112. The distance 74 of the above is input, and the distance traveled by the moving body 2 during the imaging of the first image data 12 and the second image data 10 is calculated as follows. FIG. 14 is a schematic view showing the optical geometric relationship of the angle from the axial direction with respect to the second region 16 included in the second image data 10. In FIG. 14, X is an angle (half view angle) from the distance 74, theta _B is the axial direction of the right inner surface, d _b is the axial direction to the second region 18 of the imaging device and second image data 10 The distance is 84. Of these, X and θ _B can be obtained as described above. d _b can be calculated from the following equation by the geometrical relationship (7).
_{d b = X · tan (90} ° -θ B) ··· (7)
Similarly, for the region 64 to which the extracted second region 18 corresponds in the first image data 12, the axial distance 84 between the image pickup apparatus and the second region 18 of the second image data 10 is set. You can ask. If this is the _{d b2,} the position imaged the first image data 12 is the difference 70 position where the second image data 10 of the captured becomes _d b2 -d _b.

Further, in the same manner, the first region 16 of the second image data 10 included in the correspondence information 22 of the first region 16 and the extracted first region 16 correspond in the first image data 12. Using the region 62, the difference 70 between the position where the first image data 12 is imaged and the position where the second image data 10 is imaged can be obtained. The imaging device 56 at the position where the first image data 12 is imaged and the photographing device 58 (FIG. 10) at the position where the second image data 10 is imaged are both structures (pipeways) extending in the longitudinal direction. When the facility) 60 is completely oriented in the axial direction, the two values of the difference 70 between the position where the first image data 12 is imaged and the position where the second image data 10 is imaged should be equal. .. If the two values are different, it is considered that the orientation of the imaging device 5 is misaligned, or the structure (pipeline facility) 60 extending in the longitudinal direction is bent, and the average value is used for movement. The distance is good.
As described above, the moving distance calculation unit 38 uses the distance 72 from the inner surface on the left side, the distance 74 from the inner surface on the right side, and the half-viewing angle 48 to capture the image of the first image data 12 and the second image data. The difference 70 of the position where 10 is photographed is obtained. The difference 70 between the position where the first image data 12 is captured and the position where the second image data 10 is captured, which is obtained by the moving distance calculation unit 38, is transferred to the integration unit 40 via the internal bus 112.

[Integrator]
The integrating unit 40 determines the value of the difference 70 between the position where the first image data 12 is captured and the position where the second image data 10 is captured, which is transferred from the moving distance calculation unit 38 via the internal bus 112. Accumulate in a cycle to obtain the travel distance integrated value 46. That is, when the image data captured by the image pickup device 5 mounted on the moving body 2 is continuously present, the integrating unit 40 is located between two points obtained by the above-mentioned moving distance calculation unit 38 in a predetermined cycle. The distance (the value of the difference 70 between the position where the first image data 12 is captured and the position where the second image data 10 is captured) is integrated to obtain the travel distance integrated value 46.
The integrating unit 40 transfers the travel distance integrated value 46 obtained in a predetermined cycle to the display control unit 34 via the internal bus 112.

[Display control unit and display unit screen display]
FIG. 15 is an example of a screen display example of the display unit 42. As shown in FIG. 15, the display screen 90 of the display unit 42 is composed of a first display area 91 and a second display area 92. In the example shown in FIG. 15, a direct view image in the pipeline (inside the sewer pipe 60), which is an inspection target image, is displayed in the first display area 91, and the pipeline network diagram and the movement of the moving body 2 are displayed in the second display area 92. The case where the black-painted arrow indicating the locus and the movement distance integrated value 46 are displayed as a numerical value 34 m is shown.

When the movement distance integrated value 46 obtained by the integration unit 40 in a predetermined cycle is transferred via the internal bus 112, the display control unit 34 is in the pipeline (when the movement distance integrated value 46 is obtained). By reading the direct view image of (inside the sewer pipe 60) from the storage unit 33 and transferring it to the display unit via the internal bus 112, the direct view of the inside of the pipeline (inside the sewer pipe 60), which is the image to be inspected, in the first display area 91. Display the image. Further, the display control unit 34 reads out the pipeline network diagram from the storage unit 33, generates a black arrow corresponding to the travel distance integrated value 46 in the pipeline network diagram, and displays it together with the numerical value of the travel distance integrated value 46. By transferring to the unit 42 via the internal bus 112, the pipeline network diagram, the black arrow indicating the movement locus of the moving body 2, and the numerical value 34m of the movement distance integrated value 46 are superimposed and displayed on the second display area 92. Let me.
In the example shown in FIG. 15, the pipeline network diagram buried underground in the second display area 92, the black arrow indicating the movement locus of the moving body 2, and the numerical value 34m of the movement distance integrated value 46 are superimposed and displayed. An example of making it work is shown, but it is not limited to this. For example, in addition to the pipeline network diagram, structures on the ground (roads, buildings, etc.) may be superimposed and displayed. Further, only the numerical value of the movement distance integrated value 46 may be displayed in the second display area 92.
The electronic terminal 4 having the display unit 42 (where the image processing device 7 is mounted) may be, for example, a computer having a general monitor, a mobile personal computer, a laptop computer, a tablet, a smartphone, or a mobile phone. Well, it is not particularly limited.

When actually used in the field, the worker may not be able to see the display screen 90 of the electronic terminal 4 due to the operation of the moving body 2, or the display screen 90 may not be clearly visible because the surroundings are too bright outdoors. .. In such a case, it is preferable to have a function of outputting the moving distance (moving distance integrated value 46) as voice information such as sound or voice from the speaker. As a result, the work can be carried out efficiently.

When the imaging device 5 is mounted on the moving body 2, there is a difference in the imaging time of the two image data (first image data 12 and second image data 10). Using the time information, the time difference between the two image data images can be obtained. The moving speed of the moving body can be calculated by dividing the difference 70 between the position where the first image data 12 is imaged and the position where the second image data 10 is imaged by this time difference. It is preferable that the moving speed is also displayed as a numerical value or a figure on the display screen 90.
If the moving speed of the moving body 2 on which the image pickup device 5 is mounted is excessively high, the image data to be captured may be blurred. Since it is not desirable that the image data for monitoring the inside of the sewer pipe 60 is blurred, the moving speed of the moving body 2 needs to be set to a certain level or less. In actual use, it is effective for imaging if the information on the speed of the moving body 2 can be known. However, there are cases where the worker cannot see the display screen 90 due to the operation of the moving body 2, or the display screen 90 cannot be seen well because the surroundings are too bright outdoors. Even in such a case, it is preferable to have a function of outputting the moving speed as voice information such as sound or voice from the speaker.

Further, it is preferable to have a function of issuing an alarm sound when the speed of the captured image data reaches or approaches the level at which blurring occurs. If the moving body 2 is capable of autonomous movement, the moving speed of the moving body 2 may be automatically controlled based on the moving speed obtained as described above.

The moving body not only moves forward, but may also move backward. Even in that case, in order to grasp the positions of the image pickup device 5 and the moving body, it is desirable that the integrating unit 40 can also add, that is, subtract the negative moving distance. Also at this time, it is desirable that the travel distance integrated value 46 is output as a numerical value or a figure.

In FIG. 15, an example of a display screen 90 in which the first display area 91 and the second display area 92 are arranged in different areas has been described, but the present invention is not limited to this, and the display screen 90 is limited to the first display area 91 only. The above-mentioned second display area 92 may be displayed by superimpose in, for example, the lower right portion of the first display area 91 on which the direct-view image is displayed.

FIG. 16 is another example of the screen display example of the display unit 42. As shown in FIG. 16, the display screen 90 of the display unit 42 is composed of a first display area 91 and a second display area 92. In the second display area 92, the pipeline network diagram, the black arrow indicating the movement locus of the moving body 2, and the numerical value of the movement distance integrated value 46 are superimposed and displayed. In this state, when the cursor is moved to a desired position of the black-painted arrow indicating the movement locus of the moving body 2 displayed in the second display area 92 by the worker or the user and clicked, it is designated by the click. The integrated value 46 of the movement distance to the position is displayed as a numerical value in the second display area 92. In the example shown in FIG. 16, the case where the total distance traveled to the designated position 46 is 17 m is shown. The display control unit 34 reads out the direct-view image stored in the storage unit 33 when the total movement distance 46 to a designated position is 17 m, so that the total movement distance 46 is stored in the first display area 91. A direct view image at 17 m is displayed. In this way, the user or the worker can easily confirm the direct view image at the desired position in the sewer pipe 60 even if the inspection of cracks or breakage in the sewer pipe 60 is overlooked by the direct view image. Is possible.

In this embodiment, the case where the image processing device 7 is mounted on the electronic terminal 4 has been described as an example, but a configuration in which some functions of the image processing device 7 are mounted on the mobile body 2 may be used. In this case, for example, among the functional blocks of the image processing device 7 shown in FIG. 2 above, the communication I / F 31, the area extraction unit 14, the search unit (matching unit) 20, the position difference calculation unit 26, and the left and right inner surface distance calculation. The unit 32, the viewing angle calculation unit 36, the moving distance calculation unit 38, and the integrating unit 40 may be mounted on the moving body 2, and the others may be mounted on the electronic terminal 4. Further, the communication I / F 31, the area extraction unit 14, the search unit (matching unit) 20, the position difference calculation unit 26, the left and right inner surface distance calculation unit 32, the viewing angle calculation unit 36, the movement distance calculation unit 38, and the integration unit 40. At least one of them may be mounted on the moving body 2 and the others may be mounted on the electronic terminal 4 to distribute the functions, and the method of distributing the functions may be appropriately set.
Further, in this embodiment, the case where the first region 16 and the second region 18 in the second image data 10 are used as a reference has been described as an example, but the present invention is not necessarily limited to this, and the first image. The above processing may be executed with reference to the two specific areas in the data 12.

As described above, according to the present embodiment, the moving distance of the moving body is obtained from the image data captured for inspection in the pipeline facility extending in the longitudinal direction without requiring a cable or a moving distance sensor in the horizontal direction. It becomes possible to provide an inspection system in the pipeline facility that can obtain the above.
Further, according to this embodiment, the weight of the moving body can be reduced, and the imaging position by the imaging device mounted on the moving body can be grasped while preventing the moving time of the moving body driven by the battery (battery) from being reduced. be able to.
Further, according to the present embodiment, it is possible to capture high-quality image data with less blurring based on the information on the moving speed of the moving body.

FIG. 17 is a functional block diagram of an image processing apparatus constituting the pipeline in-facility inspection system of the second embodiment according to another embodiment of the present invention. The overall configuration of the pipeline in-facility inspection system 1 is the same as in the first embodiment. In this embodiment, the image processing device 7a constituting the pipeline in-facility inspection system 1 has an inner surface distance calculation unit 32a described later. Is different from Example 1. In the following, the same components as those in the first embodiment are designated by the same reference numerals, and the description overlapping with the first embodiment will be omitted.
As shown in FIG. 17, the image processing device 7a of this embodiment is characterized in that it has an inner surface distance calculation unit 32a instead of the left and right inner surface distance calculation unit 32 in the above-described first embodiment.
FIG. 18 is a schematic view of a direct-view image captured by the image pickup apparatus 2. As is clear from comparison with FIG. 4 in Example 1 described above, the seam 54 of the structure (pipeline facility) extending in the longitudinal direction in the direct-view image shown in FIG. 18 is located substantially in the horizontal center of the direct-view image. positioned.
FIG. 19 is a schematic diagram showing an example of the position of the first region extracted from the captured image data. As shown in FIG. 19, in the present embodiment, the region extraction unit 14 extracts only the first region 16 of the second image data 10 from the second image data 10 as a specific region. That is, in this embodiment, only one specific region is extracted from the second image data 10.

FIG. 20 shows a schematic diagram showing that the first region 16 extracted from the captured image data is located at different positions in the first image data 12 and the second image data 10. The search unit (matching unit) 20 is in the first image data 12 as in the above-described first embodiment, based on the second image data 10 having the first region 16 extracted by the region extraction unit 14. Then, the region where the first region 16 of the second image data 10 is most consistent is searched. As shown in FIG. 20, the search unit (matching unit) 20 indicates a region 62 in which the extracted first region 16 corresponds to the first image data 12. The position of the first area 16 of the second image data 10 shown in the upper part of FIG. 20 and the extracted first area 16 shown in the lower part of FIG. 20 correspond to the corresponding area 62 in the first image data 12. Correspondence information 22 of the first region composed of positions is transferred to the position difference calculation unit 26 and the viewing angle calculation unit 36 via the internal bus 112.

Since the position difference calculation unit 26 and the viewing angle calculation unit 36 are the same as those in the first embodiment, description thereof will be omitted here.
FIG. 21 is a schematic diagram showing that the positions of the first region 16 extracted from the captured image data are different between the first image data 12 and the second image data 10 and have deviations thereof. .. The inner surface distance calculation unit 32a calculates the distance from the inner surface of the structure (pipeline facility) 60 extending in the longitudinal direction to the image pickup apparatus based on the value of the deviation 28 of the position of the first region shown in FIG. .. FIG. 22 is a schematic diagram showing an optical geometric relationship between the first image data 12 and the second image data 20. As shown in FIG. 22, the first image data is that the moving body 2 on which the image pickup apparatus 5 is mounted moves in a substantially central portion in the horizontal direction in the structure (pipeline facility) 60 extending in the longitudinal direction. It can be seen from the image pickup device 56 at the position of capturing the image and the image pickup device 58 at the position of capturing the second image data. As shown by the dotted line, assuming that the second region 18 is virtually extracted, the deviation 28 of the position of the first region and the deviation 30 of the position of the second region are equal to each other. Therefore, as shown in FIG. 22, since the distance 72 from the left inner surface and the distance 74 from the right inner surface are naturally equal values, the inner surface distance calculation unit 32a of this embodiment obtains only the distance 72 from the left inner surface. It turns out that this is enough.
Since the moving distance calculation unit 38, the integration unit 40, and the display unit 42 are the same as those in the first embodiment, the description thereof will be omitted here.

As described above, according to the present embodiment, it is possible to finally obtain the total distance traveled by extracting only one specific area from the image data captured by the image pickup apparatus. In addition to the effect, the processing load on the image processing apparatus 7a can be reduced.

The present invention is not limited to the above-mentioned examples, and includes various modifications. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to those having all the described configurations.

1 ... Pipeline facility inspection system 2 ... Mobile 3 ... Mobile controller (control terminal)
4 ... Electronic terminal 5 ... Imaging device 6 ... Opening (manhole)
7,7a ... Image processing device 10 ... Second image data 12 ... First image data 14 ... Area extraction unit 16 ... First area of second image data 18 ... Second area of second image data 20 ... Search section (matching section)
22 ... Correspondence information of the first region 24 ... Correspondence information of the second region 26 ... Position difference calculation unit 28 ... Deviation of the position of the first region 30 ... Deviation of the position of the second region 31 ... Communication I / F
32 ... Left and right inner surface distance calculation unit 32a ... Inner surface distance calculation unit 33 ... Storage unit 34 ... Display control unit 36 ... Viewing angle calculation unit 38 ... Movement distance calculation unit 40 ... Integration unit 42 ... Display unit 46 ... Movement distance integrated value 48 ... Semi-viewing angle 50 ... Cross-sectional dimension of the inner surface of the structure (pipeline facility) extending in the longitudinal direction 52 ... Sewage 54 ... Seam 56 of the structure (pipeline facility) extending in the longitudinal direction 56 ... First image data Imaging device 58 at a position for imaging a second image data ... Imaging device 60 at a position for imaging a second image data ... A structure (pipeline facility) extending in the longitudinal direction.
62 ... The extracted first region corresponds to the first image data 64 ... The extracted second region corresponds to the first image data 66 ... Equivalent to which the first image data is projected Image plane 68 ... Difference between the position where the image data of the equivalent image plane 70 ... 1 on which the second image data is projected and the position where the second image data is captured 72 ... Distance between the left inner surface 74 ... Right inner surface Distance 76 ... Focus distance 78 ... Image center 80 ... Number of pixels from image center 84 ... Axial distance between the image pickup device and the second area of the second image data 90 ... Display screen 91 ... First display area 92 … Second display area 112… Internal bus

Claims (8)

A moving body equipped with an imaging device and moving in a pipeline facility extending in the longitudinal direction,
A control terminal that controls the movement of the moving body,
An image processing device for processing image data captured by the image pickup device is provided.
The image processing device is
The first image data and the second image data captured by the image pickup apparatus are acquired at two points different in the longitudinal direction in the pipeline facility, and the first image data or the second image data is obtained. An area extraction unit that extracts the first area as a specific area on the right side and extracts the second area as a specific area on the left side .
A search unit in which the extracted specific area searches for a corresponding area in the other image data,
The position deviation between the first region in the first image data and second image data, a position difference extraction unit for determining the deviation of the position between the second region,
The position deviation between the extracted first region, and the inner surface distance calculating unit for determining the distance from the conduit interior surface properties based on the deviation of the position between the second region to the imaging device,
A pipeline in-facility inspection system comprising a moving distance calculation unit that obtains a distance between the two points based on a viewing angle of the specific region and a distance from the inner surface to the imaging device. In the pipeline facility inspection system according to claim 1,
The image processing device is
An inspection system in a pipeline facility, characterized in that it has an integrating unit that integrates the distances between two points obtained by the moving distance calculation unit at a predetermined cycle and calculates the integrated moving distance. In the pipeline facility inspection system according to claim 2,
The image processing device is
An inspection system in a pipeline facility including an output device that outputs the integrated travel distance obtained by the integrating unit as display information and / or audio information. In the pipeline facility inspection system according to claim 1 ,
The image processing device mounts the image processing device by dividing the distance between two points obtained by the moving distance calculation unit by the difference between the imaging times of the first image data and the second image data. A pipeline in-facility inspection system including a moving speed calculation unit for obtaining the moving speed of the moving body. In the pipeline facility inspection system according to claim 4 ,
The image processing device is
Inspection in a pipeline facility including an output device that outputs an alarm as display information or voice information when the moving speed of the moving body obtained by the moving speed calculation unit exceeds a predetermined value. system. In the pipeline facility inspection system according to claim 2,
When the moving body equipped with the imaging device retracts in the pipeline facility, the integrating unit subtracts the traveling distance between the two points from the integrated traveling distance, which is an inspection system in the pipeline facility. .. In the pipeline facility inspection system according to claim 5 ,
The output device has a display unit and has a display unit.
The display screen of the display unit displays a first display area for displaying a direct-view image in the pipeline facility imaged by the imaging device, and at least the integrated movement distance obtained by the integration unit as display information. A pipeline in-facility inspection system characterized by having two display areas. In the pipeline facility inspection system according to claim 7 ,
An inspection system in a pipeline facility, characterized in that the integrated movement distance obtained by the integration unit and a pipeline network diagram stored in advance are superimposed and displayed on the second display area.

Images