JP7708610B2 - Tubular structure investigation support device, tubular structure investigation support system, tubular structure image display method, and program - Google Patents

Description

Article 30, paragraph 2 of the Patent Act applies (Website address) Just Co., Ltd. product page https://www. just-ltd. co. jp/project/smakan/ (Published date) July 26, 2021 [Publications, etc.] (Website address) Sewerage Exhibition '21 Osaka exhibitor introduction page https://www. gesuidouten. jp/exhibitors/info/? evcid=GT14J0456 (Published date) July 26, 2021

The present invention relates to a tubular structure inspection support device, a tubular structure inspection support system, an image display method for tubular structures, and a program, and more particularly to image display and information processing technology for supporting the inspection of tubular structures such as sewer pipes.

Inspections are being conducted on the presence and condition of damage to tubular structures such as water supply and sewerage pipes, cable pipes, and tunnels. In these inspections, a camera that travels inside the pipe is used to photograph the inner wall surface along the pipe, and image data of the photographed inner wall surface is used to create an unfolded image on a computer. Workers then visually inspect the unfolded image to determine the presence of damage and prepare an inspection report.

For example, Patent Document 1 describes an intra-pipe work device monitoring system that is composed of an intra-pipe work device that can move inside a pipeline and a ground device equipped with a monitor, in which the ground device displays an image showing the current position of the intra-pipe work device on an expanded image created based on image data of the inner wall of the pipeline.

Patent Document 2 also describes an inner pipe wall image development system that creates an unfolded image based on all-sky image data of the inside of a tubular object captured while moving along the inside of the tubular object. The inner pipe wall image development system of Patent Document 2 holds cables, wires, etc. connected to an image capture means running inside the pipe so that they do not slacken, detects the position of the image capture means from the amount of cable, wire, etc. fed out to obtain position information Sp, and inserts the position information Sp at the time when image data Sv is input from the image capture means into the image data Sv as a digital signal to generate an unfolded image without distortion.

In addition, Non-Patent Document 1 describes a sewerage report creation system for creating sewer pipe inspection reports in a specified format.

Publication No. 2010-066070 Publication No. 2008-090782

Nozawa Electronics Co., Ltd., "Sewerage Report Creation System", [online], searched on August 3, 2021, Internet, <URL:http://nozawa-densi.sakura.ne.jp/CCP004.html>

However, the tasks of assessing damage and creating reports as described above have traditionally been time-consuming, as detailed information about the damage had to be entered manually. Information about pipe joints and attachment pipes also had to be determined manually. Furthermore, for damage that is difficult to determine from the unfolded image alone, such as the intrusion of tree roots, it is necessary to closely observe the condition by referring not only to the unfolded image but also to the original photographed image (the all-sky image mentioned above), but there was room for improvement in the user interface suitable for assessing and inputting damage for this purpose.

The present invention has been made in consideration of the above problems, and aims to provide a tubular structure inspection support device etc. that can reduce the workload in damage assessment work and report preparation work in the inspection of tubular structures and improve work efficiency.

A first invention for solving the above-mentioned problems includes an image acquisition means for acquiring a direct-view image, which is an image captured by a wide-angle camera while proceeding along a pipe inside a tubular structure, and an unfolded image generated based on the direct-view image, a display means for displaying the direct-view image acquired by the image acquisition means, a partial range of the unfolded image, and an overall image, which is the unfolded image covering the entire pipe, side by side, and a playback instruction input means for inputting an instruction operation for playing back the direct-view image as a video, wherein the direct-view image is recorded as a video, and an unfolded image handle provided on the unfolded image and and an overall image handle provided on the overall image, wherein the display means, when an instruction to play back the direct-view image is input by the playback instruction input means, plays back the direct-view image as a video, moves the display range of the unfolded image in conjunction with the direct-view image being displayed, and moves the unfolded image handle in conjunction with the direct-view image being displayed, and moves the overall image handle in conjunction with the direct-view image being displayed, thereby indicating the position of the direct-view image being played back in the pipeline on the overall image and the unfolded image .
In a first invention, when the exfoliated image handle is moved by a user, the display means displays the exfoliated image while moving the display range so that the position indicated by the exfoliated image handle is included in the display area of the exfoliated image, and displays the direct-view image taken at the position indicated by the exfoliated image handle, and moves the entire image handle to indicate a position corresponding to the position indicated by the entire image handle and displays it on the entire image, and when the entire image handle is moved by a user, the display means changes the display range of the exfoliated image so that the position indicated by the entire image handle is included, and displays the direct-view image taken at the position indicated by the entire image handle, and moves the exfoliated image handle to display it on the exfoliated image so that it indicates a position corresponding to the position indicated by the entire image handle.

According to the tubular structure inspection support device of the first invention, a direct-view image of the inside of a tubular structure, a partial range of an unfolded image, and a whole image, which is the unfolded image covering the entire pipe, are displayed side by side. This makes it easy to confirm the position of the observation point in the whole pipe while observing the unfolded image, and also makes it possible to immediately confirm the direct-view image taken at that position. This improves the work efficiency of damage assessment work and report writing work in the inspection of a tubular structure . Since the positions shown in the displayed direct-view image, unfolded image, and whole image are linked to each other, it is possible to smoothly and quickly compare and refer to each image, making it easier to identify damage, pipe structures, etc.

The device further includes a damage input means for displaying a damage input screen for inputting damage information when an arbitrary position on the unfolded image is specified, and a damage recording means for recording the damage information input by the damage input means in association with the information on the specified position. The damage input means is also capable of specifying the position as a range. This allows damage information to be input with a simple operation. Since damage occurs within a range, the damage range can be input with a simple instruction operation, improving work efficiency. It is preferable that the specification of the damage position can be freely switched between specification as a range (box) and specification as a point (dot).

It is also desirable to further provide a preview display means for reading out the damage information recorded in the damage recording means and displaying it in a list together with the overall image. This allows the recorded damage information to be checked in a list while viewing the overall image. Damage assessment is prone to inaccuracies due to human error, but information on each damage can be output to the screen and easily checked before creating a report.

The device further includes an enlargement operation input means for inputting an operation for enlarging and displaying the direct-view image, and when an operation is performed by the enlargement operation input means, the display means enlarges and displays the direct-view image. This makes it possible to display the direct-view image at a size that is easy to observe with a simple operation.

It is desirable to further provide a grid display switching means for displaying/hiding a grid on the unfolded image. This makes it easier to determine the actual dimensions of damage, etc. on the unfolded image.

A second invention includes an imaging device that images the inside of a tubular structure with a wide-angle camera while proceeding along a pipeline, an unfolded image generating device that generates an unfolded image of the tubular structure based on a direct-view image that is an image taken by the imaging device, and a tubular structure inspection support device connected in communication with the imaging device and the unfolded image generating device, wherein the tubular structure inspection support device includes an image acquiring means that acquires the direct-view image and the unfolded image, a display means that displays the direct-view image acquired by the image acquiring means, a partial range of the unfolded image, and an entire image that is the unfolded image covering the entire pipe, side by side, and a playback instruction input means that inputs an instruction operation for playing back a video of the direct-view image. the direct-view image is recorded as a video, and has an unfolded image handle provided on the unfolded image and an overall image handle provided on the overall image, and when an instruction to play back the direct-view image is input by the playback instruction input means, the display means plays back the direct-view image as a video, moves a display range of the unfolded image in conjunction with the direct-view image being displayed, and moves the unfolded image handle in conjunction with the direct-view image being displayed, and moves the overall image handle in conjunction with the direct-view image being displayed, thereby indicating a position of the direct-view image being played back in the pipeline on the overall image and the unfolded image .
In a second invention, when the exfoliated image handle is moved by a user, the display means displays the exfoliated image while moving the display range so that the position indicated by the exfoliated image handle is included in the display area of the exfoliated image, and displays the direct-view image taken at the position indicated by the exfoliated image handle, and moves the entire image handle to display it on the entire image so as to indicate a position corresponding to the position indicated by the entire image handle, and when the entire image handle is moved by a user, the display means changes the display range of the exfoliated image so as to include the position indicated by the entire image handle, and displays the direct-view image taken at the position indicated by the entire image handle, and moves the exfoliated image handle to display it on the exfoliated image so as to indicate a position corresponding to the position indicated by the entire image handle.

According to the tubular structure inspection support system of the second invention, the imaging device captures a direct image of the inside of the tubular structure, the unfolded image is generated by the unfolded image generating device, and the tubular structure inspection support device acquires the direct image and the unfolded image, and the direct image, a part of the unfolded image, and the whole image, which is the unfolded image covering the entire pipe, can be displayed side by side. This makes it easy to confirm the position of the observation point in the whole pipe while observing the unfolded image, and the direct image captured at that position can be immediately confirmed. Therefore, the work efficiency of damage judgment work and report writing work in the inspection of the tubular structure can be improved. Since the positions shown in the displayed direct image, unfolded image, and whole image are linked to each other, the images can be compared and referenced smoothly and quickly, making it easy to distinguish damage, tubular structures, etc. It is to be noted that the unfolded image generating device and the tubular structure inspection support device may be integrated or configured separately.

A third invention is an image display method executed by a computer, the image display method including the steps of acquiring a direct-view image, which is an image taken by a wide-angle camera while proceeding along a pipeline inside a tubular structure, and an unfolded image generated based on the direct-view image, displaying the acquired direct-view image, a portion of the unfolded image, and an overall image, which is the unfolded image covering the entire pipeline, side by side, and inputting an instruction operation for playing the direct-view image as a video, wherein the direct-view image is recorded as a video and has an unfolded image handle provided on the unfolded image and an overall image handle provided on the overall image, and the displaying step plays the direct-view image as a video when an instruction to play the direct-view image is input, moves the display range of the unfolded image in conjunction with the direct-view image being displayed, and moves the unfolded image handle in conjunction with the direct-view image being displayed, and moves the overall image handle in conjunction with the direct-view image being displayed, thereby indicating the position of the direct-view image being played on the overall image and on the unfolded image in the pipeline .
In a third invention, when the exfoliated image handle is moved by the user, the displaying step displays the exfoliated image while moving the display range so that the position indicated by the exfoliated image handle is included in the display area of the exfoliated image, displays the direct-view image taken at the position indicated by the exfoliated image handle, and moves the entire image handle to indicate a position corresponding to the position indicated by the exfoliated image handle and displays it on the entire image, and when the entire image handle is moved by the user, the displaying step can also change the display range of the exfoliated image so that the position indicated by the entire image handle is included, displays the direct-view image taken at the position indicated by the entire image handle, and moves the exfoliated image handle to display it on the exfoliated image so that it indicates a position corresponding to the position indicated by the entire image handle.

According to the third invention, a direct-view image and an unfolded image can be acquired by a computer, and the direct-view image, a partial range of the unfolded image, and an overall image, which is the unfolded image covering the entire pipe, can be displayed side by side. This allows damage assessment work and report writing work, etc., in the investigation of a tubular structure to be performed using a computer. In addition, while observing the unfolded image, the position of the observation point in the entire pipe can be easily confirmed, and the direct-view image taken at that position can be immediately confirmed. Therefore, the work efficiency of damage assessment work and report writing work, etc., in the investigation of a tubular structure can be improved. Since the positions shown in the displayed direct-view image, unfolded image, and overall image are linked to each other, the images can be compared and referenced smoothly and quickly, making it easier to identify damage, pipe structures, etc.

A fourth invention is a program for causing a computer to function as an image acquisition means for acquiring a direct-view image, which is an image captured by a wide-angle camera while proceeding along a pipeline inside a tubular structure, and an unfolded image generated based on the direct-view image, a display means for displaying the direct-view image acquired by the image acquisition means, a partial range of the unfolded image, and an overall image, which is the unfolded image covering the entire pipeline, side by side, and a playback instruction input means for inputting an instruction operation for playing the direct -view image as a video, wherein the direct-view image is recorded as a video, and has an unfolded image handle provided on the unfolded image and an overall image handle provided on the overall image, and the display means plays the direct-view image as a video when an instruction to play the direct-view image is input by the playback instruction input means, moves a display range of the unfolded image in conjunction with the direct-view image being displayed, and moves the unfolded image handle in conjunction with the direct-view image being displayed, and moves the overall image handle in conjunction with the direct-view image being displayed, thereby indicating the position of the direct-view image being played back in the pipeline on the overall image and on the unfolded image .
In a fourth invention, when the exfoliated image handle is moved by a user, the display means displays the exfoliated image while moving the display range so that the position indicated by the exfoliated image handle is included in the display area of the exfoliated image, and displays the direct-view image taken at the position indicated by the exfoliated image handle, and moves the entire image handle to display it on the entire image so as to indicate a position corresponding to the position indicated by the entire image handle, and when the entire image handle is moved by a user, the display means changes the display range of the exfoliated image so as to include the position indicated by the entire image handle, displays the direct-view image taken at the position indicated by the entire image handle, and moves the exfoliated image handle to display it on the exfoliated image so as to indicate a position corresponding to the position indicated by the entire image handle.

The fourth invention allows a computer to function as the tubular structure survey support device in the first invention.

The present invention provides a tubular structure inspection support device that can reduce the workload in damage assessment work and report preparation work in the inspection of tubular structures and improve work efficiency.

FIG. 1 is a diagram showing an example of the overall configuration of a tubular structure inspection support system 1. A block diagram showing the functional configuration of the imaging system 2 and the tubular structure inspection support device 5. Flowchart showing the overall flow of the investigation process Flowchart showing the flow of display processing FIG. 1 shows an example of a main screen 7. An example of a grid display FIG. 13 is a diagram showing a display example of objects 971 and 972 indicating tubular structures (joints). Flowchart showing the flow of damage input processing A diagram showing an example of specifying points of damaged areas An example of box (range) designation of damaged areas Flowchart showing the flow of preview processing FIG. 1 shows an example of a preview screen 15.

The following describes in detail a preferred embodiment of the present invention with reference to the drawings.

Figure 1 is a diagram showing the overall configuration of a tubular structure investigation support system 1 according to the present invention. In the following description, as an example, as shown in Figure 1, a tubular structure investigation support system 1 in which a photography system 2, a storage 3, and a tubular structure investigation support device (hereinafter, investigation support device) 5 are communicatively connected via a network 4 will be described. Note that the system configuration example in Figure 1 is only an example, and the present invention is not limited to this. For example, the storage 3 may be omitted, and the photography system 2 and investigation support device 5 may be connected via the network 4, or the photography system 2 and investigation support device 5 may not be communicatively connected via the network 4, and image data and the like may be exchanged via a recording medium or the like.

The photography system 2 includes a photography device 22 that photographs the inside of a tubular structure 10 (such as a water supply and sewerage pipe, an intake and exhaust pipe, a cable pipe, and a tunnel, hereinafter referred to as a pipe 10) to be inspected with a wide-angle camera 22A while traveling along the pipe, and an on-site PC 21 that is a computer terminal used at the inspection site. The photography device 22 is a traveling vehicle equipped with a wide-angle camera 22A and an encoder. The encoder is a cable or the like for measuring the distance from the entrance of the pipe to the wide-angle camera 22A. The on-site PC 21 includes an interface for acquiring the image (hereinafter referred to as a direct-view image) captured by the photography device 22 and the distance measured by the encoder, a communication interface for connecting to the network 4, a control unit (CPU, ROM, RAM), a memory unit, an input unit, a display unit, etc. It is preferable that the on-site PC 21 has a function for generating an expanded image based on the direct-view image acquired from the photography device 22, but the expansion image may be generated by a separate computer terminal.

The unfolded image is an image obtained by cutting open an image of the inside of the pipe 10 in the pipeline direction and unfolding it into a flat surface. The method of creating an unfolded image is known, and is described, for example, in JP 2010-066070 A. The imaging system 2 can also be configured using the work equipment and ground equipment shown in the above-mentioned patent document, or a similar known imaging system. The on-site PC 21 records the distance information obtained by the encoder (the distance from the pipe entrance to the wide-angle camera 22A) as position information (position in the pipeline direction) in the captured image (direct view image) and unfolded image.

The on-site PC 21 stores the captured direct-view image in the storage 3 via the network 4. When the on-site PC 21 generates an unfolded image, the generated unfolded image is linked to the direct-view image and stored in the storage 3. Alternatively, the on-site PC 21 may transmit the direct-view image and the unfolded image to the investigation support device 5 via the network 4. Alternatively, the on-site PC 21 may record the direct-view image and the unfolded image on a recording medium.

The storage 3 is a storage server accessible via the network 4, and has a memory area for storing direct-view images, unfolded images, construction information, etc. transmitted from the site PC 21. When the storage 3 receives an image acquisition request from the investigation support device 5, it transmits the corresponding direct-view images or unfolded images in response to the request.

Next, the investigation support device 5 will be described. As shown in FIG. 2, the investigation support device 5 is configured by a computer in which a control unit 51, a memory unit 52, a communication unit 53, an input unit 54, a display unit 55, and a peripheral device I/F (interface) unit 56, etc. are connected via a bus, and a PC, a tablet, a smartphone, etc. can be used. Note that the configuration of the investigation support device 5 can be changed as appropriate. A tubular structure investigation support program is installed in the investigation support device 5, and the control unit 51 executes processing in accordance with the tubular structure investigation support program to realize each of the functions described below.

The control unit 51 is composed of a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc. The CPU loads programs stored in the memory unit 52, ROM, etc. into the work memory area on the RAM and executes them, and drives and controls each unit connected via the bus (memory unit 52, communication unit 53, input unit 54, display unit 55, peripheral device I/F unit 56). The ROM permanently stores programs such as the boot program and BIOS, data, etc. The RAM temporarily stores loaded programs and data, and also provides a work area used by the control unit 51 to perform various processes.

The memory unit 52 is a storage device such as a flash memory or a hard disk, and stores acquired image data, input damage information, input pipe structure information, etc. The memory unit 52 also stores processing programs related to each function described below (collectively referred to as "tubular structure survey support programs" or "apps").

The communication unit 53 has a WiFi antenna, a wireless communication unit such as Bluetooth, or a communication port for a wired communication unit such as a LAN, and a communication control device, and is an interface that mediates communication with external devices.

The input unit 54 includes, for example, a touch panel, a keyboard, a pointing device such as a mouse, an input device such as a numeric keypad, and the like, and inputs the input data to the control unit 51.

The display unit 55 is composed of a display such as a liquid crystal panel and a logic circuit (such as a video adapter) for executing display processing in cooperation with the display, and causes the display to display the display data input under the control of the control unit 51. The display unit 55 may be a touch panel display in which an input device (input unit 54) such as a touch panel is integrally provided on the display screen.

The peripheral device I/F unit 56 is a port for connecting peripheral devices, and the control unit 51 transmits and receives data to and from the peripheral devices via the peripheral device I/F unit 56. The peripheral device I/F unit 56 is configured using a USB (Universal Serial Bus) or the like. The connection with the peripheral devices may be wired or wireless.

Next, the functional configuration of the research support device 5 will be described with reference to FIG.
The investigation support device 5 has, as its functional units, an image acquisition unit 511, a display processing unit 513, a tubular structure input unit 514, a damage input unit 515, and a preview display unit 516. The investigation support device 5 may also include an unfolded image generating unit 512 that generates an unfolded image. These functional units are realized by the CPU of the control unit 51 reading a processing program (tubular structure investigation support program) stored in the storage unit 52, calling it up into a work memory area on the RAM, and executing it.

The image acquisition unit 511 acquires the direct-view image captured by the imaging system 2 and the generated unfolded image via the communication unit 53 or the peripheral device I/F unit 56. Alternatively, the image acquisition unit 511 may read and acquire the direct-view image and unfolded image stored in a storage medium, or the communication unit 53 may acquire the direct-view image and unfolded image stored in another computer or the direct-view image and unfolded image stored in the storage 3 via a network 4 such as a LAN or the Internet, and import them into the control unit 51.

The unfolded image generating unit 512 generates an unfolded image based on the direct-view image acquired by the image acquiring unit 511, and stores the unfolded image in the memory unit 52. Note that the unfolded image may be generated in real time while the direct-view image is being captured, or immediately after the direct-view image is captured, at the investigation site, and it may be confirmed at the investigation site whether the conversion was appropriate. In this case, the unfolded image is generated by the on-site PC 21, etc.

The display processing unit 513 executes a display process, and displays the direct-view image 72 acquired by the image acquisition unit 511, a partial range of the unfolded image 71, and an overall image 73, which is the unfolded image 71 covering the entire tube 10, side by side on the display unit 55 (see FIG. 5). In the display process, the display processing unit 513 displays the position of the direct-view image 72 to be displayed on the tube 10 in conjunction with the display range of the unfolded image 71, and indicates the above position on the overall image 73 and the unfolded image 71. Details of the display process and examples of display screens will be described later.

The pipe structure input unit 514 accepts input of information on structures such as joints and attachment pipes of the pipe 10 for images (mainly the unfolded image 71) acquired by the image acquisition unit 511. Then, objects 971, 972 indicating the structures (joints and attachment pipes) are displayed on the unfolded image 71 or the overall image 73 (see FIG. 7). The pipe structure input unit 514 also adds identification information to each of the structures (joints and attachment pipes, etc.), links them to positional information on the pipe 10, and records them as pipe structure information.

When the user designates an arbitrary position on the unfolded image 71, the damage input unit 515 displays a damage information input field 111 for inputting damage information (see Figures 9 and 10) and accepts the input of damage information. In addition, the damage information inputted into the damage information input field 111 is linked to the position designated by the user (position on the pipe 10) and recorded. The damage position can be designated by a point (dot) or a box (range). Details of the processing of the damage input unit 515 and examples of the display screen will be described later.

The preview display unit 516 displays a damage list 150, which is a list of recorded damage information, together with the overall image 73 (see FIG. 12). Details of the preview display process and examples of the display screen will be described later.

Next, we will explain the flow of a tubular structure inspection using the tubular structure inspection support system 1. First, we will explain the overall flow of the inspection with reference to the flowchart in Figure 3.

The worker carries the imaging system 2 to the investigation site and uses the imaging device 22 to capture video of the inside of the pipe 10 (step S101). The captured video (image) is sequentially imported into the site PC 21 as a direct-view image 72. The site PC 21 executes an unfolded image generation process and generates an unfolded image 71 based on the imported direct-view image 72 (step S102).

If the generation of the unfolded image 71 is to be redone (step S103; No), the process returns to step S101. If the generation of the unfolded image 71 is successful (step S103; Yes), the direct-view image 72 and the unfolded image 71 are stored (step S104).

When the processing of steps S101 to S104 is completed, the work at the investigation site is finished and the worker moves to the office (step S105). In the office, the worker performs work using the investigation support device 5.

When the application (tubular structure investigation support processing program) is launched (step S106), the control unit 51 of the investigation support device 5 executes tubular structure investigation support processing. The control unit 51 acquires direct-view images 72 and unfolded images 71 from the storage 3 or the like in accordance with the operator's operation (step S107), and performs display processing (step S108), tubular structure input processing (step S109), damage input processing (step S110), preview display processing (step S111), and report creation processing (step S112), etc. Note that the order of the processing from step S107 to step S112 may be changed or some may be omitted depending on the work content and the operator. The processing of each step is explained below.

First, the display process of step S108 will be described with reference to FIG. 4. The control unit 51 of the research support device 5 displays the main screen 7, which is the main display screen, on the display unit 55, and displays the unfolded image 71, the direct view image 72, and the overall image 73 acquired in step S107 in each display area provided within the main screen 7 (step S301).

Figure 5 is a diagram showing a display example of the main screen 7. In the example of Figure 5, the display area for the expanded image 71 is provided horizontally at the top of the screen, the display area for the direct view image 72 is provided at the bottom left of the screen, and the display area for the overall image 73 is provided at the bottom right of the screen, but the layout and size of each display area are not limited to this.

The unfolded image 71 is displayed with the pipeline direction of the pipe 10 facing horizontally. The vertical center of the unfolded image 71 corresponds to the bottom of the pipe 10, and the upper and lower ends of the unfolded image 71 correspond to the top of the pipe 10. The unfolded image 71 is also provided with a scale 79 indicating the pipeline direction position of the pipe 10, an unfolded image handle 74, and buttons 77a and 77b for scrolling the display range of the unfolded image 71. The unfolded image handle 74 is an operation unit (first position designation means) that can be moved left and right by the user's operation, and the user can move the unfolded image handle 74 to designate any position (pipe direction position) on the unfolded image 71. The pipeline direction position indicated by the scale 79 is the distance from the entrance of the pipe to the camera 22A.

The direct-view image 72 is displayed as a direct-view image 72 taken at a position specified by the expanded image handle 74 (or the whole image handle 76) among the direct-view images 72 taken as a series of videos. That is, among the frames of the video, the frames taken at the position specified by the expanded image handle 74 (or the whole image handle 76) are displayed. It is desirable that the direct-view image 72 displays position information (distance from the entrance of the pipe to the camera 22A) and joint number (identification information of the joint part). A play button 75 (play command input means) is provided near the direct-view image 72. When the play button 75 is operated to input a video play command for the direct-view image 72, the control unit 51 plays the direct-view image 72 in the forward direction (from the start point (start point of shooting) of the pipe 10 to the end point (end point of shooting)). The control unit 51 also moves the entire display range of the expanded image 71 in conjunction with the direct-view image 72 being displayed. The control unit 51 moves the expanded image handle 74 in conjunction with the direct-view image 72 being displayed, and also moves the whole image handle 76 in conjunction with the direct-view image 72. That is, the position of the direct-view image 72 being played is shown on the whole image 73 and the expanded image 71. In addition to the play button 75, a reverse play button for playing in the reverse direction, buttons for fast-forwarding and rewinding, and buttons for moving to the start point and end point may be provided. If the play button 75 is operated again during video playback, the control unit 51 stops playback.

The whole image 73 is an expanded image 71 that is reduced and displayed over the entire pipeline, and is displayed with the pipeline direction of the pipe 10 facing sideways. As with the expanded image 71, the center of the whole image 73 in the vertical direction corresponds to the bottom of the pipe 10, and the upper and lower ends of the whole image 73 correspond to the top of the pipe 10. The whole image 73 is also provided with an whole image handle 76, and movement buttons 78a, 78b, etc. for moving the position of the whole image handle 76. The whole image handle 76 is an operation unit (second position designation means) that can be moved left and right by the user's operation, and the user can move the whole image handle 76 to designate any position (pipe direction position) on the whole image 73. The display position of the whole image handle 76 is moved left and right in conjunction with the position in the pipeline displayed by the direct view image 72 and the position of the expanded image handle 74.

The main screen 7 displays an unfolded image 71, a direct view image 72, an overall image 73, and other function buttons 81 to 85 for executing various functions. Function button 81 is a button operated when manually inputting the position of the joint of the pipe 10, function button 82 is a button operated when specifying the damaged position with a point, function button 83 is a button operated when specifying the damaged position with a box, function button 84 is a button operated when manually inputting the position of the attached pipe, and function button 85 is a button operated when moving the display position of the unfolded image 71. In addition, there are provided a display size (enlargement/reduction rate) change field 86 that is operated when changing (enlarging/reducing) the display range of the expanded image 71, a reset button 87 that is operated when returning the display size (enlargement/reduction rate) to the original, a grid display/non-display switching operation section 88, a pipe information display field 89 that displays various information (route number, manhole number, pipe length, total length, etc.) about the pipe 10 displayed on the main screen 7, pipe change buttons 90 and 91 that are operated when changing the pipe 10 displayed on the main screen 7, and a preview button 92 that is operated when displaying a preview of damage information.

Returning to the explanation of FIG.
When the playback button 75 of the direct-view image 72 is operated on the main screen 7 (step S302; Yes), the control unit 51 plays back the direct-view image 72 and links the display range of the unfolded image 71 to the direct-view image 72 (step S303). The control unit 51 also moves and displays the positions of the unfolded image handle 74 and the whole image handle 76 so as to indicate the positions corresponding to the displayed direct-view image 72 (step S304).

When the unfolded image handle 74 is operated (step S302; No -> step S305; Yes), the control unit 51 displays the unfolded image 71 while moving the display range so that the position of the unfolded image handle 74 is always included in the display area, and also displays the direct view image 72 corresponding to the position of the unfolded image handle 74 (step S306). Also, the position corresponding to the position of the unfolded image handle 74 is shown on the whole image 73 (the display position of the whole image handle 76 is moved; step S307).

When the whole image handle 76 is operated (step S305; No -> step S308; Yes), the control unit 51 changes the display range of the unfolded image 71 so that the position indicated by the whole image handle 76 is included, and displays the direct view image 72 corresponding to the position of the whole image handle 76 (step S309). In addition, the position corresponding to the position of the whole image handle 76 is displayed on the unfolded image 71 (moving the display position of the unfolded image handle 74; step S310).

When other operations are input on the main screen 7 (step S308; No → step S311; Yes), the control unit 51 executes processing according to the operation (step S312). For example, when the grid display/non-display switching operation unit 88 on the main screen 7 is operated, the control unit 51 switches between displaying and non-displaying the grid on the unfolded image 71 as shown in FIG. 6. The display color of the grid can be selected to be white or black, and the grid width can also be changed to 10 [mm], 50 [mm], 100 [mm], etc. By displaying the grid on the unfolded image 71, the position and size of damage and structures can be easily recognized. In addition, when an operation such as a right click of the mouse is input on the direct-view image 72, the control unit 51 enlarges and displays the direct-view image 72. In addition, when the function button 85 is selected and the unfolded image 71 is dragged with the mouse, the control unit 51 changes the display position of the unfolded image 71. Furthermore, when an arbitrary enlargement or reduction ratio is input in the display size (enlargement/reduction ratio) change field 86, the control unit 51 displays the expanded image 71 at the specified enlargement or reduction ratio. When the reset button 87 is operated, the control unit 51 returns the expanded image 71 to the original enlargement or reduction ratio (or a predetermined size that is initially set) and displays the expanded image 71.

In steps S311 and S312, processing is performed according to the operation of the above-mentioned function buttons 81 to 85, full preview button 92, changing the magnification ratio, etc. If no operation is input (step S311; No), the process returns to step S302 and waits for an operation.

Next, the tubular structure input process (step S109 in FIG. 3) will be described.
Function button 81 on main screen 7 is a button for specifying a pipe joint. When function button 81 is selected and any position on the unfolded image (position in the pipe direction) is specified by clicking or other operation, control unit 51 of investigation support device 5 displays objects 971, 972, ... indicating joints superimposed at the specified position. Function button 84 is a button for specifying an attached pipe. When function button 84 is selected and any position on the unfolded image (position in the pipe direction) is specified by clicking or other operation, control unit 51 of investigation support device 5 displays objects 971, 972, ... indicating joints superimposed at the specified position.

For example, as shown in FIG. 7, straight line objects 971, 972, ... are displayed superimposed at the positions of the joints. Although not shown, for example, a circular object is displayed superimposed at the position of the attachment pipe. The control unit 51 also obtains the position information of these pipe structures (distance position in the pipe 10) from the unfolded image 71 and adds identification information to each pipe structure. It is desirable to add the identification information as a combination of an alphabet indicating the type of pipe structure and a number indicating the order from the start point of the pipe 10. For example, identification information such as "J1", "J2", ... is added to the joints. The control unit 51 links the identification information and position information of each pipe structure and stores them in the memory unit 52 as pipe structure information.

As shown in FIG. 7, objects 971 and 972 are displayed at the joints in the unfolded image 71, and identification information 971a "J1" and 972a "J2" are displayed above each object 971 and 972. In addition, pipe structure information 98 such as "pipe mouth" is displayed at the start and end points of the pipe 10.

Next, the damage input process (step S110 in FIG. 3) will be described with reference to the flowchart in FIG.
When the function button 82 is operated on the main screen 7 to switch to point input (step S501; point) and then an arbitrary position (point) on the expanded image is clicked (step S502), the control unit 51 displays a point mark 110 at the clicked position (step S503) and obtains position information on the clicked position in the pipe 10 (step S504). The control unit 51 displays a damage information input field 111 (step S505).

9 is a diagram showing an example of a screen in which a damage information input field 111 is displayed at a point specified. As shown in FIG. 9, a point mark 110 is displayed at a position specified by the user on the unfolded image 71, and a damage information display field 110a is displayed near the point mark 110. The damage information display field 110a displays damage information including damage position information (pipe direction position), joint identification number, installation direction, part, content, etc. Also, a damage information input field 111 is displayed at the bottom of the screen. The damage information input field 111 is provided with a content data tab 111a and a photo tab 111b, and FIG. 9 shows a state in which the content data tab 111a is selected. The content data tab 111a is provided with a distance input field 113, a part input field 114, a joint number display field 112, a remarks field 115, a damage number 116, a damage type input field 117, and a degree input field 118. The distance input field 113 reflects and displays the position information (pipe direction position) of the specified point based on the position information added to the unfolded image 71. The part input field 114 allows the user to select and input the part of the damage in the form of a drop-down list. When pipe structure information has already been input, the joint number display field 112 reflects and displays the corresponding joint number. Any characters or numbers can be input into the remarks field 115. The type and degree of damage can be input into the damage type input field 117 and the damage extent input field 118 in the form of a drop-down list. The damage number 116 indicates the identification information of each damage. Note that since there may be multiple damages in the same location, it is desirable to provide multiple damage type input fields 117 and damage extent input fields 118.

In the photo tab 111b, a direct view image 72 or a magnified photo of the damage corresponding to the specified position can be attached (input). When the worker operates the cancel button 111c, the control unit 51 erases the damage information input field 111 and returns to the main screen 7. When the worker inputs information into the damage information input field 111 (step S506) and operates the confirm button 111d, the control unit 51 stores the information and image input into the damage information input field 111 as damage information in the memory unit 52 (step S507).

On the other hand, when the function button 83 is operated on the main screen 7 to switch to box input (step S501; box) and an arbitrary range of the unfolded image is specified by mouse operation or the like (step S508), the control unit 51 displays a box mark 120 in the specified range (step S509) and obtains position information of the specified range in the pipe 10 (step S510). The control unit 51 displays the damage information input field 111 (step S505).

Figure 10 is a diagram showing an example of a screen in which the damage information input field 111 is displayed by box specification. As shown in Figure 10, a box mark 120 is displayed in the range specified by the user on the unfolded image 71, and a damage information display field 120a is displayed nearby. The damage information display field 120a displays damage information including damage range information (range of the pipe direction position), joint location, installation direction, part, content, etc. In addition, the damage information input field 111 is displayed at the bottom of the screen. The damage information input field 111 is provided with a content data tab 111a and a photo tab 111b, as in Figure 9, and Figure 9 shows a state in which the content data tab 111a is selected. The content data tab 111a is provided with a distance input field 113, a part input field 114, a joint number display field 112, a remarks field 115, a damage number 116, a damage type input field 117, and a degree input field 118. The distance input field 113 reflects the location information of the box-specified range, and is displayed as "XXXm to △△m", etc. The rest is the same as each part of the damage information input field 111 in Figure 9. The same applies to the photo tab 111b.

When the worker inputs information into the damage information input field 111 (step S506) and operates the decision button 111d, the control unit 51 stores the information and images input into the damage information input field 111 as damage information in the memory unit 52 (step S507).

Next, the preview display process (step S111 in FIG. 3) will be described with reference to the flowchart in FIG.
When the preview button 92 is operated on the main screen 7 (step S601; Yes), the control unit 51 of the research support device 5 displays the preview screen 15 on the display unit 55 (step S602).

The control unit 51 acquires the damage information and pipe structure information stored in the memory unit 52, and acquires images (direct view image 72, unfolded image 71, etc.) linked to the damage information (step S603), and displays the damage information reflecting the pipe structure information in a list on the preview screen 15 (step S604).

12 is a diagram showing a display example of the preview screen 15. As shown in FIG. 12, the preview screen 15 displays a damage list 150, an entire image 73, and pipe information 89a. In the entire image 73, a dot mark 110 or a box mark 120 is displayed at the position of the damage recorded as the damage information. In the damage list 150, the damage information 151, 152, ... and the images 72, 71a, 71b, ... linked to each of the damage information 151, 152, ... are displayed in order of the damage number. The damage information 151, 152, ... corresponds to the input information in the damage information input field 111 in FIG. 9 and FIG. 10, and the position information, part information, joint number, remarks, type of damage, and degree (rank), etc. are displayed. In addition, each of the damage information 151, 152, ... is provided with a correction button 151a, 152a, ..., so that the content can be corrected.

When the edit buttons 151a, 152a, ... are operated (step S605; Yes), the control unit 51 accepts the edits to the damage information 151, 152, .... When the worker inputs the edits (step S606), the control unit 51 updates and stores the damage information reflecting the edits in the storage unit 52 (step S607). The control unit 51 stores the date and time of the edit, the edit content, the editer information, etc. as edit history information in the storage unit 52 (step S608). When the edit buttons 151a, 152a, ... are not operated (step S605; No), the preview process ends.

Next, the control unit 51 executes a report creation process (step S112 in FIG. 3). In the report creation process, the control unit 51 reads out construction information, pipe information, damage information, images, etc. from the storage unit 25, and generates a report in which the construction information, pipe information, damage information, images, etc. are entered according to a predetermined format determined for each local government. The control unit 51 displays the generated report on the display unit 55. In addition, the control unit 51 executes printing out the report, storing and transmitting the report data, etc., according to instructions from the worker.

As described above, in the tubular structure investigation support system 1, the tubular structure investigation support device 5 acquires the direct-view image 72 captured by the imaging system 2 and the unfolded image 71 generated by the on-site PC 21 or the tubular structure investigation support device 5, and displays the direct-view image 72, a portion of the unfolded image 71, and the overall image 73, which is the unfolded image 71 covering the entire pipe, side by side on the main screen 7. The position of the displayed direct-view image 72 in the pipe 10, the position of the handle 74 on the unfolded image 71, and the position of the handle 76 on the overall image 73 can be linked. In addition, the damaged area can be specified by a point or a box on the unfolded image 71, and the position information (point or range position information) can be automatically calculated from the specified area and reflected in the damage information. In addition, an object indicating a pipe structure (joint, attachment pipe, etc.) can be input into the unfolded image, numbered, and recorded as pipe structure information. In addition, the pipe structure information can be reflected in the damage information. In addition, a preview screen 15 can be displayed in which the recorded damage information is displayed in a list together with the overall image 73. This makes it possible to efficiently assess damage information and prepare reports.

Although the preferred embodiment of the present invention has been described above with reference to the attached drawings, the present invention is not limited to the examples. For example, the layout of each screen, the display size of each image, and the arrangement of function buttons, operation buttons, display fields, input fields, etc. are only examples, and other layouts, sizes, and arrangements may be adopted. In addition, it is clear that a person skilled in the art could come up with various modified or revised examples within the scope of the technical ideas disclosed in this application, and it is understood that these also naturally fall within the technical scope of the present invention.

1: Tubular structure survey support system 2: Photography system 21: On-site PC
22: Photographing device 3: Storage 4: Network 5: Tubular structure survey support device (computer)
51: Control unit 511: Image acquisition unit 512: Exploded image generation unit 513: Display processing unit 514: Pipe structure input unit 515: Damage input unit 516: Preview display unit 7: Main screen 71: Exploded image (partial range)
72: Direct view image; 73: Whole image; 74: Expansion image handle (first position designation means);
75: Play button 76: Whole image handle (second position designation means)
79: Scale indicating position information 81 to 85: Function button 86: Display size change field 88: Grid display/non-display switching operation section 92: Preview button 971, 972: Linear object (joint section)
110: Dot mark 120: Box mark 111: Damage information input field 15: Preview screen 10: Tubular structure (pipe)

Claims (13)

an image acquisition means for acquiring a direct-view image, which is an image captured by a wide-angle camera while moving along the inside of a tubular structure, and a development image generated based on the direct-view image;
a display means for displaying the direct-view image acquired by the image acquisition means, a partial range of the unfolded image, and a whole image which is the unfolded image covering the entire pipe, side by side;
a playback instruction input means for inputting an instruction operation for playing back the direct-view image as a moving image;
Equipped with
The direct-view image is recorded as a video;
A development image handle provided on the development image and an entire image handle provided on the entire image,
When a playback instruction for the direct-viewed image is input by the playback instruction input means, the display means plays back the direct-viewed image as a moving image, moves the display range of the unfolded image in conjunction with the direct-viewed image being displayed, moves the unfolded image handle in conjunction with the direct-viewed image being displayed, and moves the whole image handle in conjunction with the direct-viewed image being displayed, thereby indicating the position of the direct-viewed image being displayed in the pipeline on the whole image and the unfolded image.
A tubular structure inspection support device characterized by: When the developed image handle is moved by the user,
the display means displays the developed image while moving a display range so that the position indicated by the developed image handle is included in a display area of the developed image, displays the direct-view image photographed at the position indicated by the developed image handle , and moves the whole image handle so as to indicate a position corresponding to the position indicated by the developed image handle and displays it on the whole image;
When the whole image handle is moved by the user,
The tubular structure inspection support device according to claim 1, characterized in that the display means changes the display range of the unfolded image so as to include the position indicated by the overall image handle , displays the direct-view image taken at the position indicated by the overall image handle , and moves the unfolded image handle to indicate a position corresponding to the position indicated by the overall image handle and displays it on the unfolded image . a damage input means for displaying a damage input screen for inputting damage information when an arbitrary position on the development image is specified;
a damage recording means for recording the damage information input by the damage input means in association with information on the designated position;
3. The tubular structure inspection support device according to claim 1 , further comprising: 4. A tubular structure inspection support device according to claim 3 , wherein said damage input means allows said position to be designated within a range. 5. The tubular structure inspection support device according to claim 3 , further comprising a preview display means for reading out the damage information recorded in the damage recording means and displaying it as a list together with the entire image. The display device further includes an enlargement operation input unit for inputting an operation for enlarging and displaying the direct-view image,
6. The tubular structure inspection support device according to claim 1 , wherein when an operation is performed by said enlargement operation input means, said display means displays said direct-view image in an enlarged manner. 7. The tubular structure inspection support device according to claim 1 , further comprising a grid display switching means for displaying/hiding a grid on the unfolded image. An imaging device that photographs the inside of a tubular structure with a wide-angle camera while moving along the pipe;
an unfolded image generating device that generates an unfolded image of the tubular structure based on a direct-view image that is an image captured by the imaging device;
a tubular structure inspection support device that is communicatively connected to the photographing device and the unfolded image generating device,
The tubular structure inspection support device comprises:
an image acquisition means for acquiring the direct-view image and the developed image;
a display means for displaying the direct-view image acquired by the image acquisition means, a partial range of the unfolded image, and a whole image which is the unfolded image covering the entire pipe, side by side;
a playback instruction input means for inputting an instruction operation for playing back the direct-view image as a moving image;
Equipped with
The direct-view image is recorded as a video;
A development image handle provided on the development image and an entire image handle provided on the entire image,
When a playback instruction for the direct-viewed image is input by the playback instruction input means, the display means plays back the direct-viewed image as a moving image, moves the display range of the unfolded image in conjunction with the direct-viewed image being displayed, moves the unfolded image handle in conjunction with the direct-viewed image being displayed, and moves the whole image handle in conjunction with the direct-viewed image being displayed, thereby indicating the position of the direct-viewed image being displayed in the pipeline on the whole image and the unfolded image.
A tubular structure inspection support system comprising: When the developed image handle is moved by the user,
the display means displays the developed image while moving a display range so that the position indicated by the developed image handle is included in a display area of the developed image, displays the direct-view image photographed at the position indicated by the developed image handle , and moves the whole image handle so as to indicate a position corresponding to the position indicated by the developed image handle and displays it on the whole image;
When the whole image handle is moved by the user,
The tubular structure inspection support system described in claim 8, characterized in that the display means changes the display range of the unfolded image so as to include the position indicated by the overall image handle , displays the direct-view image taken at the position indicated by the overall image handle , and moves the unfolded image handle to indicate a position corresponding to the position indicated by the overall image handle and displays it on the unfolded image. 1. A computer -implemented method for displaying images, comprising the steps of :
A step of acquiring a direct-view image, which is an image captured by a wide-angle camera while traveling along the inside of a tubular structure, and an unfolded image generated based on the direct-view image;
displaying the acquired direct-view image, a portion of the expanded image, and a full image, which is the expanded image covering the entire tube, side by side;
inputting an instruction operation for playing back the direct-view image as a video;
Including,
The direct-view image is recorded as a video;
A development image handle provided on the development image and an entire image handle provided on the entire image,
The display step plays back the direct-viewed image as a moving image when a playback instruction for the direct-viewed image is input, moves the display range of the unfolded image in conjunction with the direct-viewed image being displayed, and moves the unfolded image handle in conjunction with the direct-viewed image being displayed, and moves the whole image handle in conjunction with the direct-viewed image being displayed, thereby indicating the position of the direct-viewed image being played back in the pipeline on the whole image and the unfolded image.
A method for displaying an image of a tubular structure comprising the steps of: When the developed image handle is moved by the user,
The display step includes displaying the expanded image while moving a display range so that the position indicated by the expanded image handle is included in a display area of the expanded image, displaying the direct-view image captured at the position indicated by the expanded image handle , and moving the whole image handle so as to indicate a position corresponding to the position indicated by the expanded image handle and displaying it on the whole image;
When the whole image handle is moved by the user,
The image display method for a tubular structure as described in claim 10, characterized in that the display step changes the display range of the unfolded image so as to include the position indicated by the overall image handle , displays the direct-view image taken at the position indicated by the overall image handle , and moves the unfolded image handle to indicate a position corresponding to the position indicated by the overall image handle and displays it on the unfolded image. Computer,
an image acquisition means for acquiring a direct-view image, which is an image captured by a wide-angle camera while proceeding along the inside of the tubular structure, and a development image generated based on the direct-view image;
a display means for displaying the direct-view image acquired by the image acquisition means, a partial range of the developed image, and a whole image which is the developed image covering the entire pipe, side by side;
A reproduction instruction input means for inputting an instruction operation for reproducing the direct-view image as a moving image.
A program for causing the device to function as a
The direct-view image is recorded as a video;
A development image handle provided on the development image and an entire image handle provided on the entire image,
When a playback instruction for the direct-viewed image is input by the playback instruction input means, the display means plays back the direct-viewed image as a moving image, moves the display range of the unfolded image in conjunction with the direct-viewed image being displayed, moves the unfolded image handle in conjunction with the direct-viewed image being displayed, and moves the whole image handle in conjunction with the direct-viewed image being displayed, thereby indicating the position of the direct-viewed image being displayed in the pipeline on the whole image and the unfolded image.
A program characterized by: When the developed image handle is moved by the user,
the display means displays the developed image while moving a display range so that the position indicated by the developed image handle is included in a display area of the developed image, displays the direct-view image photographed at the position indicated by the developed image handle , and moves the whole image handle so as to indicate a position corresponding to the position indicated by the developed image handle and displays it on the whole image;
When the whole image handle is moved by the user,
The program according to claim 12, characterized in that the display means changes the display range of the expanded image so as to include the position indicated by the overall image handle, displays the direct-view image taken at the position indicated by the overall image handle , and moves the expanded image handle so as to indicate a position corresponding to the position indicated by the overall image handle and displays it on the expanded image.