JP7154172B2 - Incidental equipment condition evaluation device, incidental equipment condition evaluation method, and incidental equipment condition evaluation program - Google Patents

Description

The present invention relates to an incidental facility condition evaluation device, an incidental equipment condition evaluation method, and an incidental equipment condition evaluation program for evaluating the condition of incidental equipment that is equipment incidental to a structure.

Conventionally, ancillary facilities such as lighting equipment or signboards attached to structures such as tunnels or bridges are regularly inspected. In the inspection of incidental facilities, work is performed to check whether there is rust or looseness in bolts or screws for attaching the incidental facilities to the structure, or whether the incidental facilities themselves have problems such as cracks. Such work requires a lot of labor and time because the worker checks all the ancillary equipment attached to the structure one by one, either visually or by hammering.

Patent Literature 1 discloses a technique for evaluating the soundness of bolts for attaching incidental equipment to a structure. This technique evaluates the soundness of bolts without relying on a worker's ability to distinguish sounds. be able to.

JP 2017-90057 A

However, in the technique described in Patent Document 1, it is necessary to strike each bolt for attaching the incidental equipment to the structure in the same way as in the case of distinguishing sounds by the worker, and efficient inspection is required. difficult to do.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an incidental facility condition evaluation apparatus capable of efficiently assisting inspection of incidental facility.

In order to solve the above-described problems and achieve the object, an incidental facility condition evaluation device of the present invention includes an information acquisition section, an incidental facility image extraction section, a state evaluation section, and a display information generation section. The information acquisition unit acquires information of captured images including an image of incidental equipment of the structure and an image of the structure. The incidental facility image extracting section extracts information of the image of the incidental facility from the information of the captured image acquired by the information acquiring section. The state evaluation unit determines the deformation level of each of the plurality of mesh-shaped regions of the incidental facility based on the image information of the incidental facility extracted by the incidental facility image extraction unit. The display information generation unit generates display information in which deformation level information of each of the plurality of areas is associated with image information of the incidental equipment.

ADVANTAGE OF THE INVENTION According to this invention, there exists an effect that the inspection of incidental equipment can be assisted efficiently.

FIG. 1 is a diagram showing a configuration example of an incidental equipment state evaluation device according to a first embodiment of the present invention; FIG. 4 is a diagram for explaining a processing method by the incidental equipment state evaluation device according to the first embodiment; FIG. FIG. 2 is a diagram showing a specific configuration example of the incidental equipment state evaluation device according to the first embodiment; FIG. 4 is a diagram showing an example of a captured image information table according to the first embodiment; A diagram showing an example of a related information table according to the first embodiment FIG. 4 is a diagram showing an example of a structure information acquisition method using the traveling measuring device according to the first embodiment; FIG. 4 is a diagram showing the relationship between the traveling position of the traveling measuring device according to the first embodiment and each partial captured image; A diagram showing an example of an incidental facility information table according to the first embodiment A diagram showing an example of an image of incidental equipment divided into a plurality of areas according to the first embodiment FIG. 4 is a diagram showing an example of an image of incidental equipment including an abnormal location determined by an abnormality determination unit according to the first embodiment; FIG. FIG. 4 is a diagram showing an example of the relationship between an abnormal location determined by the abnormality determination unit according to the first embodiment and a plurality of areas used by the level determination unit; FIG. 4 is a diagram showing an example of a display image displayed on the display unit by the display processing unit based on the display information according to the first embodiment; FIG. Flowchart showing an example of processing of the control unit of the incidental equipment state evaluation device according to the first embodiment 1 is a diagram showing an example of a hardware configuration of an incidental equipment state evaluation device according to a first embodiment; FIG.

An incidental equipment state evaluation device, an incidental equipment state evaluation method, and an incidental equipment state evaluation program according to embodiments of the present invention will be described below in detail with reference to the drawings. In addition, this invention is not limited by this embodiment.

Embodiment 1.
FIG. 1 is a diagram showing a configuration example of an incidental equipment state evaluation device according to a first embodiment of the present invention. As shown in FIG. 1 , the incidental equipment state evaluation device 1 according to the first embodiment includes a storage unit 10 and a control unit 20 .

The storage unit 10 stores captured image information, which is information of a captured image obtained by capturing an image of a structure and ancillary equipment that is equipment incidental to the structure. A structure whose image is included in the captured image is, for example, a structure such as a tunnel, a bridge, a road, or a sewage system. Incidental equipment whose image is included in the captured image is, for example, a lighting device, a blower, or a signboard. Hereinafter, a tunnel will be described as an example of a structure, but the structure is not limited to a tunnel, and may be a bridge, a sewage system, or other structures.

The control unit 20 includes an information acquisition unit 22 , an incidental facility image extraction unit 23 , a state evaluation unit 25 , a display information generation unit 26 and a display processing unit 27 . The information acquisition unit 22 acquires captured image information generated by the traveling measuring device 2 from the traveling measuring device 2 and stores the acquired captured image information in the storage unit 10 . Further, the information acquisition unit 22 acquires captured image information stored in the storage unit 10 from the storage unit 10 .

The traveling measuring device 2 has an imaging unit, and generates captured image information including information of a captured image obtained by the imaging unit while traveling. For example, when the structure is a tunnel, the traveling measuring device 2 generates captured image information from captured image information obtained by capturing an image of the inner wall of the tunnel while traveling in the tunnel.

The incidental facility image extraction unit 23 extracts information of the image of the incidental facility from the information of the captured image acquired by the information acquisition unit 22 . The state evaluation unit 25 determines the deformation level of each of a plurality of mesh-like regions of the incidental facility based on the image information of the incidental facility extracted by the incidental facility image extraction unit 23 .

The display information generation unit 26 generates display information in which the information of the deformation level of each area determined by the state evaluation unit 25 is associated with the information of the image of the incidental equipment. Based on the display information generated by the display information generating unit 26, the display processing unit 27 causes the display unit 3 to display an image in which the deformation level information of each region is associated with the image information of the incidental equipment.

FIG. 2 is a diagram for explaining a processing method by the incidental equipment state evaluation device according to the first embodiment. In the example shown in FIG. 2, lighting equipment is shown as incidental equipment, but incidental equipment may include an air blower, a signboard, or the like.

In step S<b>1 shown in FIG. 2 , the information acquisition unit 22 acquires information of the captured image generated by the traveling measuring device 2 . Then, in step S<b>2 shown in FIG. 2 , the incidental facility image extraction unit 23 extracts the information of the image of the incidental facility from the information of the captured image. In the example shown in FIG. 2, the captured image includes an image of the inner wall of the tunnel as well as images of four lighting devices as images of incidental equipment. information is extracted from the information of the captured image.

Next, in step S3 shown in FIG. 2, the state evaluation unit 25 performs state evaluation processing for evaluating the state of the incidental equipment based on the image information of the incidental equipment extracted in step S2. Specifically, based on the image information of the incidental facility extracted by the incidental facility image extracting section 23, the state evaluation section 25 determines the change of each of the plurality of mesh-shaped regions R1 to R9 in the incidental facility. level.

In the example shown in FIG. 2, the deformation level of the region R1 is determined to be "4", the deformation levels of the regions R7 and R9 are determined to be "3", and the deformation level of the region R3 is determined to be "2". , the deformation level of the region R8 is determined to be "1". For example, in the state evaluation process, the deformation level is determined based on the state of looseness or rust of fixtures for fixing the lighting equipment to the tunnel, and the state of cracks or breakage occurring in the lighting equipment. Fixtures include, for example, fasteners such as screws or bolts, as well as mounting hardware.

Next, in step S4 shown in FIG. 2, the display information generation unit 26 generates display information in which the deformation level information of each of the plurality of areas is associated with the image information of the incidental equipment. The display processing section 27 causes the display section 3 to display an image based on the display information generated by the display information generating section 26 . As a result, as shown in FIG. 2, the display unit 3 displays an image in which information on the deformation level of each of the plurality of areas is associated with the image of the incidental equipment. In the example shown in FIG. 2, the information on the deformation level of each of the plurality of areas is superimposed on the image of the lighting equipment, which is incidental equipment.

As described above, the incidental facility state evaluation apparatus 1 according to the first embodiment extracts the information of the image of the incidental facility from the information of the captured image including the image of the structure and the image of the incidental facility, and determines the state of the incidental facility. do. Therefore, the incidental equipment can be efficiently inspected without the inspector directly touching the incidental equipment.

In addition, the incidental facility condition evaluation device 1 displays the deformation level of each of the plurality of mesh-shaped regions R1 to R9 in the incidental facility in association with the image information of the incidental facility. Therefore, the inspector of the ancillary equipment can easily grasp which part of the ancillary equipment has a problem, and this also allows the inspector to efficiently inspect the ancillary equipment.

Hereinafter, the configuration and operation of the incidental equipment state evaluation device 1 according to the first embodiment will be described more specifically. FIG. 3 is a diagram showing a specific configuration example of the incidental equipment state evaluation device according to the first embodiment.

As shown in FIG. 3, the incidental facility condition evaluation device 1 according to the first embodiment is communicably connected to the traveling measuring device 2 via the network 5, and the structure information is received from the traveling measuring device 2. to get Such structure information includes information on captured images obtained by capturing images of structures and incidental facilities, and related information on captured images. The network 5 is, for example, a network such as a LAN (Local Area Network) or a WAN (Wide Area Network).

As shown in FIG. 3 , the incidental equipment condition evaluation device 1 according to the first embodiment includes a storage unit 10 , a control unit 20 and a communication unit 30 . Communication unit 30 is connected to network 5 . The communication unit 30 receives information transmitted from the traveling measuring device 2 to the incidental equipment state evaluation device 1 via the network 5 and outputs the received information to the control unit 20 . The control unit 20 can also transmit display information from the communication unit 30 to a device (not shown) connected to the network 5 .

The traveling measuring device 2 records structure information on a portable recording medium such as a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc), or a USB (Universal Serial Bus) memory. You can also The incidental facility state evaluation device 1 can also acquire structure information by reading the structure information from a portable recording medium in which the structure information is recorded.

The storage unit 10 has a captured image information storage unit 11 , a related information storage unit 12 , and an incidental equipment information storage unit 13 . The captured image information storage unit 11 stores a captured image information table 41 . 4 is a diagram illustrating an example of a captured image information table according to the first embodiment; FIG.

The captured image information table 41 shown in FIG. 4 includes a plurality of pieces of information including "structure ID (Identifier)", "image ID", "imaging time", and "image information". "Structure ID" is identification information of a structure. “Image ID” is identification information of the partial captured image obtained by the traveling measuring device 2 . The information of the partial captured image is information of a part of the information of the captured image. The “imaging time” is the time when the partially captured image was obtained by the traveling measuring device 2 . “Image information” is information of a partially captured image obtained by the traveling measuring device 2 .

In the example shown in FIG. 4, the image information of the structure with the structure ID "C001" is shown. Specifically, in the example shown in FIG. 4, the image information of the partially captured image with the image ID “I001” is “I001.bmp”, and the capturing times of the partially captured image are “t _d1 to t _d2 ”. . Further, in the example shown in FIG. 4, the image information of the partially captured image with the image ID "I002" is "I002.bmp", and the capturing time of this partially captured image is "t _d2 to t _d3 ".

Further, in the example shown in FIG. 4, the image information of the partially captured image with the image ID "I003" is "I003.bmp", and the capturing time of this partially captured image is "t _d3 to t _d4 ". Further, in the example shown in FIG. 4, the image information of the partially captured image with the image ID "I004" is "I004.bmp", and the capturing times of this partially captured image are "t _d4 to t _d5 ".

The related information storage unit 12 stores a related information table 42 . 5 is a diagram of an example of a related information table according to the first embodiment; FIG. The related information table 42 shown in FIG. 5 includes a plurality of pieces of information including "structure ID", "imaging location", "imaging sequence", "time", and "positional information". "Structure ID" is identification information of a structure. The “imaging location” is information indicating the imaging location by the traveling measuring device 2 and indicating the name of the structure. In the example shown in FIG. 5, the "imaging location" is tunnel A, road B, or the like.

The “imaging order” is information indicating the order of imaging by the traveling measuring device 2 . The example shown in FIG. 5 indicates that the imaging order at the imaging location “tunnel A” is from east to west, and the imaging order at the imaging location “road B” is from south to north. .

"Time" is information indicating time. "Position information" is information indicating the traveling position of the traveling measuring device 2, for example, information indicating latitude, longitude, and altitude. Note that the "position information" is not limited to information indicating an absolute position, and may be information indicating a relative position with respect to a reference position. In the example shown in FIG. 5 , the traveling position of the traveling measuring device 2 is “x ₁ , y ₁ , z ₁ ” at the time “t ₁ ”, and the traveling position of the traveling measuring device 2 is at the time “t ₂ ”. It is “x ₂ , y ₂ , z ₂ ”. Further, in the example shown in FIG. 5 , the traveling position of the traveling measuring device 2 is “x ₈ , y ₈ , z ₈ ” at the time “t ₈ ”, and the traveling position of the traveling measuring device 2 is at the time “t ₁₅ ”. The position is " _x15 , _y15 , _z15 ".

The information contained in the captured image information table 41 and the related information table 42 described above is notified from the traveling measuring device 2 to the incidental equipment state evaluating device 1 . FIG. 6 is a diagram illustrating an example of a structure information acquisition method using the traveling measuring device according to the first embodiment.

As shown in FIG. 6 , the traveling measuring device 2 is equipped with an imaging unit 51 . The traveling measuring device 2 repeatedly captures images of the surroundings of the traveling measuring device 2 with the imaging unit 51 while traveling in the structure 6, which is a tunnel, and includes an image of the inner wall of the structure 6 and an image of the incidental equipment 7. Captured image information including a plurality of partial captured images is generated. The imaging unit 51 is, for example, a line camera, and has a plurality of pixels arranged along a direction perpendicular to or intersecting the traveling direction of the traveling measuring device 2 . The traveling measuring device 2 generates captured image information including a plurality of partial captured images based on pixel information repeatedly output from the line camera. Note that the imaging unit 51 may be an area camera in which a plurality of pixels are arranged in a matrix.

FIG. 7 is a diagram showing the relationship between the traveling position of the traveling measuring device and each partial captured image according to the first embodiment. As shown in FIG. 7, the traveling measuring device 2 generates related information that associates the traveling position of the traveling measuring device 2 with the time.

In the example shown in FIG. 7 , for example, the traveling position of the traveling measuring device 2 is “x ₁ , y ₁ , z ₁ ” at the time “t ₁ ”, and the traveling position of the traveling measuring device 2 is at the time “t ₂ ”. _The position is "x2, _y2 , _z2 ". Further, in the example shown in FIG. 7 , the traveling position of the traveling measuring device 2 is “x ₈ , y ₈ , z ₈ ” at the time “t ₈ ”, and the traveling position of the traveling measuring device 2 is at the time “t ₁₅ ”. The position is " _x15 , _y15 , _z15 ".

Further, in the example shown in FIG. 7 , the traveling position of the traveling measuring device 2 is “x ₂₂ , y ₂₂ , z ₂₂ ” at the time “t ₂₂ ”, and the traveling position of the traveling measuring device 2 is at the time “t ₂₉ ”. The position is " _x29 , _y29 , _z29 ". 7 includes images of incidental facilities 7 ₁ , _{7 2} , _{7 3} , and 7 ₄ as images of incidental facilities ₇ . , ₇₄ are included in different partial captured images.

The incidental equipment information storage unit 13 stores an incidental equipment information table 43 . 8 is a diagram of an example of an incidental facility information table according to the first embodiment; FIG. The incidental equipment information table 43 shown in FIG. and multiple pieces of information including "deformation rate".

“Structure ID” is identification information of the structure 6 . The “incidental equipment ID” is identification information of the incidental equipment 7 . The "equipment type" is information indicating the type of the incidental equipment 7, such as a lighting device, an air blower, or a signboard. “Position information” is information indicating the position of the incidental equipment 7 . “Image information” is image information of the incidental equipment 7 .

The “deformation level” is information indicating the deformation level of each of a plurality of mesh-like partitioned areas in the incidental facility 7 . In the example shown in FIG. 8, the plurality of mesh-shaped regions in the incidental equipment 7 are regions R1, R2, . . . , Rm. m is an integer equal to or greater than 3, and in this embodiment, m=9. . . , Rm may be referred to as a region R hereinafter when the regions R1, R2, . . .

The "total evaluation value" is information indicating the degree of deformation of the 7 units of incidental equipment. The “deformation rate” is information indicating the rate of deformation in the incidental facility 7 as a whole. The overall evaluation value and the deformation ratio are calculated by the control unit 20 based on the deformation levels of the regions R1, R2, . . . , Rm.

In the example shown in FIG. 8, information about the incidental equipment 7 of the structure 6 with the structure ID "C001" is shown. Specifically, in the example shown in FIG. 8 , the incidental facility 7 having the incidental facility ID “F001” has the facility type of “lighting equipment” and the position information of “x _f1 , y _f1 , z _f1 ”. The image information is "F001.bmp", and the deformation levels of the regions R1, R2, . . . , Rm are "4, 0, . The incidental facility 7 having the incidental facility ID "F002" has the facility type of "lighting equipment", the position information of "x _f2 , y _f2 , z _f2 ", and the image information of "F002.bmp". , regions R1, R2, . . . , Rm are “2, 0, .

In the example shown in FIG. 8 , the incidental facility 7 with the incidental facility ID “F001” has a comprehensive evaluation value of “4”, a deformation rate of “0.33”, and an incidental facility ID “F002”. The incidental equipment 7 has a comprehensive evaluation value of "2" and a deformation ratio of "0.11".

Returning to FIG. 3, the controller 20 will be described. The control unit 20 of the incidental facility state evaluation device 1 includes an input reception unit 21, an information acquisition unit 22, an incidental facility image extraction unit 23, a position specifying unit 24, a state evaluation unit 25, and a display information generation unit 26. , and a display processing unit 27 .

The input reception unit 21 receives user operations input to the input unit 4 . The input unit 4 is, for example, a mouse or keyboard. The user operation is an operation input to the input unit 4 by the user, and includes, for example, a mouse move operation and a mouse click operation. Also, the input unit 4 may be a touch panel arranged on the display unit 3 . Hereinafter, user operations input to the input unit 4 may be simply referred to as user operations.

The information acquisition unit 22 acquires captured image information from the traveling measuring device 2 via the network 5 and the communication unit 30 and adds the acquired captured image information to the captured image information storage unit 11 . The information acquisition unit 22 also acquires related information from the traveling measuring device 2 via the network 5 and the communication unit 30 and stores the acquired related information in the related information storage unit 12 . Note that the information acquisition unit 22 can also acquire captured image information and related information by reading structure information from a portable recording medium recording structure information including captured image information and related information. Note that the captured image information and the related information may be recorded in different portable recording media.

Further, the information acquisition unit 22 acquires the captured image information and the related information image from the storage unit 10 when the user operation received by the input reception unit 21 is a request to display incidental equipment state information. The information acquisition unit 22 notifies the incidental equipment image extraction unit 23 and the position identification unit 24 of the captured image information and the captured image information acquired from the storage unit 10 .

The incidental equipment image extracting unit 23 extracts image information of a plurality of incidental equipment 7 from the captured image information included in the captured image information acquired by the information acquiring unit 22 . The incidental equipment image extracting unit 23 has a learned model generated by inputting an image of the incidental equipment 7 as a detection target through a prior machine learning process, for example. The incidental equipment image extraction unit 23 inputs the information of the captured image acquired by the information acquisition unit 22 to the learned model, thereby extracting the information of the image of the range in which the incidental equipment 7 is shown.

The incidental equipment image extraction unit 23 has a learned model for each type of incidental equipment 7, for example. The incidental facility image extraction unit 23 inputs the information of the captured image acquired by the information acquisition unit 22 to each trained model, and outputs a score of a type corresponding to the trained model with the highest score above the threshold. It can be determined that the incidental equipment 7 is included in the captured image.

Note that the incidental facility image extraction unit 23 can also extract the image of the incidental facility 7 by pattern matching processing instead of the learned model. For example, the incidental equipment image extracting unit 23 has information of a reference image in the type of the incidental equipment 7, and the information of the captured image acquired by the information acquiring unit 22 includes an image whose degree of similarity with the reference image is equal to or greater than a threshold. Determine whether information is included. When the incidental equipment image extraction unit 23 determines that the information of the captured image acquired by the information acquisition unit 22 includes the information of the image whose similarity to the reference image is equal to or greater than the threshold, the similarity to the reference image reaches the threshold. The image information described above is extracted as the image of the incidental equipment 7 .

The position specifying unit 24 specifies the position of the incidental equipment 7 whose image information is extracted by the incidental equipment image extracting unit 23 . For example, the position specifying unit 24 specifies the position of the incidental facility 7 based on the information of the captured image from which the information of the image of the incidental facility 7 is extracted by the incidental facility image extracting unit 23 and the related information of the captured image. be able to.

The position specifying unit 24 can specify the position of the incidental equipment 7 by executing the position specifying process specified by the user's operation out of the first position specifying process and the second position specifying process. The first position specifying process is a process of assigning identification information as information indicating the position of each incidental facility 7 to each incidental facility 7 according to the order of imaging by the imaging unit 51 of the traveling measuring device 2 .

First, the first position specifying process will be described. The first position specifying process is a process of specifying the position of the incidental facility 7 based on the time information associated with the information of the image of the incidental facility 7 by the incidental facility image extraction unit 23 .

In the first position specifying process, the position specifying unit 24 assigns a plurality of incidental equipment IDs that are continuous in forward or reverse order of time associated with the image information of the incidental equipment 7 by the incidental equipment image extracting unit 23 to the structure 6. Allocate to a plurality of incidental facilities 7. The multiple incidental facility IDs consecutive in the forward order of time are, for example, multiple incidental facility IDs assigned such that the number included increases as the time is newer. Also, the multiple incidental equipment IDs consecutive in reverse order of time are, for example, multiple incidental equipment IDs assigned such that the number included decreases as the time becomes newer.

Here, the captured image information table 41 is in the state shown in FIG. 4, the captured image information is in the state shown in FIG. Assume that the facility ID is assigned to a plurality of incidental facilities 7 attached to the structure 6 . In this case, the position specifying unit 24 assigns the incidental facility ID "F001" to the information of the image of the incidental facility _7-1 , and assigns the incidental facility ID " _F002 " to the information of the image of the incidental facility 7-2. Further, the position specifying unit 24 assigns the incidental facility ID “ _F003 ” to the information of the image of the incidental facility 73 and assigns the incidental facility ID “ _F004 ” to the information of the image of the incidental facility 74 .

When the information of one partial captured image includes a plurality of incidental facilities 7, the position specifying unit 24 continues the partial captured image in ascending order of the _Xd coordinate positions in the _XdYd coordinate system shown in FIG _. A plurality of incidental facility IDs can be assigned to a plurality of incidental facilities 7 incidental to the structure 6 . For example, it is assumed that the partially captured image with the image ID “I004” includes information on the images of two incidental facilities 7 . In this case, the position specifying unit 24 assigns the incidental facility ID "I004" to the image of the incidental facility 7 closer to the negative direction of the _Xd axis among the two images of the incidental facility 7, and The incidental facility ID “I005” is assigned to the image of the incidental facility 7 closer to the _d -axis positive direction.

Further, the captured image information table 41 is in the state shown in FIG. 4, the captured image information is in the state shown in FIG. Assume that an ID is assigned to a plurality of incidental facilities 7 attached to the structure 6 . In this case, the position specifying unit 24 assigns the incidental facility ID "F001" to the image information of the incidental facility 74 _, and assigns the incidental facility ID " _F002 " to the information of the incidental facility 73 image. Further, the position specifying unit 24 assigns the incidental facility ID “F003” to the information of the image of the incidental facility ₇₂ and assigns the incidental facility ID “ _F004 ” to the information of the image of the incidental facility 71 .

If the information of one partial captured image includes a plurality of incidental facilities 7, for example, the position specifying unit 24 selects the partial captured images in descending order of the _Xd coordinate positions in the _XdYd coordinate system shown in FIG _. A plurality of continuous incidental facility IDs are assigned to a plurality of incidental facilities 7 incidental to the structure 6 . For example, it is assumed that the partially captured image with the image ID “I001” includes information on the images of two incidental facilities 7 . In this case, the position specifying unit 24 assigns the incidental facility ID "I004" to the image of the incidental facility 7 closer to the positive direction of the _Xd axis among the two images of the incidental facility 7, and The incidental facility ID "I005" is assigned to the image of the incidental facility 7 closer to the _d -axis negative direction.

Note that the position specifying unit 24 assigns the incidental equipment ID to the information of the image of the incidental equipment 7 based on the serial number information included in the file name of the partially captured image instead of the information of the acquisition time of the partially captured image. can also For example, in the captured image information table 41 shown in FIG. 4, the file names of a plurality of partial captured images include "001", "002", "003", and "004" as serial number information. The position specifying unit 24 can assign a plurality of incidental equipment IDs consecutive in ascending order or descending order of the serial number information to a plurality of incidental equipment 7 .

Next, the second position specifying process will be described. In the second position specifying process, based on the traveling position and traveling time of the traveling measuring device 2 and the imaging time of the captured image from which the information of the image of the incidental facility 7 is extracted by the incidental facility image extraction unit 23, the incidental facility 7 is a process for identifying the position of .

The position specifying unit 24 acquires from the captured image information storage unit 11 the information of the capturing time of the captured image from which the information of the image of the incidental equipment 7 is extracted by the incidental equipment image extracting unit 23 . Further, the position specifying unit 24 acquires from the related information storage unit 12 information on the traveling position of the traveling measuring device 2 corresponding to the information on the imaging time acquired from the captured image information storage unit 11 . The position specifying unit 24 determines the position of the image of the incidental equipment 7 based on the acquired information about the traveling position of the traveling measuring device 2 .

For example, assume that the captured image information table 41 is in the state shown in FIG. 4 and the related information table 42 is in the state shown in FIG. It is also assumed that the relationship between the traveling position of the traveling measuring device ₂ and each partial captured image is as shown in FIG. In this case, the imaging time of the partial captured image from which the information of the image of the incidental facility ₇₁ is extracted by the incidental facility image extraction unit 23 is in the range of time t _d1 to t _d2 . Further, the traveling position of the traveling measuring device 2 in the range of times t _d1 to t _d2 is in the range of “x ₁ , y ₁ , z ₁ ” to “x ₈ , y ₈ , z ₈ ”.

_The position specifying unit 24 determines "x ₁ , y ₁ , z ₁ ” to “x ₈ , y ₈ , z ₈ ”, the range corresponding to the incidental facility 7 ₁ is calculated as the position of the incidental facility 7 ₁ . The position specifying unit 24 associates the calculated information of the position of the incidental facility _7-1 with the information of the image of the incidental facility _7-1 and adds it to the incidental facility information table 43 of the incidental facility information storage unit 13 .

Further, the position specifying unit 24 _determines "x ₁ , y ₁ , z ₁ ” to “x ₈ , y ₈ , z ₈ ”, the position corresponding to the central position of the incidental facility 7 ₁ can be calculated as the position of the incidental facility 7 ₁ .

In this case, the position specifying unit 24 determines the time corresponding to the central position of the incidental equipment 7 ₁ among the times t _d1 to t _d2 as the central position time. The position specifying unit 24 can determine the traveling position associated with the time closest to the central position time among the times t ₁ to t ₈ as the position of the incidental facility 7 ₁ . In addition, the position specifying unit 24 linearly interpolates two traveling positions associated with two times close to the central position time out of the times t ₁ to t ₈ by proportional allocation to determine the traveling position corresponding to the central position time. It can be determined as the position of the ancillary equipment _7-1 .

Further, in the example shown in FIG. 4, in the captured image information table 41, the capturing times of the partial captured images indicate a time range, but the capturing time of the partially captured images may be one time. In this case, the position specifying unit 24 specifies the range of _times when the images of the incidental equipment 71 are captured by linearly interpolating the capturing times of the two partial captured images by proportional allocation. The position specifying unit 24 specifies the travel position associated with the time closest to the oldest time in the specified time range and the travel position associated with the time closest to the latest time in the specified time range. _The position of the ancillary equipment 71 can be determined from the range covered.

For example, in FIG. 4, it is assumed that the imaging time of image ID "I001" is "t _d1 " and the imaging time of image ID "I002" is "t _d2 ". In this case, the position specifying unit 24 specifies, as the position of the incidental facility 7 ₁ , the traveling position corresponding to the range of the image of the incidental facility 7 ₁ among the times obtained by linearly interpolating the range of times t _d1 to t _d2 by proportional allocation. be able to. In addition, the position specifying unit 24 specifies the center position time by linearly interpolating the imaging times of the two partial captured images by proportional distribution, and determines the traveling position associated with the time closest to the center position time. It can also be determined as the position of 7 ₁ .

In addition, the incidental equipment state evaluation device 1 can also associate a three-dimensional position with each pixel of the partial captured image and store it in the storage unit 10 . In this case, the storage unit 10 can determine the three-dimensional position of the incidental equipment 7 as the position of the incidental equipment 7 based on the three-dimensional position associated with each pixel of the partial captured image.

The association of the three-dimensional position with each pixel of the partial captured image can be performed by the traveling measuring device 2, for example. For example, the traveling measuring device 2 has a laser scanner device (not shown), measures the surroundings of the traveling measuring device 2 with the laser scanner device while traveling in the structure 6, and measures the surroundings of the structure 6 based on the measurement results. 3D point group information including the 3D point group of 6 and the 3D point group of incidental equipment 7 is generated. Information on the three-dimensional point group includes information on the three-dimensional coordinates of each three-dimensional point. The traveling measuring device 2 can, for example, associate the three-dimensional position with each pixel of the partially captured image based on the position at which each three-dimensional point is measured and the position at which the partially captured image is captured.

In this way, the position specifying unit 24 can determine the position of each incidental facility 7, and associates the determined position information of the incidental facility 7 with the information of the image of the incidental facility 7 to store the incidental facility information storage unit 13. It can be added to the incidental equipment information table 43 .

The state evaluation unit 25 determines the deformation level of each of the plurality of mesh-like regions R of the incidental facility 7 based on the image information of the incidental facility 7 extracted by the incidental facility image extraction unit 23. . The state evaluation unit 25 includes a division unit 31 , a region deformation level determination unit 32 and a comprehensive evaluation unit 33 .

The state evaluation unit 25 has a first determination mode in which the image of the incidental equipment 7 is divided into a plurality of regions R and then determines the deformation level of each region R, and a first determination mode in which the abnormality level of each region R is determined and a second determination mode for determining the deformation level. The first determination mode and the second determination mode will be specifically described below. The determination mode executed by the state evaluation unit 25 is set to either the first determination mode or the second determination mode based on a user's operation or a setting request from the outside.

First, the first determination mode will be explained. When the determination mode is set to the first determination mode, the division unit 31 of the state evaluation unit 25 divides the image of the incidental facility 7 based on the information of the image of the incidental facility 7 extracted by the incidental facility image extraction unit 23. is divided into a plurality of regions R. FIG. 9 is a diagram illustrating an example of an image of incidental equipment divided into a plurality of areas according to the first embodiment; In the example shown in FIG. 9, the image of the incidental equipment 7 is divided into nine regions R1 to R9 in a mesh pattern. Note that the number of regions R is not limited to nine, and may be eight or less, or may be ten or more. Also, in the example shown in FIG. 9, each region R has the same size and shape, but the present invention is not limited to such an example.

A region deformation level determination unit 32 of the state evaluation unit 25 determines the deformation level of the region R based on the information of the image of the region R divided by the division unit 31 . The area change level determination section 32 includes an abnormality determination section 34 and a level determination section 35 .

The abnormality determination unit 34 inputs the image information of each region R divided by the division unit 31 into the evaluation model, and determines whether or not each region R has an abnormality. For example, if the score output from the evaluation model is equal to or greater than the threshold, the area change level determination unit 32 determines that there is abnormality in the area R, and if the score output from the evaluation model is less than the threshold, , region R can be determined to have no abnormality.

An evaluation model is a trained model generated by machine learning. Such an evaluation model is generated by machine learning, for example, using an image indicating that the attachment state of the incidental facility 7 or the state of the incidental facility 7 itself is abnormal. For example, in the evaluation model, an image of a loose mounting fixture that attaches the incidental equipment 7 to the structure 6, an image of a rusted mounting fixture, or an image of the incidental equipment 7 with a crack is displayed as an abnormal condition. Generated by machine learning as an image. The state in which the fixture is loosened is, for example, a state in which the alignment mark or alignment line of the fastener such as a screw or bolt is misaligned with the alignment mark or alignment line of the member to which the fastener is attached. .

The evaluation model may be an evaluation model for each type of incidental equipment 7, an evaluation model for each deformation type, or an evaluation model for each type of incidental equipment 7 and each deformation type. may Thereby, the abnormality determination unit 34 can identify the type of the incidental equipment 7, the type of deformation, or the like. The deformation type includes, for example, types such as a state in which the fixture is loose, a state in which the fixture is rusted, and a state in which the ancillary equipment 7 is cracked. In addition, the evaluation model may be an evaluation model for each region R, an evaluation model for each type of incidental equipment 7 and each region R, and an evaluation model for each type of incidental equipment 7, each deformation type, and An evaluation model for each region R may be used.

The level determination unit 35 analyzes the feature amount of the image of the region R determined to be abnormal by the abnormality determination unit 34, and determines the deformation level of the region R determined to be abnormal by the abnormality determination unit 34. do.

The level determination unit 35 calculates a deformation level based on the feature amount of the image of the region R obtained from the information of the image of the region R determined to be abnormal by the abnormality determination unit 34 . For example, the level determination unit 35 determines one or two of the RGB correlation value, the RGB variance value, the brightness correlation value, the brightness variance value, and the edge component value in the image of the region R determined to be abnormal by the abnormality determination unit 34. A deformation level can be calculated based on one or more combinations.

The deformation level is a positive integer with a normal state without deformation being 0, and a positive integer whose value increases as the degree of deformation is higher. The higher the value, the closer to 0 the value may be. In addition, the deformation level may be a value that allows the degree of deformation to be grasped. For example, a normal state without deformation is set to 0, and the higher the degree of deformation, the smaller the value. good too. Note that the higher the degree of deformation, the higher the level of deformation.

Moreover, when the type of the incidental equipment 7 is specified by the abnormality determining unit 34 , the level determining unit 35 can determine the abnormality level based on different criteria for each type of the incidental equipment 7 . Further, when the abnormality determination unit 34 specifies the type of deformation, the level determination unit 35 can determine the deformation level based on different criteria for each type of deformation.

Further, the level determination unit 35 inputs the image of the region R determined to be abnormal by the abnormality determination unit 34 to the evaluation model generated by machine learning, thereby determining the deformation level of the region R. can also Such an evaluation model is generated by machine learning for each deformation type using, for example, an image showing that the attachment state of the incidental facility 7 or the state of the incidental facility 7 itself is abnormal.

The level determination unit 35 inputs the image information of the region R determined to be abnormal by the abnormality determination unit 34 to the evaluation model for each deformation type. The level determination unit 35 determines the deformation level of the region R determined to be abnormal by the abnormality determination unit 34 based on the score output from the evaluation model for each type of deformation.

For example, the level determination unit 35 determines the deformation level for each deformation type based on the score output from the evaluation model for each deformation type. The level determination unit 35 increases the deformation level as the score is higher. The level determination unit 35 can determine the highest deformation level among the determined deformation levels for each deformation type as the deformation level of the region R.

In addition, when the abnormality determination unit 34 identifies the type of deformation of the region R determined to be abnormal by the abnormality determination unit 34, the level determination unit 35 evaluates the deformation type notified from the abnormality determination unit 34. A model can be used to determine the level of deformation of region R.

The level determination unit 35 can also determine the deformation level of the region R using an evaluation model for each deformation type and each deformation level. The evaluation model may be the same as the evaluation model used by the abnormality determination unit 34, and the level determination unit 35 increases the abnormality level as the score output from the evaluation model is higher.

Next, the second determination mode of the state evaluation section 25 will be described. When the determination mode is set to the second determination mode, the abnormality determination unit 34 of the state evaluation unit 25 inputs the image information of the incidental equipment 7 extracted by the incidental equipment image extraction unit 23 into the evaluation model, The abnormal part of the incidental equipment 7 is determined.

The evaluation model used in the second determination mode is an evaluation model that distinguishes abnormal locations from normal locations in the image of the incidental equipment 7 . Such an evaluation model includes, for example, an evaluation model for each location where an abnormality occurs. An evaluation model for each location where an abnormality occurs is generated by machine learning using an image of a state in which an abnormality has occurred at the corresponding location.

10 is a diagram illustrating an example of an image of incidental equipment including an abnormal portion determined by the abnormality determination unit according to the first embodiment; FIG. The example shown in FIG. 10 indicates that four locations in the image of the incidental equipment 7 are determined as abnormal locations by the abnormality determination unit 34 .

The level determination unit 35 shown in FIG. 3 analyzes the feature amount of the image of the region R including the abnormal portion determined by the abnormality determination unit 34, and changes the area R including the abnormality determined by the abnormality determination unit 34. level. For example, the level determination unit 35 determines one or two of the RGB correlation value, the RGB variance value, the brightness correlation value, the brightness variance value, and the edge component value in the image of the region R including the abnormal location determined by the abnormality determination unit 34. The deformation level of region R can be calculated based on one or more combinations.

Region R is the same as region R divided in the first mode. FIG. 11 is a diagram illustrating an example of a relationship between an abnormal location determined by the abnormality determination unit and multiple regions used by the level determination unit according to the first embodiment; In the example shown in FIG. 11 , regions R1, R3, R8, and R9 include the abnormal locations determined by the abnormality determination unit 34 . In this case, the level determination unit 35 analyzes the feature amount of the image of each region R1, R3, R8, R9 and determines the deformation level of each region R1, R3, R8, R9.

The level determination unit 35 inputs the image of the region R including the abnormal portion determined by the abnormality determination unit 34 to the evaluation model generated by machine learning, and determines the deformation level of the region R. can also Such evaluation model may be the same as or different from the evaluation model used in the first mode.

A neural network, for example, can be used as the evaluation model and the trained model described above. The neural network includes a convolution neural network (CNN), a recurrent neural network (RNN), or the like.

The level determination unit 35 can calculate the average value of the deformation level for each of the plurality of regions R1 to R9 when a plurality of images of the incidental equipment 7 of the same type are included in the captured image. The type of the incidental equipment 7 is, for example, a lighting device, an air blower, a signboard, or the like, but may be the model of the incidental equipment 7 or the like.

A comprehensive evaluation unit 33 of the state evaluation unit 25 determines the degree of deformation of each incidental facility 7 based on the deformation levels of the plurality of regions R determined by the region deformation level determination unit 32 . The comprehensive evaluation unit 33 can associate the determined degree of deformation of the incidental equipment 7 with the information of the image of the incidental equipment 7 and add it to the incidental equipment information table 43 of the incidental equipment information storage unit 13 .

The comprehensive evaluation unit 33 can set the highest deformation level among the deformation levels of the plurality of regions R determined by the region deformation level determination unit 32 as the degree of deformation of the incidental equipment 7 . For example, as shown in FIG. 2, the deformation level of the region R1 is "4", the deformation level of the region R3 is "2", the deformation level of the region R7 is "3", and the deformation level of the region R8 is "4". is "1", and the deformation level of the region R9 is "3". It is also assumed that the deformation level of the region R where there is no abnormality is "0". In this case, the comprehensive evaluation unit 33 sets the degree of deformation of the incidental equipment 7 to "4".

Further, the comprehensive evaluation unit 33 determines a region R whose deformation level is equal to or higher than the threshold level among the plurality of regions R determined by the region deformation level determination unit 32 . Then, the comprehensive evaluation unit 33 can determine the degree of deformation of the incidental facility 7 based on the proportion of the area R having the threshold level or higher in the image of the incidental facility 7 .

For example, assume that the deformation level of each region R is the state shown in FIG. 2 and the threshold level is "3". It is also assumed that the deformation level of the region R where there is no abnormality is zero. In this case, regions R where the deformation level is equal to or higher than the threshold level are regions R1, R7, and R9. Therefore, the comprehensive evaluation unit 33 sets the deformation degree of the incidental equipment 7 to "0.33". In this way, the comprehensive evaluation unit 33 can obtain the degree of deformation of the incidental equipment 7 as an evaluation value different from the five deformation levels.

Further, the comprehensive evaluation unit 33 calculates a value obtained by dividing the sum of the deformation levels of the plurality of regions R determined by the region deformation level determination unit 32 by the sum of the maximum deformation levels. The degree of deformation of incidental equipment 7 can also be used. For example, assume that the deformation level of each region R is as shown in FIG. It is also assumed that the deformation level of the region R where there is no abnormality is zero. In this case, the comprehensive evaluation unit 33 sets the deformation degree of the incidental equipment 7 to 12/45. In addition, the denominator is a value obtained by summing up "5", which is the maximum value of the deformation level of each region R. In this way, the comprehensive evaluation unit 33 can obtain the degree of deformation of the incidental equipment 7 as an evaluation value different from the five deformation levels.

The comprehensive evaluation unit 33 can express the degree of deformation of the incidental equipment 7 as a comprehensive evaluation value, a deformation ratio, or the like. For example, the comprehensive evaluation unit 33 sets the highest deformation level among the deformation levels of the plurality of regions R as a comprehensive evaluation value, and the ratio of the regions R above the threshold level in the image of the incidental equipment 7 or the plurality of regions R A value obtained by dividing the sum of the deformation levels of , by the maximum value can be used as the deformation ratio of the incidental equipment 7 . The deformation ratio is the ratio of deformations in the image of the incidental equipment 7 .

The display information generation unit 26 generates display information based on the information in the incidental equipment information table 43 stored in the incidental equipment information storage unit 13 . For example, the display information generation unit 26 generates display information in which the information of the deformation level of each region R, the information of the comprehensive evaluation level, and the information of the degree of deformation of the incidental equipment 7 are associated with the information of the image of the incidental equipment 7. be able to. Further, the display information generation unit 26 generates display information in which the information indicating the position of the incidental facility 7 determined by the position specifying unit 24 in addition to the information of the deformation level of each region R is associated with the information of the image of the incidental facility 7. can be generated. The display information generation unit 26 can also transmit the generated display information from the control unit 20 to devices connected to the network 5 .

The display processing section 27 displays an image on the display section 3 based on the display information generated by the display information generating section 26 . 12 is a diagram illustrating an example of a display image displayed on the display unit by the display processing unit based on the display information according to the first embodiment; FIG.

A display image 80 shown in FIG. 12 includes a display area 81 for displaying the name of the structure 6, a display area 82 for displaying the measurement direction by the traveling measuring device 2, a display area 83 for displaying the imaging plane, and a display A display area 84 for displaying the name of the target incidental equipment 7 is also included. The imaging surface is a region of the structure 6 that is imaged by the imaging unit 51 .

In the example shown in FIG. 12, the characters "Tunnel A" are displayed in the display area 81 as the name of the structure 6, and the characters "East→West" are displayed in the display area 82 as the measurement direction of the traveling measuring device 2. ing. In the example shown in FIG. 12, the characters “left side” are displayed in the display area 83 and the characters “lighting device” are displayed in the display area 84 .

A display image 80 shown in FIG. 12 includes a button 85 for displaying a list box for selecting one imaging surface from a plurality of imaging surfaces, and a type of incidental equipment 7 to be displayed from among multiple types of incidental equipment 7. includes a button 86 that displays a list box for selecting The user of the incidental facility condition evaluation apparatus 1 can select an imaging surface by operating the button 85 or select the type of the incidental facility 7 to be displayed by operating the button 86 . The control unit 20 extracts image information of the incidental equipment 7 of the type selected by the user operation from the information of the captured image of the imaging surface selected by the user operation, and based on the extracted image information of the incidental equipment 7 Therefore, the deformed state of the incidental equipment 7 can be evaluated by the processing described above.

In the traveling measuring device 2 shown in FIG. 6, only the imaging unit 51 that takes an image of the left side of the plurality of imaging planes is shown. and an imaging unit (not shown) for imaging the right side of the structure 6 are mounted. The control unit 20 of the incidental facility condition evaluation apparatus 1 processes the captured image of the imaging unit (not shown) that captures the upper surface and the captured image of the imaging unit (not shown) that captures the right side in the same manner as the captured image of the imaging unit 51. Then, determination of the deformation level of the region R and determination of the degree of deformation of the incidental equipment 7 can be performed.

The display image 80 shown in FIG. 12 includes an incidental facility state information display area 87 indicating incidental facility state information of the incidental facility 7 . In the incidental equipment status information display area 87, the incidental equipment ID of the incidental equipment 7, information indicating the position of the incidental equipment 7, information indicating the comprehensive evaluation value of the deformation of the incidental equipment 7, and the deformation of the incidental equipment 7 with respect to the image are displayed. Each incidental facility 7 includes information indicating the proportion of the incidental facility 7 and information indicating an image of the incidental facility 7 .

For example, the incidental facility ₇ with the incidental facility _ID " _I001 " shown in FIG. The ratio is "0.33". Further, the incidental facility ₇ with the incidental facility _ID " _I002 " shown in FIG. The ratio is "0.11". As a result, the user of the incidental facility state evaluation apparatus 1 can easily grasp how much the incidental facility 7 at which position of the structure 6 is in a deformed state. Further, in the example shown in FIG. 12, incidental equipment IDs are assigned with serial numbers in the sequence of arrangement of incidental equipment 7, and this also shows how the incidental equipment 7 at which position in the structure 6 is deformed to what degree. you can figure out if there is.

In addition, in the image of the incidental equipment 7, there are lines for dividing the image of the incidental equipment 7 into nine regions R, and a Deformation level is indicated. As a result, the user of the incidental facility condition evaluation apparatus 1 can easily grasp, for example, which area R of the incidental facility 7 is deformed and which area R is likely to break down.

In the image of the incidental equipment 7 shown in FIG. 12, the deformation level of each region R is indicated by characters, but the display information generation unit 26 indicates the deformation level of each region R by colors, symbols, or the like. can also For example, the display information generation unit 26 can generate display information that expresses the deformation level of each region R by the density of the image of the region R or the color of the image of the region R. FIG. Specifically, the display information generator 26 can generate display information such that the lower the deformation level of the region R, the darker it becomes than the original image. This also makes it possible to easily grasp which region R of the incidental equipment 7 is deformed and which region R is likely to break down.

In addition, the display information generation unit 26 generates display information such that regions R with lower deformation levels become lighter than the original image, or generates display information that expresses the deformation level of each region R using colors and characters. You can This also makes it possible to easily grasp which region R of the incidental equipment 7 is deformed and which region R is likely to break down.

In addition, the display information generation unit 26 can generate display information including information on characters, colors, or symbols indicating the average deformation level of each region R determined by the level determination unit 35 . The display processing unit 27 can display on the display unit 3 information of characters, colors, or symbols indicating the average value of the deformation level of each region R. FIG. As a result, the user of the incidental facility condition evaluation apparatus 1 can easily grasp in which region R deformation is likely to occur for each type of incidental facility 7 .

Next, the operation of the incidental facility condition evaluation device 1 will be explained using a flowchart. 13 is a flowchart illustrating an example of processing of the control unit of the incidental equipment state evaluation apparatus according to the first embodiment; FIG. Such processing is started, for example, when the user's operation is a request to display incidental equipment status information.

As shown in FIG. 13, the control unit 20 of the incidental facility condition evaluation device 1 acquires the information of the captured image from the storage unit 10 (step S10), and extracts the information of the image of the incidental facility 7 from the acquired information of the captured image. Extract (step S11). Further, the control unit 20 acquires the information of the captured image and related information from the storage unit 10, and determines the position of the incidental equipment 7 based on the acquired information of the captured image and related information (step S12).

Next, the control unit 20 determines whether or not the first determination mode is set (step S13). When determining that the first determination mode is set (step S13: Yes), the control unit 20 divides the image of the incidental equipment 7 into a plurality of regions R (step S14), and each region R is abnormal. It is determined whether or not (step S15). Then, the control unit 20 determines the deformation level of the abnormal region R (step S16).

If the control unit 20 determines that the first determination mode is not set (step S13: No), that is, if it is determined that the second determination mode is set, the abnormal part of the image of the incidental equipment 7 is determined. (Step S17), and determine the deformation level of the region R including the abnormal portion (Step S18).

When the process of step S16 is finished, or when the process of step S18 is finished, the control unit 20 determines the degree of deformation of the incidental equipment 7 (step S19). The control unit 20 generates display information including incidental equipment state information (step S20), displays the incidental equipment state information on the display unit 3 based on the display information (step S21), and performs the processing shown in FIG. exit.

14 is a diagram illustrating an example of a hardware configuration of the incidental equipment state evaluation apparatus according to the first embodiment; FIG. As shown in FIG. 14 , incidental facility condition evaluation device 1 includes a computer having processor 101 , memory 102 , input/output circuit 103 , and communication device 104 .

The processor 101, the memory 102, the input/output circuit 103, and the communication device 104 can transmit and receive data to and from each other via the bus 105, for example. The communication unit 30 is implemented by the communication device 104 . Storage unit 10 is implemented by memory 102 . The processor 101 reads out and executes the programs stored in the memory 102 to obtain an input reception unit 21, an information acquisition unit 22, an incidental equipment image extraction unit 23, a position identification unit 24, a state evaluation unit 25, and a display information generation unit. 26 and the functions of the display processing unit 27 are executed. The processor 101 is an example of a processing circuit, for example, and includes one or more of a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a system LSI (Large Scale Integration).

The memory 102 includes one or more of RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (Registered Trademark) (Electrically Erasable Programmable Read Only Memory). include. The memory 102 also includes a recording medium in which a computer-readable program is recorded. Such recording media include one or more of nonvolatile or volatile semiconductor memories, magnetic disks, flexible memories, optical disks, compact disks, and DVDs (Digital Versatile Disks). Incidentally, the incidental equipment state evaluation device 1 may include integrated circuits such as ASIC (Application Specific Integrated Circuit) and FPGA (Field Programmable Gate Array).

As described above, the incidental facility state evaluation device 1 according to the first embodiment includes the information acquisition section 22, the incidental facility image extraction section 23, the state evaluation section 25, and the display information generation section . The information acquisition unit 22 acquires information of captured images including an image of the incidental equipment 7 of the structure 6 and an image of the structure 6 . The incidental facility image extraction unit 23 extracts information of the image of the incidental facility 7 from information of the captured image acquired by the information acquisition unit 22 . The state evaluation unit 25 determines the deformation level of each of the plurality of mesh-like regions R of the incidental facility 7 based on the image information of the incidental facility 7 extracted by the incidental facility image extraction unit 23. . The display information generation unit 26 generates display information in which the deformation level information of each of the plurality of regions R is associated with the image information of the incidental equipment 7 . As a result, the incidental equipment 7 can be efficiently inspected without the inspector directly touching the incidental equipment 7 . In addition, the deformation level of each of the plurality of meshed regions R in the incidental equipment 7 can be displayed in association with the information of the image of the incidental equipment 7 . Therefore, the inspector of the ancillary equipment 7 can easily grasp which part of the ancillary equipment 7 has a problem. can.

The state evaluation unit 25 also includes a division unit 31 and a region deformation level determination unit 32 . The dividing unit 31 divides the image of the incidental equipment 7 into a plurality of regions R. FIG. The region deformation level determination unit 32 determines the deformation level of each of the plurality of regions R based on the image information of each of the plurality of regions R divided by the dividing unit 31 . Since the information amount of the image information of each region R is smaller than the information of the image of the incidental equipment 7, the deformation level of the region R can be easily determined.

Further, the area change level determination section 32 includes an abnormality determination section 34 and a level determination section 35 . The abnormality determination unit 34 determines whether or not each of the plurality of regions R has an abnormality based on the image information of each of the plurality of regions R divided by the dividing unit 31 . The level determination unit 35 determines the deformation level of the region R determined to be abnormal by the abnormality determination unit 34 . As a result, since it is sufficient to determine the deformation level of only the region R determined to be abnormal, the processing load on the incidental equipment state evaluation device 1 can be reduced.

Further, the abnormality determination unit 34 determines an abnormal portion of the incidental equipment 7 based on the information of the image of the incidental equipment 7 . The level determination unit 35 determines a deformation level of a region R including an abnormal portion determined to be abnormal by the abnormality determination unit 34 among the plurality of regions R. As a result, the deformation level of the region R can be determined without performing processing for dividing the image of the incidental equipment 7 .

Moreover, the level determination unit 35 calculates the average value of the deformation level for each of the plurality of regions R when the captured image includes a plurality of images of the incidental equipment 7 of the same type. The display information generation unit 26 generates display information in which the information of the deformation level of each of the plurality of regions R and the information of the average value of the deformation level are associated with the information of the image of the incidental equipment 7 . As a result, it is possible to easily grasp in which region R deformation is likely to occur for each type of incidental equipment 7 .

Moreover, the state evaluation unit 25 includes a comprehensive evaluation unit 33 that determines the degree of deformation of each incidental facility 7 based on the deformation level of each of the plurality of regions R. The display information generation unit 26 generates display information in which the information on the deformation level of each of the plurality of regions R and the information on the degree of deformation for each incidental facility 7 are associated with the information on the image of the incidental facility 7 . As a result, the inspector can grasp the degree of deformation of the incidental equipment 7 unit obtained from the deformation level of each region R in addition to the deformation level of each region R.

The incidental facility condition evaluation device 1 also includes a position specifying unit 24 that specifies the position of the incidental facility 7 in the structure 6 . A captured image is an image including a plurality of images of incidental equipment 7 of the same type. The display information generation unit 26 generates display information in which the information on the deformation level of each of the plurality of regions R and the information on the position of the incidental facility 7 are associated with the information on the image of the incidental facility 7 . As a result, for example, it is possible to easily grasp at which position of the incidental equipment 7 among the incidental equipment 7 of the same type the deformation has occurred.

The captured image is an image obtained by repeatedly capturing images by the imaging unit 51 mounted on the traveling measuring device 2 while the traveling measuring device 2 is traveling. The position specifying unit 24 specifies the position of each of the incidental facilities 7 based on the time when the imaging unit 51 captured the area R of the incidental facilities 7 in the captured image. Thereby, the position of each of the plurality of incidental facilities 7 can be specified with high accuracy.

The captured image is an image obtained by repeatedly capturing images by the imaging unit 51 mounted on the traveling measuring device 2 while the traveling measuring device 2 is traveling. The position specifying unit 24 specifies the position of each of the plurality of incidental facilities 7 by assigning identification information to the incidental facilities 7 in the order of imaging by the imaging unit 51 . As a result, for example, it is possible to easily ascertain at what position of the incidental equipment 7 among the incidental equipment 7 of the same type the out-of-field occurrence occurs.

The configuration shown in the above embodiment shows an example of the content of the present invention, and it is possible to combine it with another known technology, and one configuration can be used without departing from the scope of the present invention. It is also possible to omit or change the part.

1 Incidental equipment state evaluation device 2 Traveling type measuring device 3 Display unit 4 Input unit 5 Network 6 Structure 7, 7 ₁ , 7 ₂ , 7 ₃ , 7 ₄ Incidental equipment 10 Storage unit 11 Imaging Image information storage unit 12 Related information storage unit 13 Incidental equipment information storage unit 20 Control unit 21 Input reception unit 22 Information acquisition unit 23 Incidental equipment image extraction unit 24 Position specifying unit 25 State evaluation unit 26 Display information generation unit 27 Display processing unit 30 Communication unit 31 Division unit 32 Area deformation level determination unit 33 Comprehensive evaluation unit 34 Abnormality determination unit 35 Level determination unit 41 Captured image information table 42 Related information Table 43 Incidental facility information table 51 Imaging unit 80 Display image 81, 82, 83, 84 Display area 85, 86 Button 87 Incidental facility status information display area R, R1 to R9 areas.

Claims (11)

an information acquisition unit that acquires information of a captured image including an image of ancillary equipment of a structure and an image of the structure;
an incidental equipment image extracting unit for extracting image information of the incidental equipment from the information of the captured image acquired by the information acquiring unit;
a state evaluation unit that determines a deformation level of each of a plurality of mesh-shaped regions of the incidental facility based on the image information of the incidental facility extracted by the incidental facility image extraction unit;
and a display information generation unit that generates display information in which information on the deformation level of each of the plurality of areas is associated with information on the image of the incidental facility. The state evaluation unit
a dividing unit that divides the image of the incidental equipment into the plurality of areas;
and a region deformation level determination unit that determines a deformation level of each of the plurality of regions based on image information of each of the plurality of regions divided by the dividing unit. Item 1. The incidental facility condition evaluation device according to Item 1. The region deformation level determination unit
an abnormality determination unit that determines whether or not there is an abnormality in each of the plurality of regions based on image information of each of the plurality of regions divided by the dividing unit;
The incidental facility condition evaluation device according to claim 2, further comprising a level determination unit that determines a level of deformation of the area determined to be abnormal by the abnormality determination unit. The state evaluation unit
an abnormality determination unit that determines an abnormal location of the incidental equipment based on image information of the incidental equipment;
2. The incidental equipment according to claim 1, further comprising: a level determination unit that determines a deformation level of an area including an abnormal portion determined to be abnormal by the abnormality determination unit among the plurality of areas. condition assessment device. The level determination unit
calculating an average value of the deformation level for each of the plurality of areas when the captured image includes a plurality of images of the incidental equipment of the same type;
The display information generation unit
5. The method according to claim 3 or 4, wherein the display information is generated by associating the information of the deformation level of each of the plurality of areas and the information of the average value of the deformation level with the information of the image of the incidental equipment. Ancillary facility condition evaluation device described. The state evaluation unit
a comprehensive evaluation unit that determines the degree of deformation of the incidental equipment unit including the plurality of regions based on the deformation level of each of the plurality of regions;
The display information generation unit
6. Display information is generated in which, in addition to information on the level of deformation of each of the plurality of areas, information on the degree of deformation for each incidental facility is associated with information on the image of the incidental facility. Incidental equipment state evaluation device according to any one of. A position specifying unit that specifies the position of the incidental equipment in the structure,
The captured image is an image containing a plurality of images of incidental equipment of the same type,
The display information generation unit
7. The incidental facility according to claim 6, wherein display information is generated by associating position information of the incidental facility with information on the image of the incidental facility in addition to information on the deformation level of each of the plurality of areas. condition assessment device. The captured image is an image obtained by repeatedly capturing images by an imaging unit mounted on the traveling measuring device while the traveling measuring device is traveling,
The position specifying unit
The incidental facility state evaluation according to claim 7, wherein the position of each of the plurality of incidental facilities is specified based on the time when the area of the incidental facility was imaged by the imaging unit in the captured image. Device. The captured image is an image obtained by repeatedly capturing images by an imaging unit mounted on the traveling measuring device while the traveling measuring device is traveling,
The position specifying unit
The incidental facility condition evaluation apparatus according to claim 7, wherein identification information is given to each of the plurality of incidental facilities in order of imaging by the imaging unit, thereby specifying the position of each of the plurality of incidental facilities. . A computer-implemented incidental equipment condition evaluation method comprising:
an information acquisition step of acquiring information of a captured image including an image of ancillary equipment of a structure and an image of the structure;
an incidental equipment image extracting step of extracting information of an image of the incidental equipment from the information of the captured image acquired by the information acquiring step;
a state evaluation step of determining a deformation level of each of a plurality of mesh-shaped regions of the incidental facility based on the image information of the incidental facility extracted by the incidental facility image extraction step;
and a display information generation step of generating display information in which information on the deformation level of each of the plurality of areas is associated with image information of the incidental facility. an information acquisition step of acquiring information of a captured image including an image of ancillary equipment of a structure and an image of the structure;
an incidental equipment image extracting step of extracting information of an image of the incidental equipment from the information of the captured image acquired by the information acquiring step;
a state evaluation step of determining a deformation level of each of a plurality of mesh-shaped regions of the incidental facility based on the image information of the incidental facility extracted by the incidental facility image extraction step;
and a display information generation step of generating display information in which information on the deformation level of each of the plurality of areas is associated with information on the image of the incidental facility.

Images