JP7339185B2 - Outdoor structure part identification device, outdoor structure part identification method, and outdoor structure part identification program - Google Patents

Description

TECHNICAL FIELD The present invention relates to an outdoor structure part identification device, an outdoor structure part identification method, and an outdoor structure part identification program, and in particular, identifies an outdoor structure part from an image analysis program incorporated in a computer and identification results by artificial intelligence. relating to a device for

The deterioration of infrastructure structures is progressing at an accelerated pace, and it is said that the need for maintenance, management, and renewal will increase further in the future. As the working population continues to decline, improving the efficiency of labor related to maintenance, management and renewal has become an urgent issue. In order to improve the efficiency of labor related to maintenance, management and renewal, there are an increasing number of research cases that apply image recognition technology using AI, whose performance has improved significantly in recent years. By using advanced neural networks called deep learning, it is possible to realize image recognition that is close to human intuition, which has been difficult in the past, and is expected to assist or replace visual inspection of damaged parts. Many studies have already been reported, such as detecting structural damage in a rectangular area (bounding box) (Non-Patent Document 1), and detecting the degree and shape of damage in an area (segmentation) (Non-Patent Document 2). It is expected that these technologies will be used to replace or assist visual inspections. In inspection and diagnosis work, it is necessary to take measures for maintenance and renewal, such as assessing the risk of damage, estimating the cause of occurrence, and considering countermeasures based on damage confirmed by visual inspection. In order to make such a secondary judgment, it is important not only to know the type, position, and shape of the damage, but also to have background information about which structural site the damage has occurred.

Also, among infrastructure structures, for example, bridges, unlike tunnels and pipelines, are outdoor structures, and due to restrictions on shooting locations, there are many situations in which structures are photographed in a relatively full view. It often contains multiple parts. According to the Ministry of Land, Infrastructure, Transport and Tourism (“Bridge Periodic Inspection Guidelines”), there are 29 types of superstructures including main girders, 8 types of substructures including beams, and 6 types of bearings as parts and members of bridges. A total of 57 types of parts/members are specified, including 8 types of road parts/members including bearings, railings, etc., and 6 types of others such as drainage equipment, and many parts/members are arranged throughout the structure. It is The guidelines also stipulate methods for judging damage and its degree, judging damage countermeasure categories, and evaluating the soundness of bridges. The classification of damage countermeasures varies depending on the location of the damage. For example, if there is free lime or exposed reinforcing bars on the floor slab, it is judged that "urgent countermeasures are required." Sites are important reference information when considering the risk of injury and countermeasures.

Journal of Japan Society of Civil Engineers F6 (Safety Issues) Vol.73 (2017) No.2 "Crack Detection System for Concrete Surface Based on Deep Convolutional Neural Network" Proceedings of the 33rd Annual Conference of Japanese Society for Artificial Intelligence 2019

However, at present, most of the methods for identifying parts of infrastructure structures rely on the visual inspection of workers. There are large discrepancies in accuracy. Therefore, it was not possible to specify the parts of the infrastructure structure with high accuracy. An object of the present invention is to identify a part of an infrastructure structure with high accuracy by machine learning using artificial intelligence.

In order to achieve the above object, the outdoor structure part identification device according to the present invention includes:
an image acquisition unit configured to acquire a first image of a first outdoor structure including a plurality of parts;
a boundary determination unit that determines a boundary between the first outdoor structure and a portion of the first outdoor structure in the acquired first image;
a first identifying unit that identifies the type of the first outdoor structure and the part of the first outdoor structure based on the determined boundary;
a colored image generation unit for coloring each of the identified portions of the first outdoor structure according to the portion of the first outdoor structure to generate a colored first image;
a model generation unit that inputs the identified first outdoor structure type, the colored first image, and the first image to artificial intelligence to perform machine learning to generate a learned outdoor structure part identification model;
a reception unit that receives input of a second image of a second outdoor structure including a plurality of parts;
Identifying the type of the second outdoor structure and the part of the second structure in the second image received by the reception unit using the learned outdoor structure part identification model generated by the model generation unit a second specifying unit to
provided.

In order to achieve the above object, a method for identifying an outdoor structure part according to the present invention comprises:
an image acquisition step of acquiring a first image of a first outdoor structure comprising a plurality of parts;
a boundary determination step of determining a boundary between the first outdoor structure and a portion of the first outdoor structure in the acquired first image;
a first identifying step of identifying the type of the first outdoor structure and the part of the first outdoor structure based on the determined boundary;
a colored image generation step of coloring each of the identified portions of the first outdoor structure according to the portion of the first outdoor structure to generate a colored first image;
a model generation step of inputting the identified first outdoor structure type, the colored first image, and the first image to artificial intelligence for machine learning to generate a learned outdoor structure part identifying model;
a receiving step of receiving input of a second image of a second outdoor structure including a plurality of parts;
Identifying the type of the second outdoor structure and the part of the second structure in the second image received in the receiving step, using the learned outdoor structure part identifying model generated in the model generating step. a second identifying step to
including.

In order to achieve the above object, the outdoor structure site identification program according to the present invention includes:
an image acquisition step of acquiring a first image of a first outdoor structure comprising a plurality of parts;
a boundary determination step of determining a boundary between the first outdoor structure and a portion of the first outdoor structure in the acquired first image;
a first identifying step of identifying the type of the first outdoor structure and the part of the first outdoor structure based on the determined boundary;
a colored image generation step of coloring each of the identified portions of the first outdoor structure according to the portion of the first outdoor structure to generate a colored first image;
a model generation step of inputting the identified first outdoor structure type, the colored first image, and the first image to artificial intelligence for machine learning to generate a learned outdoor structure part identification model;
a receiving step of receiving input of a second image of a second outdoor structure including a plurality of parts;
Identifying the type of the second outdoor structure and the part of the second structure in the second image received in the receiving step, using the learned outdoor structure part identifying model generated in the model generating step. a second identifying step to
run on the computer.

According to the outdoor structure part identification device of the present invention, the part of the outdoor structure can be identified with high accuracy by machine learning using artificial intelligence.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure for demonstrating the outdoor structure site|part identification system containing the outdoor structure site|part identification apparatus which concerns on 1st Embodiment of this invention. BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram for demonstrating the structure of the outdoor structure site|part identification apparatus which concerns on 1st Embodiment of this invention. It is a figure for demonstrating the colored image by the outdoor structure site|part identification apparatus which concerns on 1st Embodiment of this invention. 4 is a flowchart for explaining a processing procedure of learned outdoor structure part identification model generation processing of the outdoor structure part identification device according to the first embodiment of the present invention; FIG. 7 is another flowchart for explaining the processing procedure of the part identification process by the outdoor structure part identification device according to the first embodiment of the present invention; FIG. FIG. 4 is a diagram for explaining a combination of a learning image and a correct image that are input to the outdoor structure part identification device according to the first embodiment of the present invention; FIG. 3 is a diagram for explaining an example of an output image and a correct image when an image of a bridge portion is input as image data for verification to the outdoor structure portion identification device according to the first embodiment of the present invention; FIG. 5 is a diagram for explaining evaluation results of an outdoor structure part identification model generated by the outdoor structure part identification device according to the first embodiment of the present invention; It is a figure for demonstrating an example of the table which the outdoor structure site|part identification apparatus which concerns on 1st Embodiment of this invention has. FIG. 5 is a block diagram for explaining the configuration of an outdoor structure part identification device according to a second embodiment of the present invention; FIG. 9 is a flow chart for explaining the processing procedure of the outdoor structure part identification device according to the second embodiment of the present invention; FIG.

EMBODIMENT OF THE INVENTION Below, the form for implementing this invention is demonstrated in detail with reference to drawings. However, the configuration, numerical values, process flow, functional elements, etc. described in the following embodiments are merely examples, and modifications and changes are free, and the technical scope of the present invention is limited to the following descriptions. It is not intended to

[First embodiment]
An outdoor structure part identification device 104 as a first embodiment of the present invention will be described with reference to FIGS. 1 to 9. FIG. The outdoor structure part identification device 104 is used to identify the type and part of an outdoor structure such as a bridge, for example. FIG. 1 is a diagram for explaining an outdoor structure part identification system including an outdoor structure part identification device according to this embodiment.

That is, the outdoor structure 120 shown in FIG. 1 was built during the period of high economic growth, and 40 years or more have passed since its construction, so various obstacles may have occurred. In this way, in order to repair a failure that has occurred in the outdoor structure 120, first, before identifying what kind of failure has occurred, it is necessary to determine which part of the outdoor structure 120 the failure has occurred. It is necessary to identify whether the For that purpose, it is necessary to specify the site of the outdoor structure 120 accurately.

The outdoor structure part identification device of this embodiment is implemented using, for example, an outdoor structure part identification system 100 having the system configuration shown in FIG. The outdoor structure site identification system 100 includes, for example, a drone 102 having a function capable of imaging an outdoor structure 120, which is connected to each other via a wired or wireless communication network 101, and a management office, for example. A personal computer 103 installed in a computer, and AI (artificial intelligence), preferably a neural network using a convolutional neural network (CNN) or PSPNet (Pyramid Scene Parsing Network) as an algorithm. and an outdoor structure site identification device 104 incorporating known machine learning tools (software).

The outdoor structure site identification device 104 preferably uses an image of the outdoor structure 120 captured by the drone 102 to identify the type and site of the outdoor structure 120 . The personal computer 103 controls imaging of the outdoor structure 120 by the drone 102 to image a part or the whole of the outdoor structure 120 . The captured image is transmitted from the drone 102 to the personal computer 103 . The personal computer 103 transmits the received image of the outdoor structure 120 to the outdoor structure part identification device 104, and displays the identification result of the part of the outdoor structure 120 transmitted from the outdoor structure part identification device 104 on a display or the like. is displayed on the display unit 105 of .

FIG. 2 is a block diagram for explaining the configuration of the outdoor structure part identification device 104 according to this embodiment. The outdoor structure part identification device 104 has an image acquisition unit 201 , a boundary determination unit 202 , an identification unit 203 , a colored image generation unit 204 and a model generation unit 205 . Outdoor structure site identification device 104 further includes reception unit 206 and identification unit 207 . In the following description, a bridge will be described as an example of the outdoor structure 120. FIG.

Here, the outdoor structure part identification device 104 has a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a network interface, and a storage. Here, the CPU is a processor for arithmetic control, and implements each functional configuration of the outdoor structure part identification device 104 shown in FIG. 2 by executing a program. The CPU may have multiple processors and execute different programs, modules, tasks, threads, etc. in parallel. ROM stores fixed data such as initial data and programs and other programs. Also, the network interface communicates with other devices and the like via the network. Note that the number of CPUs is not limited to one, and may include a plurality of CPUs or a GPU (Graphics Processing Unit) for image processing. Moreover, it is desirable that the network interface has another CPU independent of the CPU, and writes or reads the transmission/reception data to/from the area of the RAM. It is also desirable to provide a DMAC (Direct Memory Access Controller) for transferring data between RAM and storage. Furthermore, the CPU recognizes that data has been received or transferred to the RAM and processes the data. Also, the CPU prepares the processing result in the RAM and leaves the subsequent transmission or transfer to the network interface or DMAC.

The RAM is a memory used by the CPU as a work area for temporary storage. The RAM has an area for storing data necessary for implementing the present embodiment. The storage stores databases, various parameters, modules, or data or programs necessary for implementing this embodiment. For example, the storage stores a control program for controlling the entire outdoor structure part identification device 104 .

Furthermore, the outdoor structure site identification device 104 may further include an input/output interface. A display unit, an operation unit, and a storage medium are connected to the input/output interface. The input/output interface may further be connected to a speaker as an audio output unit, a microphone as an audio input unit, or a GPS (Global Positioning System) position determination unit. Note that the RAM and storage may store programs and data relating to general-purpose functions of the outdoor structure part identification device 104 and other realizable functions.

The image acquisition unit 201 acquires an image 210 of a bridge 211 including multiple parts. The image 210 acquired by the image acquisition unit 201 is captured by the drone 102, but the method of capturing the image 210 is not limited to this. For example, images taken by a digital camera, a film camera, a camera attached to a mobile terminal such as a smartphone, a camera mounted on an artificial satellite, or the like may be used.

The image acquisition unit 201 acquires, for example, 440 images 210, but the number of images 210 acquired by the image acquisition unit 201 is not limited to this. It is preferable that the number of images 210 acquired by the image acquisition unit 201 is increased because the accuracy of learning by artificial intelligence is improved. Further, the resolution of image 210 may be either high resolution or low resolution. A high-resolution image is preferable because it improves the accuracy of learning by artificial intelligence, but the resolution is selected according to the processing capability of the outdoor structure part identification device 104, the communication capability of the communication network 101, and the like. The image acquisition unit 201 then transmits the acquired image 210 to the boundary determination unit 202 .

A boundary determination unit 202 determines the boundary of the part of the bridge 211 in the acquired image 210 . The boundary determination unit 202 determines the boundaries of the parts forming the bridge 211 shown in the image 210 transmitted from the image acquisition unit 201 .

Boundary determination is performed using techniques such as instance segmentation and semantic segmentation, for example. Here, instance segmentation is a method of dividing regions for each object. In other words, it is a method of dividing an area in which an object exists by pixels while distinguishing each object. Also, semantic segmentation is a method of dividing regions by type. Since many parts of the bridge 211 cannot be determined individually as unique objects, semantic segmentation is used in this embodiment.

Also, for determining the boundary, for example, the difference in brightness of the image 210, the shape and density of the shadow, the difference in contrast, etc. may be used. Furthermore, before determining the boundary, the boundary determining unit 202 may adjust the contrast, brightness, color balance, etc. of the image 210 before determining the boundary. Note that the boundary determination unit 202 may draw the determined boundary on the image 210 as a boundary line. This is because the visibility for the user is improved by drawing the boundary line in this manner. The boundary determining unit 202 then transmits the image 210 with the determined boundary to the specifying unit 203 .

The identifying unit 203 identifies the part of the bridge 211 based on the determined boundaries. The identification of the part by the identification unit 203 is performed based on, for example, the shape and size of the part of the bridge 211 represented by the determined boundary. The specifying unit 203 associates the position in the image 210 of the specified part with the coordinate data (x, y) in the image 210 , for example, and records it in the image 210 . The part name data and the coordinate data may be stored in a predetermined storage instead of being recorded in the image 210, and these data may be appropriately read as required.

Here, the identification unit 203 includes at least one of main girders, cross girders, longitudinal girders, floor slabs, columns/walls, beams, parapets, hard walls, bearing bodies, and balustrades as main parts of the bridge 211. identify. These main parts are large parts and important parts among the parts that constitute the bridge 211 . Among the main parts, main girders, cross girders, longitudinal girders and floor slabs are called superstructures. Columns, walls, and beams constitute bridge piers, and parapets and hard walls constitute abutments, and these are called substructures. Further, the bearing body is called the bearing part and the balustrade is called the street. These parts are defined in the "Periodical Inspection Procedures for Bridges" stipulated by the Ministry of Land, Infrastructure, Transport and Tourism.

In addition, when the main parts overlap each other in the image 210, the identifying unit 203 preferentially identifies the main part on the front side. This is because when the main parts overlap each other, the machine learning accuracy is higher for the main part shown in the near side because the whole part is shown.

The colored image generation unit 204 colors each of the identified parts of the bridge 211 according to the part to generate a colored image 212 . FIG. 3 is a diagram for explaining a colored image by the outdoor structure part identification device according to this embodiment. The colored image 212 that has been colored in this manner becomes teacher data. In this embodiment, the 440 images 210 acquired by the image acquisition unit 201 are colored, and the 440 colored images 212 that have been colored serve as teacher data.

The model generating unit 205 inputs the colored image 212 and the image 210 to the artificial intelligence and performs machine learning to generate a learned outdoor structure site identification model. Machine learning by artificial intelligence is performed using known algorithms. Here, for example, PSPNet is used. PSPNet is an algorithm that can detect widely distributed features with higher performance compared to conventional semantic segmentation models. In the image 210 of the bridge 211, parts are often distributed widely in the local image of the bridge 211, and it is considered suitable as a model for detecting the shapes of widely distributed parts. In machine learning, the loss function was unweighted.

In addition, the model generation unit 205 inflates the number of images 210 to be learned by artificial intelligence in order to improve the accuracy of machine learning by artificial intelligence and generate a more accurate outdoor structure part identification model. Let The model generation unit 205 obtains padding data using, for example, any one of enlargement, rotation and inversion.

Also, the model generation unit 205 may use transfer learning to improve the accuracy of machine learning by artificial intelligence. Here, transfer learning is a technique that aims to improve the performance of a model by diverting a trained model using a different data set to another problem and performing partial learning. In particular, this method can be expected to improve inference performance and reduce learning time when training data is insufficient. In this embodiment, transfer learning is performed using an image dataset Pascal VOC2012 Semantic Segmentation Dataset in which 22 types of objects are labeled by semantic segmentation.

The reception unit 206 receives an input of an image 220 of a bridge 221 including multiple parts. In this embodiment, the image 220 to be received is an image captured by the drone 102 , but the method of capturing the bridge 221 is not limited to the drone 102 .

The identifying unit 207 identifies the part of the bridge 221 in the image 220 received by the receiving unit 206 using the trained outdoor structure part identifying model generated by the model generating unit 205 . The identification unit 207 identifies the part of the bridge 221 from the received image 220 using the trained outdoor structure part identification model based on the correlation between the image 220 and the part in the learned outdoor structure identification model. . The identification unit 207 identifies, as the part of the bridge 221, the part of the bridge 221 that has a high degree of correlation with the part in the learned outdoor structure part identification model. After that, the outdoor structure part identification device 104 outputs the result image 230 .

FIG. 4 is a flowchart for explaining the processing procedure of the learned outdoor structure part identification model generation process of the outdoor structure part identification device 104 . FIG. 5 is another flow chart for explaining the processing procedure of the part identification processing by the outdoor structure part identification device 104 . The flowcharts shown in FIGS. 4 and 5 are executed by a CPU (Central Processing Unit) (not shown) of the outdoor structure part identification device 104 using a ROM (Read Only Memory) and a RAM (Random Access Memory). 2 are realized.

First, with reference to FIG. 4, a process of generating a learned outdoor structure site identification model by machine learning using artificial intelligence will be described. In step S<b>401 , the image acquisition unit 201 acquires the image 210 of the bridge 211 . In step S<b>403 , the boundary determining unit 202 determines the boundary of the part 211 of the bridge in the acquired image 210 . In step S405, the identifying unit 203 identifies the part of the bridge 211 based on the determined boundary. In step S407, the identifying unit 203 determines whether or not there is an overlap between main parts. Determination of overlap is made, for example, based on whether the contour line is interrupted or a surface that should be visible is not visible for a certain main part. When determining that there is no overlap between main parts (step S407 NO), the outdoor structure part identification device 104 proceeds to step S411. If it is determined that there is an overlap between the main parts (YES in step S407), the outdoor structure part identification device 104 proceeds to step S409. In step S<b>409 , the identification unit 203 identifies the parts by prioritizing the main parts on the near side.

In step S<b>411 , the colored image generating unit 204 generates the colored image 212 by coloring each of the identified parts of the bridge 211 according to the part. In step S413, the colored image 212 and the image 210 (original image) are input to artificial intelligence for machine learning to generate a learned outdoor structure site identification model. In step S<b>415 , the outdoor structure part identification device 104 determines whether machine learning by artificial intelligence has been completed for all images 210 . If it is determined that machine learning has not been completed for all images 210 (NO in step S415), the outdoor structure part identification device 104 returns to step S403. If it is determined that machine learning has been completed for all images 210 (YES in step S415), the outdoor structure part identification device 104 proceeds to step S417. In step S417, the model generation unit 205 generates a learned outdoor structure part identification model based on the learning result.

Next, referring to FIG. 5, the process of identifying the part of the bridge 221 in the newly acquired image 220 using the generated trained outdoor structure part identifying model will be described. In step S501, the reception unit 206 receives an input of an image 220 of a bridge 221 including multiple parts. In step S<b>503 , the identification unit 207 identifies the parts of the bridge 221 using the learned outdoor structure part identification model generated by the model generation unit 205 . In step S505, the outdoor structure part identification device 104 outputs the identification result. In step S<b>507 , the outdoor structure part identification device 104 determines whether or not output of results for all the images 220 received by the receiving unit 206 has been completed. If it is determined that the output of results has not been completed for all images 220 (NO in step S507), the outdoor structure part identification device 104 returns to step S503. If it is determined that the results have been output for all images 220 (YES in step S507), the outdoor structure part identification device 104 ends the process.

FIG. 6 is a diagram for explaining a combination of a learning image 210 and a colored image 212 (correct image) input to the outdoor structure part identification device 104. As shown in FIG. As illustrated, an image 210 for which learning by artificial intelligence has been completed, a colored image 212, and a label 601 are arranged in parallel and displayed. A label 601 indicates the relationship between the color applied to the colored image 212 and the portion of the bridge 211 . For example, as shown in Fig. 6, the coloring according to the part is reddish brown for main girders, dark green for cross girders, ocher for vertical girders, blue for floor slabs, purple for columns and walls, and purple for beams. is sky blue, the battlements are gray, the hard walls are brown, the main body of the support is red, and the balustrade is yellowish green.

In this embodiment, 110 images 220 were used to evaluate the generated outdoor structure site identification model. FIG. 7 is a diagram for explaining an example of an output image and a correct image when an image of a bridge portion is input as image data for verification to the outdoor structure portion identification device 104. In FIG. FIG. 7(a) shows an example in which the output image from the outdoor structure part identification device 104 and the correct image correspond well. Also, FIG. 7B shows an example in which the correspondence between the output image and the correct image by the outdoor structure part identification device 104 is not preferable.

ｉ、ｊ：画像番号、部位番号
Ｉ、Ｊ：評価画像総数（＝１１０）、部位総数（＝１１）
ＰＲＥＤ_ｉ，ｊ：画像ｉにおける部位ｊと出力した画素
ＴＲＵＥ_ｉ，ｊ：画像ｉにおける部位ｊの正解画素 The generated outdoor structure site identification model was evaluated using Mean IoU (mIoU: Mean Intersection Over Union). The IoU is used as an evaluation index in the field of object detection, and quantifies the matching rate between the correct region and the output region. In this embodiment, it is obtained by Equation (1) using the image number i and the part number j as parameters. This is averaged over all 110 images to be evaluated and the mIoU _j for each site is calculated using formula (2). We evaluated the performance of

i, j: image number, part number I, J: total number of evaluation images (=110), total number of parts (=11)
PRED _i,j : Region j in image i and output pixel TRUE _i,j : Correct pixel of region j in image i

FIG. 8 is a diagram for explaining the evaluation result of the outdoor structure part identification model generated by the outdoor structure part identification device 104. As shown in FIG. As shown in the figure, mIoU when neither padding nor transfer learning was performed was 25.1%, and mIoU when padding was performed but transfer learning was not performed was 30.2%. , the mIoU when padding and transfer learning were performed was 47.0%. In this way, when the 440 images 210 are used as training data, the padding and transfer learning can be used to obtain an IoU evaluation of close to 50%, indicating that a highly accurate outdoor structure part identification model can be obtained. Ta.

In the above description, a bridge was used as an example of an outdoor structure, but the outdoor structure is not limited to this. )), etc. In this case, the outdoor structure part identification device 104 preferably has a table 900 shown in FIG. The table 900 stores parts 902 in association with types 901 of outdoor structures. The outdoor structure site identification device 104 refers to the table 900 and identifies the site of each outdoor structure. The outdoor structure part identification device 104 identifies the type 901 of the outdoor structure from the shape (silhouette), size, etc. of the outdoor structure in the image. Metadata such as position information (for example, GPS data) may be added to the image of the outdoor structure, and the type 901 of the outdoor structure may be specified by taking this position information into consideration. Furthermore, bridges include, for example, girder bridges, truss bridges, arch bridges, rigid-frame bridges, cable-stayed bridges, and suspension bridges. can also be applied.

According to this embodiment, machine learning is performed by giving artificial intelligence a colored image in which parts of a bridge are colored as training data. The parts of the bridge can be identified quickly with high accuracy.

[Second embodiment]
Next, an outdoor structure site identification device according to a second embodiment of the present invention will be described with reference to FIG. FIG. 10 is a block diagram for explaining the configuration of the outdoor structure part identification device according to this embodiment. The outdoor structure site identification device according to the present embodiment differs from that of the first embodiment in that it has an estimation unit. Since other configurations and operations are the same as those of the first embodiment, the same configurations and operations are denoted by the same reference numerals, and detailed description thereof will be omitted.

The outdoor structure site identification device 1000 further has an estimation unit 1001 . In the image 210, if the main part and the non-main part that are parts other than the main part overlap and the main part is hidden by the non-main part, the estimation unit 1001 determines whether the main part is hidden by the non-main part. Guess the shape of the part. Estimation of the shape of the hidden portion is performed, for example, by extrapolating the outline of the main portion that is hidden by the non-main portion and interrupted. As described above, when a part of the main part is missing in the image 210, the estimation unit 1001 estimates the shape of the missing part and makes up for it, thereby improving the accuracy of machine learning by artificial intelligence. I can expect it. Then, the identifying unit 203 identifies the part of the bridge 211 based on the estimated shape.

FIG. 11 is a flow chart for explaining the processing procedure of the outdoor structure part identification device according to this embodiment. In step S1101, the outdoor structure part identification device 104 determines whether or not the main part is hidden by the non-main parts. When determining that the main part is not hidden (NO in step S1101), the outdoor structure part identification device 104 proceeds to step S411. When determining that the main part is hidden (YES in step S1101), the outdoor structure part identification device 104 proceeds to step S1103. In step S1103, the estimating unit 1001 estimates the shape of the hidden portion by a method such as extrapolation.

According to the present embodiment, the shape of the main part hidden in the non-main part is estimated and given as training data to the artificial intelligence, so that a more accurate outdoor structure part identification model can be generated, and the bridge can be constructed. Sites can be identified more accurately and quickly.

[Other embodiments]
Although the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. Also, any system or apparatus that combines separate features included in each embodiment is also included in the scope of the present invention.

Further, the present invention may be applied to a system composed of a plurality of devices, or may be applied to a single device. Furthermore, the present invention is also applicable when an information processing program that implements the functions of the embodiments is directly or remotely supplied to a system or apparatus. Therefore, in order to implement the functions of the present invention on a computer, a program installed in a computer, a medium storing the program, and a WWW (World Wide Web) server from which the program is downloaded are also included in the scope of the present invention. . In particular, non-transitory computer readable media storing programs that cause a computer to perform at least the processing steps included in the above-described embodiments are included within the scope of the present invention.

Claims (9)

an image acquisition unit configured to acquire a first image of a first outdoor structure including a plurality of parts;
a boundary determination unit that determines a boundary between the first outdoor structure and a portion of the first outdoor structure in the acquired first image;
a first identifying unit that identifies the type of the first outdoor structure and the part of the first outdoor structure based on the determined boundary;
a colored image generation unit for coloring each of the identified portions of the first outdoor structure according to the portion of the first outdoor structure to generate a colored first image;
a model generating unit that inputs the colored first image and the first image to artificial intelligence to perform machine learning to generate a learned outdoor structure site identification model;
a reception unit that receives input of a second image of a second outdoor structure including a plurality of parts;
The type of the second outdoor structure and the part of the second outdoor structure in the second image received by the reception unit are determined using the learned outdoor structure part identification model generated by the model generation unit. a second specifying part to specify;
outdoor structure site identification device. The first specific part includes main girders, cross girders, vertical girders, floor slabs, columns, walls, beams, parapets, hard walls, bearing bodies, and balustrades of a bridge as main parts of the first outdoor structure. 2. The outdoor structure part identification device according to claim 1, which identifies at least one of them. 3. The outdoor structure site identification device according to claim 2, wherein when the main sites overlap in the first image, the first identifying unit preferentially identifies the main site on the front side. In the first image, when the main portion and a non-major portion that is a portion other than the main portion overlap and the main portion is hidden by the non-major portion, the non-major portion of the main portion It further has a guessing part that guesses the hidden part,
4. The outdoor structure site identification according to claim 2, wherein the first identification unit identifies the type of the first outdoor structure and the site of the first outdoor structure using the result of estimation by the estimation unit. Device. 5. The outdoor structure part identification device according to any one of claims 1 to 4, wherein the model generation unit generates the learned outdoor structure part identification model using PSPNet. The outdoor structure according to any one of claims 1 to 5, wherein the model generating unit generates the learned outdoor structure specific model using any one of enlargement, rotation and inversion as padding data. Part identification device. 7. The outdoor structure part identification device according to any one of claims 1 to 6, wherein the model generation unit generates the learned outdoor structure identification model using transfer learning. an image acquisition step of acquiring a first image of a first outdoor structure comprising a plurality of parts;
a boundary determination step of determining a boundary between the first outdoor structure and a portion of the first outdoor structure in the acquired first image;
a first identifying step of identifying the type of the first outdoor structure and the part of the first outdoor structure based on the determined boundary;
a colored image generation step of coloring each of the identified portions of the first outdoor structure according to the portion of the first outdoor structure to generate a colored first image;
a model generation step of inputting the colored first image and the first image to artificial intelligence and performing machine learning to generate a learned outdoor structure site identification model;
a receiving step of receiving input of a second image of a second outdoor structure including a plurality of parts;
The type of the second outdoor structure and the part of the second outdoor structure in the second image received in the receiving step are identified using the learned outdoor structure part identification model generated in the model generating step. a second identifying step of identifying;
An outdoor structure part identification method including. an image acquisition step of acquiring a first image of a first outdoor structure comprising a plurality of parts;
a boundary determination step of determining a boundary between the first outdoor structure and a portion of the first outdoor structure in the acquired first image;
a first identifying step of identifying the type of the first outdoor structure and the part of the first outdoor structure based on the determined boundary;
a colored image generation step of coloring each of the identified portions of the first outdoor structure according to the portion of the first outdoor structure to generate a colored first image;
a model generation step of inputting the colored first image and the first image to artificial intelligence and performing machine learning to generate a learned outdoor structure site identification model;
a receiving step of receiving input of a second image of a second outdoor structure including a plurality of parts;
The type of the second outdoor structure and the part of the second outdoor structure in the second image received in the receiving step are identified using the learned outdoor structure part identification model generated in the model generating step. a second identifying step of identifying;
Outdoor structure site identification program that causes a computer to execute