JP7730679B2 - How to create a damage diagram - Google Patents

Description

The present invention relates to a damage mapping method, a damage mapping device, a damage mapping system, and a recording medium, and in particular to a technology for detecting damage based on multiple images acquired by photographing a subject in segments.

Systems that analyze the deterioration and damage of large structures such as bridges based on photographed images of the structure have been known for some time. For example, Patent Document 1 describes a method of photographing a structure such as a bridge in multiple divided areas, stitching the images together to generate a single composite image, identifying cracks and other damaged areas, and creating related data (such as the length and width of the cracks). To create the composite image, the divided images are photographed so that overlapping areas are created, and then projective transformation is performed on the images based on the characteristics of the overlapping areas. The relative positions of the images are determined so that the feature positions match, and a single composite image is generated. To prevent image degradation in the overlapping areas, the system does not combine two images to generate an image of the overlapping areas, but instead uses (adopts) only one of the images to obtain the overlapping area image. The system also describes a method for selecting the image to be adopted based on image brightness, subject clarity, etc.

WO/2019/003796

However, if the selection of images in the overlapping area was determined based on image brightness or sharpness, the accuracy of the damage (cracks, etc.) data could be reduced.

Therefore, the present invention, which solves the above-mentioned problems, includes the steps of: inputting a plurality of images obtained by photographing a subject in divided portions so that some areas overlap; calculating synthesis parameters (e.g., projective transformation parameters) for synthesizing the plurality of images based on corresponding points between the images;
detecting damage to the subject from the images constituting the plurality of images; and synthesizing the detection results for the plurality of images based on the synthesis parameters;
In the combining step, in the overlapping area, one of the overlapping images is selected and combined, and the selected image has a smaller value of the combining parameter than the other image.

The present invention improves the accuracy of measuring the shape of damage in overlapping regions.

1 is a block diagram showing a configuration of a damage diagram creation system according to a first embodiment. FIG. FIG. 2 is a diagram illustrating a configuration of a processing unit. FIG. 4 is a diagram illustrating information stored in a storage unit. FIG. 2 is a diagram illustrating a configuration of a server. FIG. 2 is a flowchart illustrating a damage diagram creation method according to the first embodiment. FIG. 10 is another flowchart showing the damage diagram creation method according to the first embodiment. FIG. 10 is a diagram showing an example of a procedure for photographing a deck. FIG. 10 is a diagram illustrating an example of a procedure for photographing a case. FIG. 2 is a diagram showing the shooting range of each image. FIG. FIG. 2 is a diagram showing a hierarchical structure of folders. FIG. 10 is a diagram showing a state in which photographed images are stored in a folder. 10 is a flowchart showing a process of determining an image layout. FIG. 10 is a diagram showing a state of setting a reference image. FIG. 10 is a diagram illustrating an example of image arrangement for each image group. FIG. 10 is another diagram showing an example of image arrangement for each image group. FIG. 10 is a diagram showing a state in which vectorized detection results are superimposed on an image. FIG. 10 is a diagram showing how the relative arrangement of each image group is changed. FIG. 10 is a diagram showing how corresponding points are set in an image group. FIG. 10 is a diagram showing a synthesis of detection results. FIG. 10 is another diagram showing the synthesis of detection results. FIG. 10 illustrates the merging of detection results in an area where images overlap. FIG. 10 is a diagram showing a composite image. FIG. 10 is another diagram showing a composite image. FIG. 10 is a diagram showing a state in which detection results and the like are stored in the same folder as images. FIG. 10 is a block diagram showing the configuration of a damage diagram creation device according to a second embodiment. 10A and 10B are diagrams illustrating another example of combining detection results in an area where images overlap. 10A and 10B are diagrams illustrating another example of combining detection results in an area where images overlap. 10A and 10B are diagrams illustrating an example of calculation of synthesis parameters in an area where images overlap. FIG. 11 is a flowchart illustrating a damage diagram creation method according to a third embodiment. FIG. 10 is a conceptual diagram of a damage diagram creation method according to a third embodiment.

Embodiments of the damage diagram creation method, damage diagram creation device, damage diagram creation system, and recording medium according to the present invention will be described in detail below with reference to the accompanying drawings.

First Embodiment
<Bridge structure>
A bridge is a preferred example of a structure to which the damage mapping method and damage mapping device of the present invention can be applied. For details of this structure, see FIG. 1 of Patent Document 1. While the following embodiment will be described with a bridge as the target (subject), the target structure is not limited to a bridge; it may also be a tunnel, a building, a road, or the like.

<Image acquisition>
When photographing images of a bridge to detect damage, an inspector uses a digital camera 100 (see FIG. 1) to photograph the bridge from below and acquires multiple images of the inspection area (multiple images each capturing a different part of the bridge). The photographs are taken while moving appropriately in the direction of extension of the bridge and in a direction perpendicular to the direction. If the conditions around the bridge make it difficult for the inspector to move, the digital camera 100 may be mounted on a mobile object that can move along the bridge to photograph the bridge. Such a mobile object may be equipped with an elevation mechanism and a rotation mechanism (a mechanism for panning and/or tilting) for the digital camera 100. Examples of mobile objects include, but are not limited to, vehicles, robots, and flying objects (drones, etc.).

<Configuration of the damage diagram creation system>
1 is a block diagram showing a schematic configuration of a damage diagram creation system 10. The damage diagram creation system 10 includes a digital camera 100, a client 200, and a server 300, and creates a damage diagram by detecting damage in multiple images acquired by photographing a subject in sections and synthesizing the detection results. In the damage diagram creation system 10, a device (information terminal) such as a personal computer, tablet terminal, or smartphone that inputs images and receives results from the server 300 can be used as the client 200, and a computer connected to the client 200 via a network can be used as the server 300.

<Digital camera configuration>
The digital camera 100 acquires images using an imaging optical system 110 equipped with a photographing lens and an imaging element (not shown). Examples of imaging elements include a CCD (Charge Coupled Device) type imaging element and a CMOS (Complementary Metal-Oxide Semiconductor) type imaging element. R (red), G (green), or B (blue) color filters are provided on the light-receiving surface of the imaging element, and a color image of a subject can be acquired based on the signals of each color. The digital camera 100 wirelessly communicates with the client 200 via the wireless communication unit 130 and antenna 132, and the captured image is input to the processing unit 210 for processing, as described below. The digital camera 100 may be incorporated into a separate housing from the client 200, or may be integrated with the client 200.

<Overall client configuration>
The client 200 includes a processing unit 210, a storage unit 220, a display unit 230, and an operation unit 240, and these units are connected to each other to transmit and receive necessary information. The client 200 also communicates wirelessly with the digital camera 100 via an antenna 212 to acquire images captured by the digital camera 100. The client 200 is also connected to a server 300 via a network NW, and transmits and receives, between the server 300, acquired images, processing results for the transmitted images (synthesis detection results, composite images, etc.), processing requests and their responses, and the like.

<Configuration of Processing Unit>
FIG. 2 shows the configuration of the processing unit 210. The processing unit 210 includes an image input unit 210A (image input unit), a file management unit 210B, a display control unit 210C, and a communication control unit 210D, and performs functions such as transmitting captured images acquired by the digital camera 100 to the server 300, receiving processed results, and controlling the display of the processed results on the monitor 232. The image input unit 210A inputs captured images of a bridge (multiple images of the bridge captured in segments) from the digital camera 100 (or a recording medium, a network, etc.). The file management unit 210B creates folders in response to user operations via the keyboard 242 and/or mouse 244. The display control unit 210C controls the display of captured images, received processed results, etc. on the monitor 232. The communication control unit 210D transmits and receives images and information to and from the digital camera 100 via the antenna 212, and also transmits and receives images and information to and from the server 300 via the network NW. The ROM 210E (ROM: Read Only Memory, non-transitory recording medium) stores computer-readable code of a program required for image acquisition, transmission and reception, etc. (including a program or part thereof for executing the damage diagram creation method according to the present invention).

The functions of the processing unit 210 described above can be realized by various processors or electrical circuits referencing software recorded on a recording medium, similar to the server 300 described in detail below.

<Configuration of storage unit>
The storage unit 220 is composed of a non-transitory recording medium such as a CD (Compact Disk), a DVD (Digital Versatile Disk), a hard disk, or various semiconductor memories, and its control unit, and stores the images and information shown in FIG. 3 in association with each other. The captured image 220A is a plurality of images of the subject, a bridge (deck portion), photographed in sections by the digital camera 100 and input by the image input unit 210A. Note that instead of images input by the digital camera 100 and the image input unit 210A, images acquired via a network or recording medium may be stored. The detection result 220B includes damage detection results for each image constituting the captured image 220A and a detection result obtained by combining the detection results for each image. The composite image 220C is an image obtained by combining the captured images (including a group of partially combined images). The damage mapping image 220D is an image in which information indicating damage (detection results, etc.) is mapped. These images and information can be stored in folders (see Figures 11 and 12).

<Configuration of display unit and operation unit>
The display unit 230 includes a monitor 232 (display device) and can display input images, images and information stored in the storage unit 220, and results of processing by the server 300. The operation unit 240 includes a keyboard 242 and a mouse 244 as input devices and/or pointing devices, and a user can perform operations required for executing the damage diagram creation method according to the present invention, such as creating folders, storing images in folders, and specifying corresponding points, via these devices and the screen of the monitor 232 (described later).

<Server configuration>
FIG. 4 is a diagram showing the configuration of the server 300. The server 300 includes an image acquisition unit 300A (image acquisition unit), a synthesis parameter calculation unit 300B (synthesis parameter calculation unit), an image synthesis unit 300C, a damage detection unit 300D (damage detection unit), a detection result synthesis unit 300E (detection result synthesis unit), and a corresponding point designation unit 300F (corresponding point designation unit). The server 300 further includes a damage mapping unit 300G, a detection result output unit 300H (detection result output unit), a display control unit 300I (display control unit), a communication control unit 300J, and a ROM 300K (non-transitory recording medium). The server 300 is connected to the client 200 via the network NW, acquires a captured image (captured image 220A in FIG. 3 ) from the client 200, and performs damage detection, synthesis of the detection results, etc.

The image acquisition unit 300A inputs captured images (captured images 220A in Figure 3) from the client 200. The synthesis parameter calculation unit 300B calculates synthesis parameters for synthesizing the captured images based on corresponding points between the images. The image synthesis unit 300C synthesizes the captured images based on the synthesis parameters. The damage detection unit 300D detects (extracts and measures) damage (cracks, peeling, corrosion, etc.) to the subject (bridge) from the captured images. The detection result synthesis unit 300E synthesizes the damage detection results (detection results) for the captured images based on the synthesis parameters calculated by the synthesis parameter calculation unit 300B. The corresponding point designation unit 300F designates corresponding points between one group of displayed images and another group of images based on user instructions. The damage mapping unit 300G maps information indicating damage onto the composite image. The detection result output unit 300H outputs damage detection results, composite detection results, identification information, composite images, damage mapping images, etc. to the client 200. The display control unit 300I displays captured images, detection results, etc. on the monitor 232 (display device). The communication control unit 300J transmits and receives images and information to and from the client 200 via the network NW. The ROM 300K (non-transitory recording medium) stores computer-readable code for various programs for operating the damage diagram creation system 10, such as a damage diagram creation program for executing the damage diagram creation method of the present invention. In addition to the above-mentioned components, the server 300 also includes a recording device (e.g., a magneto-optical recording medium such as a hard disk) (not shown) that records images and information acquired from the client 200 and processing results (e.g., damage detection results) by each component of the server 300. Recorded images, etc. can be transmitted to the client 200 upon request.

The functions of each part of the server 300 described above can be realized using various processors. These include, for example, a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) to realize various functions. The above-mentioned various processors also include programmable logic devices (PLDs), such as FPGAs (Field Programmable Gate Arrays), whose circuit configuration can be changed after manufacture. Furthermore, the above-mentioned various processors also include dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with a circuit configuration designed specifically to execute specific processes.

The functions of each section may be realized by a single processor, or by a combination of multiple processors. Multiple functions may also be realized by a single processor. Examples of multiple functions configured by a single processor include, first, a configuration where a single processor is configured by combining one or more CPUs and software, as in client and server computers, and this processor realizes multiple functions. Second, a configuration where a processor is used to realize the functions of the entire system on a single IC (Integrated Circuit) chip, as in a system-on-chip (SoC). In this way, various functions are configured as a hardware structure using one or more of the various processors described above. Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit (circuitry) that combines circuit elements such as semiconductor devices.

When the above-mentioned processor or electrical circuit executes software (programs), processor-readable code for the software to be executed (including programs for executing the damage map creation method of the present invention) is stored in a non-transitory recording medium such as ROM 300K (see Figure 4), and the processor references the software. Instead of ROM 300K, the code may be recorded in a non-transitory recording medium such as various types of magneto-optical recording devices or semiconductor memory. When processing using software, for example, RAM (Random Access Memory) is used as a temporary storage area, and data stored in, for example, EEPROM (Electronically Erasable and Programmable Read Only Memory) is referenced. Note that devices such as RAM and EEPROM are not shown in Figure 4.

In addition, the functions of the client 200 described above can be realized by various processors, electrical circuits, and software, just like the server 300.

<Image processing procedure>
Image processing by the damage diagram creation system 10 will now be described. Figures 5 and 6 are flowcharts showing the image processing procedure (including each step of the damage diagram creation method according to the present invention). In these figures, steps S100 to S112 indicate the processing performed by the client 200, and steps S200 to S236 indicate the processing performed by the server 300.

photograph
In the procedure shown in FIGS. 5 and 6, the bridge 1 (structure) is photographed in sections by the digital camera 100 to obtain a plurality of photographed images (step S100).

In this embodiment, the case of photographing a deck slab 6 will be described. Figure 7 is a diagram showing an example of the procedure for photographing a deck slab 6. Figure 7 shows how photographing is performed in units of area A, including the lattice spacing GO defined by the main girders (members extending in the x direction) and cross girders (members extending in the y direction), and how photographing is repeated while the photographing area is moved sequentially in the y direction and x direction (in the direction of the arrows). As long as an image of the entire photographing range can be obtained, photographing may be performed using other procedures. Note that in Figure 7, the extension direction of the bridge (deck slab) is x, the direction perpendicular to x within the plane of the deck slab is y, and the direction perpendicular to the deck slab (vertical downward direction) is z, and the coordinate system is formed as (x, y, z).

Figure 8 shows an example of the shooting procedure for one grid GO. In the example of Figure 8, shooting begins at area A1 at the +x end of the grid GO, moving to area Ai at the -x end, then returns to the +x end, starting at area Aj and ending at area An at the -x end, capturing a total of n images (n is an integer greater than or equal to 2). A different pattern (e.g., in the order of areas A1-Ai-An-Aj) may also be used. When shooting, the shooting position may be changed after each image to always capture a straight-on image, or multiple images may be taken from a single shooting position while changing the shooting direction (in this case, images captured from an oblique angle may be included). It is also preferable to appropriately set the shooting position and direction to create sufficient overlap (e.g., about 30%) between adjacent images, making it easier and more accurate to detect and set corresponding points.

Figure 9 is an example of captured images, showing 10 images i1 to i10 taken with overlapping images ensured. Figure 10 shows images i1 to i10 individually. Note that Figures 9 and 10 show the frame F of the coffer GO (a rectangle defined by the main girders and cross girders), and do not show damage to other components and floorboards.

The client 200 inputs the above-mentioned multiple captured images via the digital camera 100 (image capturing optical system 110, wireless communication unit 130, antenna 132) and processing unit 210 (communication control unit 210D, image input unit 210A, antenna 212) (step S102).

Image and information storage
In the damage diagram creation system 10, folders are created in the storage unit 220 of the client 200 to store the captured images. Fig. 11 is a diagram showing an example of a folder structure. In the example of Fig. 11, a main folder MF is created for the entire bridge, and subfolders SF1 and SF2 are created within this main folder for each capture area (inspection area 1A, inspection area 1B). Within the subfolders for each capture area, subfolders SS1 to SS5 are further created for each lattice spacing GO defined by the main girder and cross girder, and the captured images are stored therein. Fig. 12 shows the state in which 10 captured images are stored for the lattice spacing A001 (subfolder SS1).

The folder structure shown in Figures 11 and 12 can be created by the file management unit 210B in response to operations on the operation unit 240 (keyboard 242, mouse 244). Note that the folder structure may be different from the examples in Figures 11 and 12; for example, subfolders may be created for each component number. As will be described in detail below, the damage diagram creation system 10 combines the damage detection results (detection results) for each folder (subfolders are also included in the term "folder") and stores the combined detection results in the same folder as the captured image.

<Image acquisition>
The operation unit 240 of the client 200 accepts a damage detection and synthesis instruction operation via the keyboard 242 and/or mouse 244 (step S104), and the server 300 (image acquisition unit 300A) acquires captured images in response to this operation (step S200). The image acquisition unit 300A acquires images stored in the same folder in the client 200 (storage unit 220) as images belonging to the same group. For example, the image acquisition unit 300A acquires 10 captured images stored in the subfolder SS1 shown in FIGS. 11 and 12 as images belonging to the same group.

<Calculation of synthesis parameters>
After the captured images are acquired in step S200, the server 300 (combination parameter calculation unit 300B) calculates a combination parameter for combining multiple images based on corresponding points between the images (step S202). For example, the combination parameter can be calculated as a projective transformation matrix of each captured image relative to a reference image. Note that, although the flowchart in FIG. 5 illustrates an example in which the combination parameter calculation in step S202 is performed first, damage detection in step S204 may be performed first, or steps S202 and S204 may be performed in parallel.

<Damage detection>
The server 300 (damage detection unit 300D) detects (extracts and measures) damage from the acquired captured image (step S204). Damage classifications include peeling, water leakage, cracks, rust, etc., but the specific type of damage to be detected may be set according to conditions such as the type and characteristics of the building (subject), the purpose of the inspection, etc. Furthermore, detection items include position, size, direction, range, shape, etc., but the detection items may also be set according to the damage classification, the type and characteristics of the building, the purpose of the inspection, etc. In damage detection, the damage detection unit 300D vectorizes the detection result and represents it as a line segment or a set of lines having a start point and an end point (in the case of linear damage such as cracks), or as a polygon or other figure composed of such line segments (in the case of spreading damage such as peeling or corrosion).

Damage can be detected using a variety of techniques depending on the classification, but for cracks, the crack detection method described in Patent Publication No. 4006007 can be used, for example. This method involves creating a wavelet image and determining crack areas based on the wavelet image. In the wavelet image creation step, wavelet coefficients corresponding to the two densities to be compared are calculated, and wavelet coefficients are calculated for each of the two densities to create a wavelet coefficient table. An input image of the concrete surface to be detected for cracks is then subjected to wavelet transformation. In the crack area determination step, the wavelet coefficient corresponding to the average density of neighboring pixels in a local area and the density of the pixel of interest in the wavelet coefficient table is used as a threshold, and the wavelet coefficient of the pixel of interest is compared with the threshold to determine whether the area is cracked or not.

In addition, one method for detecting rust and peeling is the processing method of the steel bridge paint film inspection system described in JP 2010-538258 A. This processing method detects rust and peeling using color information from image files of the steel bridge paint film, video processing, watersheds, and Parzen windows.

In this way, the damage mapping system 10 according to the first embodiment detects damage from captured images before compositing. Therefore, damage detection performance does not deteriorate due to degradation in image quality in overlapping image areas, and damage can be detected with high accuracy based on multiple images acquired by capturing the subject in separate images. The damage detection results are combined using a combination parameter for the images, as described below.

<Determining whether or not synthesis is possible>
The server 300 (combining parameter calculation unit 300B) determines whether all captured images can be combined into one image based on the combining parameters calculated in step S202 (step S206). Whether combining is possible can be determined based on the number of corresponding points, whether the corresponding points are sufficiently reliable (whether the corresponding points are characteristic points), whether a projective transformation matrix can be calculated using the corresponding points, and other factors. This determination may be made by repeatedly calculating the projective transformation matrix and its evaluation value while changing the combination of corresponding points using a RANSAC (RANdom Sample Consensus) algorithm or the like. If all images can be combined into one image (YES in step S206), the process proceeds to step S222, where the detection results are combined. If all images cannot be combined into one image (NO in step S206), the process proceeds to step S222 after performing the processes of steps S208 to S220 as described below. Steps S208 to S220 are performed when all images cannot be combined into a single image (automatic combination). In these steps, for image groups that can be combined, combination parameters are calculated for each image group, and damage detection results are combined for each image group. On the other hand, for image groups that cannot be combined, combination parameters are calculated by specifying corresponding points based on a user operation, as described below, and detection results are combined based on the calculated combination parameters. For example, in the case of image groups G1 and G2 (see FIGS. 15 to 18), which will be described later, the detection results for the images that make up image groups G1 and G2 are combined, and between image groups G1 and G2, the combined detection results for image groups G1 and G2 are combined into a single image based on combination parameters calculated based on a user operation (see FIGS. 19 to 21).

<Image Classification>
If it is determined in step S206 that "all captured images cannot be combined into one image," the combining parameter calculation unit 300B divides the captured images into groups of images that can be combined (step S208), and determines the image layout (calculates the combining parameters) for each image group based on corresponding points between the images (step S210). Once the image layout has been determined, the detection results can be combined based on that layout.

<Determining image layout>
13 is a flowchart showing the details of the image layout determination process in step S210 of FIG. 5. The synthesis parameter calculation unit 300B sets a reference image from among the multiple captured images to serve as a basis for projective transformation (step S210A). The reference image can be set (selected) based on image characteristics such as orientation and clarity, but an image captured in a specific order (e.g., the first image captured) may also be used as the reference image. In the examples of FIGS. 9 and 10, image i1 can be set as the reference image for images i1 and i2, as shown in part (a) of FIG. 14.

Once the reference image has been set, the synthesis parameter calculation unit 300B calculates the projective transformation matrices for the images other than the reference image relative to the reference image based on corresponding points between the images (step S210B). In the example of Figures 9 and 10, as shown in part (a) of Figure 14, image i2 (an image other than the reference image) is projected onto the same plane as image i1 (reference image) and a projective transformation matrix is calculated to match corresponding points. Once the projective transformation matrix for image i2 relative to image i1 (reference image) has been calculated, the transformed image i2 is set as the reference image, as shown in part (b) of Figure 14, and the projective transformation matrix for image i3 relative to the transformed image i2 is calculated. In this way, the projective transformation matrices are calculated for all images in the image group while changing the reference image (until step S210C returns YES).

Once the projective transformation matrices for all images have been calculated, the synthesis parameter calculation unit 300B moves, rotates, enlarges or reduces, and deforms each image based on the calculated projective transformation matrix, thereby determining the arrangement of each image in the image group (step S210D). In the following description, it is assumed that images i1-i4 and images i6-i9 form a compositeable image group G1, and images i5 and i10 form a compositeable image group G2. Note that, while each image is moved, rotated, deformed, etc. using the projective transformation matrix when determining the image arrangement described above, the illustrated example conceptually illustrates the arrangement of the images and does not accurately represent the movement, rotation, deformation, etc. of the images.

The image layout determined in step S210D may be an image layout in which overlapping areas of the images overlap (see Figure 15), or an image layout in which the images do not overlap (see Figure 16). In the case of an image layout in which the images do not overlap, the images can be arranged with a gap in the x and y directions as shown in Figure 16, and the orientation of the images can be left unchanged from the orientation determined by the projective transformation matrix (the images are not rotated).

<Image display>
Once the image layout is determined in step S210 (steps S210A to S210D), the server 300 (display control unit 300I, communication control unit 300J) instructs the client 200 to display a group of images that can be combined (step S212). The instruction to the client 200 includes the images to be displayed, information on the layout of the images, and information on the vectorized damage detection results obtained in step S204. In response to the display instruction, the client 200 (display control unit 210C) superimposes the vectorized damage detection results on the images and displays them on the monitor 232 for each group of images that can be combined (step S106). In the example described above, the display control unit 210C displays the image groups G1 and G2 in the image layout shown in FIG. 15 or FIG. 16. FIG. 17 shows the image layout of FIG. 15 in which the cracks in images i4, i5, i9, and i10 are vectorized and superimposed. When displaying, the display control unit 300I and/or the display control unit 210C may perform processing such as surrounding each of the image groups G1 and G2 with a frame, displaying each image group in a different color, or displaying the number of the image group, so that the image groups can be easily identified.

When displaying images in step S106, the image group placement may be inappropriate. For example, as shown in FIG. 18, image group G2, which should be placed below image group G1 (in the -x direction), may be placed to the side of image group G1. In this case, the server 300 (composite parameter calculation unit 300B) determines whether to change the placement of the image groups (step S214). If so (YES in step S214), the process returns to step S212, and the client 200 redisplays image groups G1 and G2 in the changed placement. The determination in step S214 can be made based on a user instruction (e.g., moving image group G2 via the keyboard 242 and/or mouse 244, i.e., changing the relative placement). In the example of FIG. 18, the composite parameter calculation unit 300B and display control unit 300I move image group G2 below image group G1 (in the direction of the arrow) based on a user operation (e.g., dragging with the mouse 244), resulting in the display shown in FIG. 15.

If the relative positioning of images is inappropriate when displayed, it may take time to specify corresponding points. However, the damage diagram creation system 10 allows you to quickly and easily specify corresponding points by changing the positioning in this way.

<Specifying corresponding points>
Once the arrangement of the images has been determined by the processing up to step S214, the server 300 (corresponding point designation unit 300F) designates corresponding points between one of the displayed images and the other of the displayed images (step S216). For example, if the image groups G1 and G2 are arranged and displayed as shown in FIG. 15, point P1a is designated in image group G1 by clicking the mouse 244 or the like (the corresponding point designation operation of step S108), and point P1b, which corresponds to point P1a, is designated in image group G2, as shown in FIG. 19. Corresponding points may be, for example, the start point, end point, or branch point of a crack determined to be "identical," and/or characteristic points such as the end, side, or joint of a component. In this case, the corresponding point designation unit 300F and the display control unit 300I display the designated points P1a and P1b in a distinguishable manner, for example, by connecting the designated points P1a and P1b with a straight line (see FIG. 19), thereby making it easy to understand that the designated points correspond to each other. Similarly, points P2a and P2b, points P3a and P3b, points P4a and P4b, points P5a and P5b, and points P6a and P6b are also designated.

Note that while Figure 19 shows an example in which six corresponding points are specified for each of the image groups G1 and G2, the number of corresponding points that can be specified is not particularly limited. Furthermore, even if the image groups G1 and G2 are arranged and displayed as shown in Figure 16, corresponding points can be similarly specified for images i4, i5 and images i9, i10.

In this way, in the damage diagram creation system 10 according to the first embodiment, the image groups that can be combined (image groups G1 and G2) are displayed for each image group, making it easy to understand which image groups were able to be (automatically) combined and which were not. Furthermore, since it is only necessary to specify corresponding points for the image groups that could not be combined (image groups G1 and G2), there is no need to specify corresponding points for all images, and the specification of corresponding points and the calculation of combination parameters based on the specified corresponding points can be performed quickly and easily.

If the synthesis parameters cannot be calculated accurately using the corresponding points specified as described above, the server 300 (synthetic parameter calculation unit 300B, corresponding point specification unit 300F, display control unit 300I, etc.) and the client 200 (display control unit 210C, etc.) may display a warning message on the monitor 232 to prompt the user to specify the corresponding points again.

<Image layout determination based on specified corresponding points>
Once the corresponding points are designated in step S216, the server 300 (combining parameter calculation unit 300B) calculates combining parameters for each image group that can be combined based on the designated corresponding points (step S218). In the example of FIG. 19, the combining parameter calculation unit 300B calculates a projective transformation matrix for the image group G2 relative to the image group G1 (or a projective transformation matrix for the image group G1 relative to the image group G2) based on the corresponding points P1a to P6b. The image combining unit 300C and the combining parameter calculation unit 300B determine the image arrangement by moving, rotating, transforming, etc., the images constituting the image group G2 (images i5 and i10) using the projective transformation matrix calculated in this manner (step S220). The image arrangement determination in step S220 can be performed using the same procedure as in step S210.

<Combining detection results>
The server 300 (detection result synthesis unit 300E) synthesizes the detection results (detection results) based on the synthesis parameters (projection transformation matrix) calculated in steps S202 and S220 (step S222). For example, suppose different portions (partial overlap) of damage vectors V1 and V2 are detected in images i1 and i2 as shown in parts (a) and (b) of FIG. 20, and the start and end points of the damage vectors V1 and V2 are points P7 to P12. Note that parts (a) and (b) of FIG. 21 are tables showing the start and end points of the damage vectors V1 and V2 corresponding to parts (a) and (b) of FIG. 20, respectively. In this case, the detection results after synthesis are as shown in part (c) of FIG. 20 (image i2 is synthesized by moving, rotating, etc. using the projective transformation matrix), with the start and end points of the damage vector V1 being points P7 and P8, respectively, and the start and end points of the damage vector V2 being points P8 and P12, respectively. Part (c) of FIG. 21 corresponds to part (c) of FIG. 20, and shows the start and end points of the combined damage vectors V1 and V2.

<Combining detection results in overlapping image areas>
When images are acquired so that part of the imaging range overlaps as described above, in the overlapping region of multiple images, the accuracy of synthesis may be degraded, for example, detection results that should be synthesized into one may be multiple due to misalignment of the images. Therefore, the damage diagram creation system 10 can synthesize the detection results with high accuracy by selecting one of the overlapping images in the overlapping region and synthesizing the detection results. Such synthesis is shown in FIG. 22. In the example of FIG. 22, since images i1 and i2 overlap in region OL, image i2, which has a smaller value of the synthesis parameter, is selected in this region OL, and the detection results for image i2 are used for synthesis (see (b) of FIG. 22).

This reduces deformation and misalignment of damage, improving the accuracy of damage detection results and data calculations.

In the case of the projective transformation described above, the value of the synthesis parameter represents the degree of projective transformation (the attitude of the deformation due to the projective transformation), and the greater the amount of coordinate movement due to the projective transformation in the synthesis area, the larger the value of the synthesis parameter.

This will be explained in detail using Figure 29. Consider the case of combining two original images, i1 and i2, in Figure 29(a). Using the method described above, a projective transformation matrix is calculated for i1 using i0 (not shown) as the reference image, and i1 is projectively transformed as shown in Figure 29(b). During this process, specific coordinates P1a, P2a, P3a, and P4a in i1 are moved to P1a', P2a', P3a', and P4a' by projective transformation. Next, to calculate the projective transformation matrix for i2 using i1 after projective transformation as the reference image, corresponding points P1a-P4a and P1b-P4b in each image are specified (Pxa and Pxb are corresponding points, respectively). The projective transformation matrix for i2 is calculated based on these corresponding points, and it is projectively transformed as shown in Figure 29(b). At this time, P1b, P2b, P3b, and P4b in i2 are moved to P1b', P2b', P3b', and P4b' by projective transformation. The image layout is then determined so that the corresponding points P1a' to P4a' in i1 and P1b' to P4b' in i2 coincide with each other.

Here, we calculate how much P1a to P4a of i1 have moved due to the projective transformation. Specifically, we calculate the distance between P1a and P1a' (|P1a - P1a'|), repeat this process for P2a to P4a, and finally take the sum. This is the synthesis parameter for i1. Similarly, for i2, we calculate the distance between P1b and P1b' (|P1b - P1b'|), repeat this process for P2b to P4b, and finally take the sum. This is the synthesis parameter for i2. Using the synthesis parameter calculated in this way, we select the image with the smallest synthesis parameter in the overlapping area of i1 and i2 (i2 in the example in Figure 29(c)), and synthesize them.

As explained above, the synthesis parameter represents the amount of coordinate movement due to projective transformation, so the greater the degree of transformation (degree of deformation), the greater the value of the synthesis parameter. In this embodiment, the synthesis parameter is defined as the sum of the coordinate movement distances, but other calculation methods are acceptable as long as the parameter represents the degree of deformation. For example, one method is to calculate the similarity in shape between the figure formed by connecting P1a to P4a and the figure formed by connecting P1a' to P4a'. In this case, by normalizing the size of the figures to be compared in advance, it is possible to ignore the difference in shape and calculate the synthesis parameter by focusing only on the degree of deformation. In this case, the image with the greater shape similarity is considered to have a smaller degree of deformation (smaller synthesis parameter), and the overlapping area is selected.

The above-mentioned combination of detection results is performed on images stored in the same folder (multiple images divided into groups). In the example shown in Figure 12, the detection results are combined for 10 images stored in subfolder SS1 (detection results are combined for each group).

<Image Composition>
The server 300 (image synthesis unit 300C) synthesizes the images based on the synthesis parameters (projection transformation matrix) calculated in steps S202 and S218 (step S224). Fig. 23 shows the synthesized image G3. Fig. 24 shows an image G3a synthesized by selecting one image in the overlapping area as in Fig. 22. Note that since a damage diagram can be created once the detection results are synthesized, the synthesis of the images may be omitted.

<Post-composite facing correction>
As described above, the damage diagram creation system 10 calculates the projective transformation matrix of other images relative to the reference image to determine the image layout. However, if the reference image is not directly facing the imaging direction, an area that should be rectangular in the combined image may not be rectangular. For example, the frame F of the grid may become trapezoidal in the combined image. In this case, the server 300 (the synthesis parameter calculation unit 300B, the image synthesis unit 300C, etc.) specifies points that form the rectangle (e.g., the four corners of the frame F) based on user operation via the keyboard 242 and/or the mouse 244, and forms a rectangle using these four points through projective transformation. This allows an image in which the subject is directly facing (a front-facing image) to be obtained even after the images are synthesized.

<Mapping of measurement results>
The server 300 (damage mapping unit 300G) may map the damage detection results onto the composite image. Mapping can be performed, for example, by displaying text, graphics, symbols, etc. associated with the detection results on the composite image. The text, graphics, symbols, etc. to be displayed can be selected via the operation unit 240 (keyboard 242 and/or mouse 244). Depending on the selection, the server 300 (damage mapping unit 300G, display control unit 300I, etc.) and the client 200 (display control unit 210C) display the mapped image on the monitor 232. The text, graphics, symbols, etc. may be simplified or emphasized versions of the actual damage, or may be displayed in different formats depending on the type, size, etc. of the damage. The image onto which the measurement results are mapped is stored in the memory unit 220 (damage mapping image 220D in FIG. 3 ) and displayed on the monitor 232 under the control of the display control unit 210C. Damage information may be input into the damage mapping image.

This type of mapping of measurement results may be performed on drawing data (e.g., CAD data, CAD: Computer-Aided Design) containing line drawing information showing the shape of the bridge 1. In this case, if the coordinate system defining the CAD data differs from the coordinate system shown in Figures 23 and 24, etc., coordinate transformation (movement, rotation, mirroring, etc.) is performed according to the relationship between the coordinate systems. Such transformations can be performed by the server 300 (damage mapping unit 300G).

<Result display>
The server 300 (detection result output unit 300H, display control unit 300I, communication control unit 300J, etc.) instructs the client 200 to display the detection results (step S226), and in response to this instruction, the client 200 (display control unit 210C, etc.) displays the detection results on the monitor 232 (step S110). The detection results can be displayed using letters, numbers, symbols, etc., and the detection results for each captured image may be displayed, or a composite detection result may be displayed (see FIGS. 20 and 21). Furthermore, captured images, composite images, damage mapping images, etc. may be displayed together with or instead of the detection results. Furthermore, vectorized information of the detection results (see FIGS. 20 and 21) may be displayed superimposed on the image. The display control unit 210C can select the content and format to be displayed on the monitor 232 in response to operation of the operation unit 240. The display control unit 210C measures the time since the results were displayed and determines whether an output instruction has been issued after the results were displayed. If the display time is long without an output instruction, or if the screen transitions to another screen without an output instruction, it is possible that fraud such as visual copying is being attempted, and such fraud can be detected by measuring the display time and determining whether an output instruction was given. If fraud is detected, measures can be taken, such as stopping the display.

<Result output>
The server 300 (detection result output unit 300H, etc.) determines whether an output instruction operation (e.g., an output instruction operation via the operation unit 240) has been performed for the detection results displayed in step S110 (step S228). Only if an output instruction operation has been performed (YES in step S228), the process proceeds to step S230, instructing the client 200 to output the detection results (detection results for each image and the combined detection results), and the client 200 (file management unit 210B) outputs the detection results in response to the output instruction (step S112). In the example of FIG. 12, the detection results for the images stored in subfolder SS1 are stored in the same subfolder SS1 as the images (the detection results are output in association with the group; see FIG. 25). The detection results may be output in the same format as the drawing data (e.g., CAD data) containing line drawing information indicating the shape of the bridge 1 (subject). Alternatively, the detection results may be printed using a printer (not shown).

In this way, in the first embodiment, the detection results are stored in the same folder as the image, which clarifies the correspondence between the input image and the combined detection results, making it easy to manage and use the image and detection results. The detection results are only output when an output operation instruction is received (YES in step S228). If no output operation instruction is received, identification information is assigned to the image (step S232), and the assigned identification information is notified to client 200, which then stores the image in the same folder as the folder in which the image is stored (subfolder SS1 in the examples of Figures 12 and 25).

Once the detection results or identification information have been output, the server 300 determines whether processing has been completed for all folders (step S234). If the determination is positive, the server 300 notifies the client 200 of completion in step S236 (step S236) and terminates processing. If processing has not been completed for all folders, the server 300 returns to step S202 and repeats steps S202 to S234 for the other folders.

As described above, the damage diagram creation system 10 according to the first embodiment can detect damage with high accuracy based on multiple images acquired by photographing a subject in sections.

Second Embodiment
In the first embodiment described above, a damage diagram creation system 10 including a server 300 and a client 200 has been described. In the second embodiment, a damage diagram creation device 20 will be described. FIG. 26 is a diagram showing the configuration of the damage diagram creation device 20. The damage diagram creation device 20 is composed of a digital camera 100 and a device main body 500. Since the configuration of the digital camera 100 is the same as that of the digital camera 100 in the first embodiment, the same reference numerals are used and detailed description thereof will be omitted. Note that the digital camera 100 may be configured integrally with the device main body 500.

<Configuration of the damage diagram creation device>
The damage diagram creation system 10 according to the first embodiment includes a server 300 and a client 200, and the server 300 performs the majority of the processing, such as damage detection and synthesis. However, in the damage diagram creation device 20 according to the second embodiment, a processing unit 510 of the device main body 500 performs the processing. Specifically, the processing unit 510 has the functions of the client 200 shown in FIG. 2 and the functions of the server 300 shown in FIG. 4. The storage unit 520 stores information similar to that of the storage unit 220 according to the first embodiment (see FIG. 3). The configuration and function of the operation unit 540 (keyboard 542, mouse 544) are similar to those of the operation unit 240 (keyboard 242, mouse 244) according to the first embodiment. The configuration and function of the display unit 530 (monitor 532) are similar to those of the display unit 230 (monitor 232) according to the first embodiment. In the damage diagram creation device 20, a device (information terminal) such as a personal computer, tablet terminal, or smartphone that inputs images and performs processing, such as damage detection and synthesis, can be used as the device main body 500.

<Processing for creating damage diagrams>
The processing in the damage diagram creation device 20 (processing of the damage diagram creation method according to the present invention) is similar to the flowcharts in Figures 5, 6, and 13. For example, a plurality of images acquired by photographing an object in separate images are input, a synthesis parameter (projection transformation matrix) for synthesizing the plurality of images is calculated based on corresponding points between the images, damage to the object is detected from the images constituting the plurality of images, and the detection results for the plurality of images are synthesized based on the synthesis parameter. Such processing is performed by the processing unit 510 of the device main body 500. Note that, while communication is performed between the client 200 and the server 300 in the first embodiment, communication is performed within the device main body 500 in the damage diagram creation device 20.

In the damage mapping system 20 according to the second embodiment, damage is detected from the captured images before compositing, just as in the damage mapping system 10 according to the first embodiment. As in the first embodiment, images with small values of the compositing parameter are used for the images in the overlapping region. This makes it possible to suppress deformation and positional shifts of damage, thereby improving the accuracy of damage detection results and associated data calculations.

As a result, damage detection performance does not deteriorate due to image quality degradation in overlapping image areas, making it possible to detect damage with high accuracy based on multiple images acquired by capturing the subject in separate images.

In both the first and second embodiments, the images to be used may be selected based on the relative positions of the overlapping area and the damage (cracks, etc.). For example, as shown in Figure 27(a), there may be damage in the overlapping area, and the damage in image i2 straddles the overlapping area from inside to outside the area, while the damage in image i1 does not straddle the area. In such a case, as shown in Figure 27(b), one image i2 may be used as the image of the overlapping area (regardless of the value of the synthesis parameter). In other words, if one image i2 has damage extending from inside to outside the overlapping area, while the other image i1 has damage that is contained within the overlapping area (but not extending), one image i2 may be used as the image of the overlapping area (regardless of the value of the synthesis parameter). This ensures continuity of the damage and improves the accuracy of damage detection results and data calculation. Furthermore, as shown in Figure 28(a), if damage is located both inside and outside the overlapping area in both image i2 and image i1, it is recommended to use image i2, which has the most damage located inside and outside the overlapping area, in the overlapping area, as shown in Figure 28(b). This reduces the amount of damage that loses continuity, improving the accuracy of damage detection results and data calculations.

Furthermore, for overlapping areas where no damage exists, an image with better image quality may be used as the image for the overlapping area, regardless of the value of the synthesis parameter. In this case, the quality of image quality can be determined based on image quality assessment information such as the appropriateness of brightness, the appropriateness of saturation, and the degree of focus. This not only improves the accuracy of damage detection results and associated data calculation, but also improves the quality of the synthesized image, making it possible to provide users with better damage maps. It is also preferable to determine the quality of image quality based on two or more pieces of assessment information, rather than determining it based on a single piece of image quality assessment information (such as the appropriateness of brightness, the accuracy of saturation, or the degree of focus). This makes it possible to provide users with better damage maps. In this way, for overlapping areas that do not fall into the above cases, an image with a lower synthesis parameter value is selected and synthesized, and for overlapping areas that do fall into the above cases, an image with the above conditions is selected and synthesized, thereby improving the accuracy of damage detection results and associated data calculation, and achieving the various benefits described above.

Third Embodiment
A third embodiment will be described with respect to the processing in the damage diagram creation devices 10 and 20 described above.

Here, the configuration of the damage diagram creation system 10 in the first embodiment will be used for explanation, but it can also be applied to the configuration of the damage diagram creation device 20 in the second embodiment. Furthermore, the processing in the damage diagram creation device is similar to the flowcharts in Figures 5 and 6, but the image layout determination processing in steps S210 and S220 is different.

Specifically, the image layout determination process in the third embodiment will be explained using Figures 30 and 31. As an example, we will explain a method for creating a damage diagram from a group of images taken from below of an arched bridge, as shown in Figure 31(a).

<Determining image layout>
30 is a flowchart showing the details of the image layout determination process in steps S210 and S220 in FIG. 5. The synthesis parameter calculation unit 300B sets a reference image from among the multiple captured images to serve as the basis for projective transformation (step S210E). The reference image can be set (selected) based on image characteristics such as orientation and clarity, but an image in a specific shooting order (e.g., the first image captured) may also be used as the reference image. In the example of FIG. 31, image i1 can be set as the reference image for images captured in the order i1, i2, i3, ...

Once the reference image has been set, the synthesis parameter calculation unit 300B calculates a projective transformation matrix for the reference image of the image adjacent to the reference image based on corresponding points between the images (step S210F). In the example of Figure 31, as shown in Figure 31(b), image i2 (the image adjacent to the reference image) is projected onto the same plane as image i1 (the reference image) and a projective transformation matrix is calculated to match corresponding points.

Once the projective transformation matrix for image i2 has been calculated, the synthesis parameter calculation unit 300B calculates synthesis parameters for i2 based on the calculated projective transformation matrix. The synthesis parameter here is a value that represents the degree of projective transformation (the degree of deformation due to projective transformation), and the value of the synthesis parameter increases as the angle between the captured image plane (a plane normal to the optical axis) and the projection plane increases. Specifically, as explained in the first embodiment, the synthesis parameter can be calculated from the amount of movement of the coordinates of each corresponding point due to projective transformation. Note that while in the first embodiment, the synthesis parameters were calculated focusing on the overlapping area between i1 and i2, in the third embodiment, it is desirable to calculate the degree of projective transformation focusing on the entire image i2, and therefore the synthesis parameters may be calculated from the amount of movement of the coordinates of the four vertices of image i2.

After calculating the synthesis parameters for image i2, the synthesis parameter calculation unit 300B determines whether or not to permit projective transformation of image i2 based on the calculated synthesis parameters (step S210H). Here, if the calculated synthesis parameters are equal to or less than a predetermined value, projective transformation is permitted, and if they are greater than the predetermined value, projective transformation is not permitted.

If projective transformation is permitted, the synthesis parameter calculation unit 300B performs projective transformation on image i2 based on the calculated projective transformation matrix (step S210I) and arranges it next to image i1 (step S210J). At this time, the images may be arranged so that overlapping areas of the images overlap, or so that the images do not overlap.

If projective transformation is not permitted, the synthesis parameter calculation unit 300B places image i2 next to image i1 without performing projective transformation on image i2 (step S210J). In this case, the images are arranged so that overlapping areas between the images do not overlap.

Image i2 is then set as the reference image, and the above-described process (steps S210E to S210J) is similarly performed on image i3, which is adjacent to image i2. This process is repeated for all images in the image group (until step S210K returns YES), while changing the reference image.

In the example of Figure 31, because the bridge is arch-shaped, as projective transformation is repeated using image i1 as the initial reference image, the degree of deformation due to projective transformation gradually increases (because the images are projected onto the same plane as image i1). Therefore, as shown in Figure 31(b), images i2 and i3 can be projected and combined without any problems, but images i4 and beyond experience a greater degree of deformation, making them unsuitable for use as damage diagrams. Therefore, by setting an appropriate value in step S210H of the third embodiment, it is possible to disallow projective transformation of image i4 (Figure 31(c)). In this case, image i4 cannot be combined with image i3, but each image can maintain an image quality (shape) suitable for use as a damage diagram.

As shown in Figure 31(c), if projective transformation is not permitted in step S210H, the image may be displayed in a manner that allows the user to recognize this, such as by adding a dotted line between images i3 and i4.

In addition, when determining whether to permit projective transformation in step S210H, the determination may be made based on the degree of orientation of the captured image. For example, if the captured image is positioned in a normal orientation and projective transformation would result in deformation in a direction that is not normal, projective transformation may not be permitted. However, if the captured image is not positioned in a normal orientation to begin with and projective transformation would result in deformation in a direction that is normal, a determination may be made to permit projective transformation.

As described above, according to the third embodiment, it is possible to create appropriate damage diagrams even for structures that are not on the same plane, such as arched bridges.

The above describes an embodiment of the present invention, but the present invention is not limited to the above-described aspects, and various modifications are possible without departing from the spirit of the present invention.

200 Client 210 Processing section 210A Image input section 220 Storage section 220A Captured image 300 Server 300A Image acquisition section 300B Synthesis parameter calculation section 300C Image synthesis section 300D Damage detection section 300E Detection result synthesis section

Claims (10)

inputting a plurality of images acquired by dividing and photographing an object so that some areas of the object overlap;
calculating synthesis parameters for synthesizing the plurality of images based on corresponding points between the images;
detecting damage to the subject from the images constituting the plurality of images;
synthesizing the detection results for the plurality of images based on the synthesis parameters;
and
In the combining step, in the overlapping region, one image is selected from the plurality of overlapping images and combined, and the selected one image has a smaller value of the combining parameter than the other images;
In the combining step, if there is no damage in the overlapping area, the selected one image has a better judgment result of image quality judgment information than the other images.
A damage diagram creation method characterized by: 2. The damage diagram creation method according to claim 1 , wherein the image quality judgment information is any one of brightness suitability, saturation suitability, and focusing suitability. 3. The damage diagram creation method according to claim 2 , wherein the judgment result is determined based on at least two pieces of judgment information. The damage diagram creation method described in claim 1, characterized in that, in the combining step, when an image in which damage is contained within the overlapping area and an image in which damage straddles both the inside and outside of the overlapping area are combined, the selected image is the image in which damage straddles both the inside and outside of the overlapping area. The damage diagram creation method described in claim 1, characterized in that in the combining step, when images in which damage straddles both the inside and outside of an overlapping area are combined, the selected image is an image in which there is a greater number of damages straddling both the inside and outside of the overlapping area than the other images. inputting a plurality of images acquired by dividing and photographing an object so that some areas of the object overlap;
calculating synthesis parameters for synthesizing the plurality of images based on corresponding points between the images;
detecting damage to the subject from the images constituting the plurality of images;
synthesizing the detection results for the plurality of images based on the synthesis parameters;
and
In the combining step, in the overlapping region, one image is selected from the plurality of overlapping images and combined, and the selected one image has a smaller value of the combining parameter than the other images;
In the combining step, when combining an image in which damage is contained within the overlapping area and an image in which damage straddles the inside and outside of the overlapping area, the selected one image is an image in which damage straddles the inside and outside of the overlapping area.
A damage diagram creation method characterized by: inputting a plurality of images acquired by dividing and photographing an object so that some areas of the object overlap;
calculating synthesis parameters for synthesizing the plurality of images based on corresponding points between the images;
detecting damage to the subject from the images constituting the plurality of images;
synthesizing the detection results for the plurality of images based on the synthesis parameters;
and
In the combining step, in the overlapping region, one image is selected from the plurality of overlapping images and combined, and the selected one image has a smaller value of the combining parameter than the other images;
In the combining step, when images having damage straddling both inside and outside of an overlapping area are combined, the selected image is an image having a larger number of damages straddling both inside and outside of the overlapping area than the other images.
A damage diagram creation method characterized by: an image input unit that inputs a plurality of images obtained by dividing and photographing a subject so that some areas overlap;
a synthesis parameter calculation unit that calculates synthesis parameters for synthesizing the plurality of images based on corresponding points between the images;
a damage detection unit that detects damage to the subject from the images that constitute the plurality of images;
a detection result synthesis unit that synthesizes the detection results for the plurality of images based on the synthesis parameters;
and
The detection result synthesis unit selects one image from the plurality of overlapping images in the overlapping area and synthesizes the selected image, and if the selected image has a smaller value of the synthesis parameter than the other images and there is no damage in the overlapping area, the selected image has a better judgment result of image quality assessment information than the other images.
A damage diagram creation device characterized by: A damage diagram creation system including a server and a client,
The client:
an image input unit for inputting a plurality of images obtained by dividing and photographing a subject so that some areas of the image overlap;
The server
an image acquisition unit that acquires the plurality of images from the client;
a synthesis parameter calculation unit that calculates synthesis parameters for synthesizing the acquired plurality of images based on corresponding points between the images;
a damage detection unit that detects damage to the subject from the images that constitute the plurality of images;
a detection result synthesis unit that synthesizes the detection results for the plurality of images based on the synthesis parameters;
a detection result output unit that outputs the detection result to the client;
Equipped with
The detection result synthesis unit selects one image from the plurality of overlapping images in the overlapping area and synthesizes the selected image, and if the selected image has a smaller value of the synthesis parameter than the other images and there is no damage in the overlapping area, the selected image has a better judgment result of image quality assessment information than the other images.
A damage diagram creation system characterized by: A recording medium on which computer-readable code for a program for causing a computer to execute the damage diagram creation method according to any one of claims 1 to 7 is recorded.