JP7638778B2 - Information processing device, information processing method, and program - Google Patents

Description

The present disclosure relates to an information processing device, an information processing method, and a program.

The damage level of a structure is evaluated, for example, by taking into account the scale of the damage itself that is the subject of the damage level evaluation, redundancy according to the allowable damage for each part of the structure, and the progression of the damage due to aging. Such damage level evaluation results can be used, for example, to determine the soundness of the structure and to develop a repair plan for the damage. For example, Patent Document 1 discloses an example of a technology related to determining the soundness of a structure. Also, Patent Document 2 discloses an example of a technology related to developing a repair plan for damage to a structure.

Patent No. 6454793 Patent No. 6472894

On the other hand, as multiple damages that have occurred in a structure progress, the multiple damages may come into contact with each other and become larger. In such cases, even if the individual damages that are independent of each other are not considered to affect the soundness of the structure in determining its soundness, it is possible that the multiple damages may come into contact with each other in the future and become larger damage, which may affect the soundness of the structure.

In consideration of the above problems, the present invention aims to make it possible to record information for assessing the degree of damage to a structure in a more suitable manner.

The information processing device according to the present invention is characterized by comprising: an aggregation means for aggregating, for each second partial region larger than the first partial region, first information on the degree of damage of a portion of the structure corresponding to the corresponding first partial region associated with the first partial region, as second information based on at least one of density and continuity in the second partial region; and a recording means for recording, in association with each other, the second information corresponding to the second partial region and the first information corresponding to each of the first partial regions included in the second partial region.

The present invention makes it possible to record information for assessing the degree of damage to a structure in a more suitable manner.

FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device. FIG. 2 is a functional block diagram showing an example of a functional configuration of an information processing device. FIG. 11 is a diagram for explaining an example of damage level information. FIG. 11 is a diagram for explaining an example of damage level information. FIG. 11 is a diagram for explaining an example of damage level information. 11 is a diagram showing an example of a presentation mode of damage level information. FIG. FIG. 13 is a diagram for explaining a method of aggregating damage level information. FIG. 13 is a diagram illustrating an example of a table related to control of an operation pattern of the aggregation process. FIG. 13 is a diagram showing an example of a correspondence relationship before and after application of an aggregation process. FIG. 13 is a diagram showing an example of a damage level table. FIG. 11 is a diagram showing an example of a result of aggregating processing of damage level information. FIG. 11 is a diagram showing an example of a result of aggregating processing of damage level information. 11 is a flowchart showing an example of processing by an information processing device. 11 is a flowchart showing an example of processing by an information processing device. 11 is a diagram showing an example of a presentation mode of damage level information. FIG. 11 is a diagram showing an example of a presentation mode of damage level information. FIG. 13 is a diagram showing an example of a UI related to presentation of damage level information. FIG. 10 is a flowchart showing another example of the process of the information processing device. 10 is a flowchart showing another example of the process of the information processing device. FIG. 13 is a diagram showing an example of a display screen relating to presentation of damage level information. 10 is a flowchart showing another example of the process of the information processing device. 10 is a flowchart showing another example of the process of the information processing device.

A preferred embodiment of the present disclosure will be described in detail below with reference to the attached drawings. Note that in this specification and drawings, components having substantially the same functional configuration are designated by the same reference numerals to avoid redundant description.

<Hardware Configuration>
An example of a hardware configuration of an information processing device according to an embodiment of the present disclosure will be described with reference to FIG. 1A. The information processing device 100 according to this embodiment includes a central processing unit (CPU) 101, a random access memory (RAM) 103, and a read only memory (ROM) 102. The information processing device 100 also includes an auxiliary storage device 104 and a network I/F 107. The information processing device 100 may also include at least one of an input device 105 and an output device 106. The CPU 101, the RAM 102, the ROM 103, the auxiliary storage device 104, the input device 105, the output device 106, and the network I/F 107 are each connected to each other via a bus 108.

The CPU 101 is a central processing unit that controls various operations of the information processing device 100. For example, the CPU 101 may control the operation of the entire information processing device 100.
The RAM 102 is a main storage memory for the CPU 101, and is used as a work area or a temporary storage area for developing various programs.
The ROM 103 stores a program (eg, BIOS) for the CPU 101 to control the operation of the information processing device 100 .

The auxiliary storage device 104 stores various data and programs such as the OS (Operating System), which is the basic software, and various applications. The auxiliary storage device 104 can be realized, for example, by a non-volatile memory such as a HDD (Hard Disk Drive) or a SSD (Solid State Drive).

The network I/F 107 is an interface that is connected to a specific network (e.g., a LAN or the Internet) and communicates with external devices via that network.

The input device 105 is a device for receiving instructions from a user, and may be realized by, for example, an operation device such as a pointing device such as a mouse, a keyboard, or a touch panel.
The output device 106 is a device for presenting various types of information to a user. The output device 106 can be realized by a display device, such as a display, that presents information to a user by displaying various types of display information, a screen, or the like.

Note that the above-described configuration is merely an example and does not necessarily limit the hardware configuration of the information processing device 100. As a specific example, some of the components of the information processing device 100 described above (for example, at least one of the input device 105 and the output device 106) may be externally attached to the outside of the information processing device 100.
In addition, the functional configuration shown in Figure 1B and the processes shown in Figures 7A, 7B, 10A, 10B, 12A, and 12B are realized by CPU 101 expanding a program stored in ROM 103 or auxiliary storage device 104 into RAM 102 and executing it.

<Functional configuration>
An example of a functional configuration of an information processing device according to an embodiment of the present disclosure will be described with reference to Fig. 1B. The information processing device 100 according to the present embodiment includes a damage level determination unit 111, a correction processing unit 112, an aggregation processing unit 113, a storage unit 114, and a display control unit 115.

The damage level determination unit 111 performs image analysis on the captured image of the structure to be inspected (hereinafter also referred to as the "inspection image"), and determines the degree of damage (hereinafter also referred to as the "damage level") in each part of the structure based on the result of the image analysis. As a specific example, the damage level determination unit 111 may divide the inspection image into a plurality of partial regions, perform image analysis for each divided partial region, detect damage in the part of the structure corresponding to the partial region, and determine the damage level in the part based on the detection result. The damage level determination unit 111 may also determine the damage level in each part of the target structure based on the inspection images at a plurality of time points including past time points. In this manner, the damage level determination unit 111 associates information (hereinafter also referred to as the "damage level information") corresponding to the determination result of the damage level of the part corresponding to the partial region with each partial region divided from the inspection image. The damage level determination unit 111 may hold the damage level information corresponding to each partial region divided from the inspection image by storing it in a predetermined storage area.

The correction processing unit 112 acquires information on redundancy corresponding to the damage that can be tolerated for each part from the structural information when the target structure was designed. The correction processing unit 112 also acquires information on the progression of damage based on a comparison of the aging changes of each damage occurring in the target structure. Then, the correction processing unit 112 performs a correction process on the damage level information corresponding to the damage level assessment result by the damage level assessment unit 111 based on the acquired information on redundancy and progression.

The aggregation processing unit 113 aggregates the damage level information associated with each of the multiple partial regions divided from the inspection image into damage level information for a new partial region that is larger than the multiple partial regions and that includes the multiple partial regions.
At this time, the aggregation processing unit 113 may determine the range (the range of a new partial area including a plurality of partial areas) in which to aggregate the damage level information for each partial area divided from the inspection image, based on desired conditions related to the aggregation of damage level information (hereinafter also referred to as "aggregation processing conditions"). Note that information related to the aggregation processing conditions may be stored in a predetermined storage area (for example, the storage unit 114 described later).
In addition, when aggregating damage level information associated with each of multiple partial areas divided from the inspection image, the aggregation processing unit 113 may take into account the density of damage in each partial area and the continuity of damage in each partial area when determining the damage level corresponding to the new partial area.
Each of the multiple partial regions divided from the inspection image corresponds to an example of a "first partial region", and the damage level information corresponding to the first partial region corresponds to an example of the "first information". Moreover, the new region that includes the multiple first partial regions and is larger than the first partial region corresponds to an example of a "second partial region", and the damage level information corresponding to the second partial region corresponds to an example of the "second information".

The aggregation processing unit 113 associates the damage level information for each partial area (first partial area) divided from the inspection image with the aggregated damage level information corresponding to a new area (second partial area) including multiple partial areas, and stores the associated information as aggregated damage level information in the storage unit 114. At this time, the aggregation processing unit 113 may record the damage level information for each partial area divided from the inspection image and the aggregated damage level information corresponding to the new area including multiple partial areas by assigning them to different hierarchies.

The process of detecting the damage level by the damage level determination unit 111, the process of correcting the damage level detection result by the correction processing unit 112, and the process of aggregating the damage level information by the aggregation processing unit 113 will be described in detail later.

The memory unit 114 is a schematic representation of a memory area that stores various types of information and thereby holds the information. For example, the memory unit 114 may store aggregated damage level information output from the aggregation processing unit 113 and thereby hold the aggregated damage level information. The memory unit 114 may also store other information or data in addition to the aggregated damage level information. As a specific example, the memory unit 114 may store information related to the aggregation processing conditions described above and structure information described below.

The display control unit 115 presents desired information to the user by displaying the information on a predetermined output unit (e.g., a display, etc.). For example, the display control unit 115 may present the damage level information of the target structure to the user by displaying the damage level information on a predetermined output unit based on the aggregated damage level information stored in the storage unit 114. At this time, the display control unit 115 may control the display mode of the damage level information in response to a specification of a condition related to the display of the damage level information. The display control unit 115 may also display information on the target structure (hereinafter also referred to as "structure information") on a predetermined output unit in association with the damage level information. The structure information may be stored in a predetermined storage area such as the storage unit 114. An example of the display mode of the damage level information based on the control of the display control unit 115 will be described in detail later with reference to FIG. 2, FIG. 3A, and FIG. 3B.

<Damage information>
An example of the damage level information according to the present embodiment will be described with reference to Figs. 2, 3A, and 3B, focusing in particular on a method for generating the damage level information.

First, FIG. 2 will be described. FIG. 2(a) shows an example of an inspection image 201 based on the imaging results of a target structure. The grid lines labeled 202 show a schematic boundary dividing the inspection image 201 into a number of partial regions. Also, 203 to 206 each show a schematic example of various types of damage that have occurred to the target structure, which have been captured as a subject in the inspection image 201.

2B shows an example of a conceptual diagram in which information is superimposed on the inspection image 201 shown in FIG. 2A, which represents the damage level of the part of the structure corresponding to each partial region divided by the boundary 202, in numerical form. The numerical values 1 to 3 superimposed on each partial region are information indicating the level of damage of the part of the structure corresponding to the partial region, and the higher the numerical value, the higher the degree of damage. In addition, the damage level of the part of the structure corresponding to each partial region may be determined based on the degree of damage detected (e.g., the number, length, size, thickness, intersection state, etc. of the damage) after detecting damage present in the partial region. Note that the process related to the damage level determination can be performed using existing technologies such as a trained model built based on machine learning and image analysis, so a detailed description will be omitted.

Figure 2(c) shows an example of a conceptual diagram in which information expressing the degree of progression of each damage (in other words, the degree of growth of the damage) in alphabetical form is superimposed on the inspection image 201 on which the information showing the degree of damage shown in Figure 2(b) is superimposed. For example, in the example shown in Figure 2(c), the degree of progression of the damage for each partial region is shown as subscripts a to c to the numerical value showing the degree of damage. Of the alphabetical subscripts showing the degree of progression of the damage, a indicates the state in which the damage is most progressive, followed in order by b, c.

The degree of progression of each damage can be determined, for example, by comparing the inspection image 201 corresponding to the imaging result of the target structure with an inspection image corresponding to the imaging result of the target structure obtained in the past. As a specific example, the progression of the damage can be determined by comparing data on the length, thickness, etc. of each detected damage between the inspection images to be compared. Note that the detected damage is, for example, polyline data information having thickness information, so that the positions and number of segments of the polyline data do not necessarily match when comparing the target images. Therefore, for example, the center of gravity position, etc. of each polyline data can be calculated, and the damage can be associated between the images based on the center of gravity position, etc., to compare information on the length, thickness, etc. of the polyline data.

For example, as shown in FIG. 2B, the damage level of the damage 204 is determined to be "2" based on the length and thickness of the damage. On the other hand, the progression of the damage 204 is determined to be greater based on the results of a comparison over time between the inspection image 201 and inspection images acquired by capturing images of the same location of the target structure in the past. Therefore, the progression of the damage 204 is determined to be "a", and "2a" is set as the damage level information 208 in FIG. 2C. Similarly, "1c" and "3a" are set as the damage level information 207 and 209 for the damages 203 and 205. Furthermore, since no progression was observed for the damage 206, the damage level information 210 is set to "1", indicating the damage level, and no information indicating progression is set.

Next, FIG. 3A will be described. FIG. 3A(a) shows a schematic example of a map of redundancy in the range of the target structure captured as the inspection image 201. Redundancy is an index showing characteristics related to allowable damage in parts of a structure, and damage occurring in areas with low redundancy is evaluated as having a higher degree of damage. Information on redundancy can be obtained, for example, from structural information at the time of designing the structure (for example, information on the material and characteristics of each part of the structure). In FIG. 3A(a), R1 to R4 are set as information showing redundancy for each partial region. This information showing redundancy shows that R1 shows the lowest redundancy state, followed by R2, R3, and R4, in that order.

Figure 3A (b) shows an example of a correction table 305 that shows the correction values that are added to the results of the damage level determination based on the inspection image, based on redundancy and progression. Setting values 306 to 309 indicate redundancy values. Setting values 310 to 312 indicate damage progression values. Setting value 313 is a value that means that there is no progression of damage.

Each of the correction values 314 to 316 indicates an example of a correction value that is added according to redundancy and progression. As a specific example, the correction value 314 is a correction value that is added to the damage degree judgment result when the redundancy is "R1" indicated as the set value 306 and the progression is "a" indicated as the set value 310, and "+2" is set as the correction value. Similarly, the correction value 315 is a correction value that is added to the damage degree judgment result when the redundancy is "R3" indicated as the set value 308 and the progression is "a" indicated as the set value 310, and "+1" is set as the correction value. Furthermore, the correction value 316 is a correction value that is added to the damage degree judgment result when the redundancy is "R4" indicated as the set value 309 and the progression is "b" indicated as the set value 311, and "0" is set as the correction value.

Next, FIG. 3B will be described. FIG. 3B(c) is a conceptual diagram that shows an example of the result of applying correction based on the redundancy map shown in FIG. 3A(a) and the correction table shown in FIG. 3A(b) to the damage level information that takes into account the progression of damage shown in FIG. 2(c). In the example shown in FIG. 3B(c), the numerical value presented for each partial region indicates the level of damage level, and a circle surrounding the numerical value indicates that the damage in the target partial region is progressive.

Figure 3B(d) shows an example of a case where the image of damage is excluded from the conceptual diagram shown in Figure 3B(c) and damage level information is the main subject of presentation. That is, in the example shown in Figure 3B(d), the damage level information shown in Figure 2(c) is corrected based on the correction table shown in Figure 3A(b) and the result is presented as information indicating the damage level.

It is also possible to judge the soundness of the target structure based on the damage degree information for each partial region shown in FIG. 3B(d). As a specific example, when the damage degree for each partial region exceeds a threshold, it is possible to judge that some effect on the soundness of the part of the structure corresponding to the partial region or the entire structure may become apparent. On the other hand, with such a method, for example, when the damage degree for each partial region is less than the threshold, but there are multiple partial regions in a closer range that show a damage degree less than the threshold, it may be difficult to judge that some effect on the soundness may become apparent. In addition, in a situation where a large amount of damage degree information that has little effect on the soundness judgment is presented, for example, it may be time-consuming to check each piece of damage degree information, and as a result, it may be assumed that the work related to the soundness judgment becomes more cumbersome.

The above-mentioned method of determining the damage level information for each partial region is merely one example, and the size and granularity of the damage level, redundancy, and progression, and the method of correction based on redundancy and progression for the damage level determined from the image are not limited to the above-mentioned example. Therefore, in this embodiment, the following explanation will be given on the premise that for each partial region in the inspection image corresponding to the imaging results of the structure, there is information regarding redundancy and information regarding the level of damage level that takes into account the presence or absence of progression, as shown as a conceptual diagram in Figure 3B (d).

Here, a specific example of a presentation mode of damage level information will be described with reference to Fig. 4. In the example shown in Fig. 4, damage level information according to the damage level determination result for each partial region is presented for a part 402 of a pier of a bridge structure 401.
Region 403 indicates a region corresponding to a portion 402 of a pier of bridge structure 401. Partial region 405 indicates a partial region that is the subject of damage degree judgment based on an inspection image in accordance with the imaging result of portion 402. Boundary line 404 indicates a boundary line that divides region 403 into a plurality of partial regions 405.
The boundary line 406 indicates the boundary line on the redundancy map of the bridge structure 401. In the example shown in FIG. 4, the area above the boundary line 406 corresponds to the area where the areas of "R1" and "R2" with lower redundancy are combined, and the area below the boundary line 406 corresponds to the area where the areas of "R3" and "R4" with higher redundancy are combined. Therefore, in the following, for the sake of simplicity, the redundancy will be described assuming that one of two patterns, "low area" and "high area", is applied. Similarly, for the progression, one of two patterns is applied depending on whether or not there is progression, which is indicated by whether or not a circle is combined with a numerical value indicating the level of damage.
As a specific example, the damage level information 407 indicates that the damage level determination result is "2", indicating that the damage is not progressing. As another example, the damage level information 408 indicates that the damage level determination result is "1", indicating that the damage is progressing.
Further, the partial region 409 is a schematic representation of a region that aggregates a plurality of partial regions 405. For example, in the example shown in Fig. 4, the partial region 409 corresponds to a region surrounded by a thin solid line, and is a schematic representation of a region that aggregates nine (3 x 3) partial regions 405. In this case, the partial region 405 corresponds to an example of a "first partial region" divided from the inspection image (captured image of the target structure), and the partial region 409 corresponds to an example of a "second partial region" that is larger than the first partial region.

Before the partial areas (partial area 405) divided from the inspection image are aggregated, information including the progressive degree of damage and redundancy map boundaries, as exemplified by area 403, may be presented for a series of parts of structural part 401 including part 402.
In addition, in the example shown in Fig. 4, partial regions 405 in which no damage was detected are excluded from the presentation of damage level information. That is, in the example shown in Fig. 4, partial regions 405 in which no damage level information is presented indicate regions in which no damage was detected or regions in which no target structure exists.

<Aggregation process>
The damage level information aggregation process will be outlined below with reference to FIGS. 5A to 5D, 6A, and 6B.
First, FIG. 5A will be described. FIG. 5A is a diagram for explaining an overview of a method for aggregating damage level information. In the example shown in FIG. 5A, an aggregation process is applied to nine (3×3) partial regions 405 divided by a boundary line 404. As a result, a partial region 501 in which nine (3×3) partial regions 405 are aggregated and new damage level information 504 indicating the damage level corresponding to the partial region 501 are shown. In the following description, as in the example shown in FIG. 4, the partial region in which damage level information is presented is assumed to be a partial region in which damage has been detected, and various processes can be performed based on the number of partial regions in which the damage has been detected and the continuity of the arrangement.

Next, Fig. 5B will be described. Fig. 5B is a diagram showing an example of a control table 505 related to control of the operation pattern of the aggregation process. In the control table 505, the pattern of the aggregation process is managed according to a combination of the characteristics of redundancy and damage progression in the partial area to be aggregated and the density and continuity of damage present in the partial area. Specifically, the density and continuity of damage are determined based on the number and order of partial areas in which damage level information is presented in the partial area to be aggregated.
The conditions 506 and 507 relating to redundancy are determined according to, for example, whether or not a partial area with low redundancy is included in the partial areas to be aggregated.
In addition, the conditions 508 and 509 relating to the progression of damage are distinguished according to, for example, whether or not progressive damage is present in the partial area that is the subject of the aggregation process (whether or not a circled number indicating the level of damage is present).
In addition, the conditions 510 and 511 relating to the density and continuity of damage are determined according to, for example, whether the damage present in the partial area to be subjected to the aggregation process is dense or not (e.g., whether there are five or more damages) and whether there is continuity or not (e.g., whether there are three or more adjacent damages).

Correction values 512, 513, and 514 show examples of increase/decrease values applied to the level of damage of the target partial region in each combination of conditions. The increase/decrease value applied to the level of damage may be controlled according to the level of damage of the target partial region. For example, correction values 515 and 516 show examples of increase/decrease values applied when the level of damage of the target partial region is "3 or more" and when it is "less than 3" in each combination of conditions, respectively.

Next, FIG. 5C will be described. FIG. 5C is a diagram showing an example of the correspondence relationship of partial areas in which damage level information is set before and after the application of the aggregation process. In FIG. 5C, the area 521 corresponds to an example of a state before the aggregation process of the damage level information is applied, and is divided into 6×6 partial areas, and the damage level is determined for each of the partial areas. In contrast, the area 522 corresponds to an example of a state after the aggregation process of the damage level information is applied, and the damage level determination results for each of the partial areas in which the area 521 is divided into 6×6 are aggregated in a partial area in which the range corresponding to the area 521 is divided into 2×2. In addition, each of the dashed borders 523 and 524 is a schematic diagram showing a border indicating the boundary between areas with different redundancy. In the example shown in FIG. 5C, the upper and left sides of the border 523 in the area 521 indicate areas with low redundancy. Similarly, the upper and left sides of the border 524 in the area 522 indicate areas with low redundancy. Additionally, the numbers shown outside the frames of areas 521 and 522 (i.e., on the upper and left sides) are numbers assigned as positional indexes when referencing damage level information according to the damage level judgment results for each partial area.

Next, we will explain Figure 5D. Figure 5D shows an example of a management table (hereinafter also referred to as a "damage level table") that stores damage level information corresponding to areas 521 and 522 shown in Figure 5C.

Specifically, the damage level table 525 corresponds to the area 521 , and the damage level table 533 corresponds to the area 522 .
Damage level information 527 and 535 correspond to damage level information according to the damage level determination results for each partial area of areas 521 and 522. As with the example described with reference to Fig. 4, damage level information 527 and 535 indicates the level of damage by a numerical value, and if the level is progressive, the numerical value is circled.
Redundancy 529 and 537 indicate the redundancy of each partial area of the areas 521 and 522, respectively, and a setting value of "L" is assigned to a partial area having "low" redundancy.

The indexes 526 and 534 are indexes for referencing damage level information 527 and 535 stored in the damage level tables 525 and 533, respectively, and are associated with each partial area of the areas 521 and 522. In the example shown in Fig. 5D, each of the indexes 526 and 534 is composed of three numbers, the first number indicating whether or not aggregation processing has been performed, and the latter two numbers indicating the vertical and horizontal numbers of the position index described with reference to Fig. 5C.
For example, the setting value 528 of the index 526, "0-1-1", indicates that the information is about the damage level corresponding to a partial area having a vertical index number of "1" and a horizontal index number of "1" in the area 521 before the aggregation process is performed. Also, the setting value 536 of the index 534, "1-1-1", indicates that the information is about the damage level corresponding to a partial area having a vertical index number of "1" and a horizontal index number of "1" in the area 522 after the aggregation process is performed.

The pattern information 538 indicates the size of the partial area after the aggregation process, which is indicated by a pattern using the partial area before the aggregation process as a unit of size. The partial area before the aggregation process and the partial area after the aggregation process are associated with each other based on the index 526, the index 534, and the pattern information 538.
For example, for a partial region after the aggregation process in which the setting value of index 534 is "1-1-1", the setting value of pattern information 538 is "3×3". Therefore, the partial region after the aggregation process is associated with the nine partial regions before the aggregation process in which the setting values of index 526 are "0-1-1" to "0-1-3", "0-2-1" to "0-2-3", and "0-3-1" to "0-3-3". In other words, when setting the value of damage level information 535 for a partial region in which the setting value of index 534 is "1-1-1", the values of damage level information 527 for partial regions corresponding to the range indicated by 530 to 532 are referenced.
As a result, even if the number of partial areas aggregated at the end of a structure, for example, is different from that at other parts, it is possible to obtain damage level information for the partial areas aggregated based on index 534 by counting the partial areas based on pattern information 538.

For example, FIG. 6A shows an example of the results of an aggregation process performed on damage level information (i.e., damage level information for area 403) that takes into account redundancy and progression of the damage level determination results based on the inspection image, for part 402 of structure 401 shown in FIG. 4. For example, damage level information 601 shows an example of damage level information after the aggregation process has been performed. Also, partial area 602 shows an example of a partial area that has been excluded from the presentation of damage level information as a result of the aggregation process. Details of damage level information 601 and partial area 602 will be described separately below.

Figure 6B also shows an example of a display mode of the damage level information for presenting the results of the aggregation process performed on the damage level information to the user. In the example shown in Figure 6B, each partial area after the aggregation process is color-coded according to the level of damage level so that it can be distinguished. More specifically, in the example shown in Figure 6B, the partial area with a higher level of damage level is controlled so that the partial area is colored darker.

<Processing>
An example of processing by the information processing device according to this embodiment will be described with reference to FIGS. 7A and 7B.
First, Fig. 7A will be described. Fig. 7A is a flowchart showing an example of a process related to display of damage level information according to a result of determining the damage level of each part of a target structure based on an inspection image.

In S701, the information processing device 100 acquires an inspection image according to the imaging result of the target structure, and acquires information according to the damage detection result for each part of the structure based on the inspection image. An example of the inspection image is the inspection image 201 described with reference to FIG.
In S702, the information processing device 100 performs an aging comparison for each damage detected based on the inspection image in S701, thereby acquiring information regarding the progression of the damage.
In S703, the information processing device 100 acquires information regarding redundancy of each part of the target structure based on structure information (e.g., design drawings, etc.) at the time of designing the structure.

In S704, the information processing device 100 determines the degree of damage of the part of the target structure corresponding to each partial region previously divided from the inspection image, based on the amount of damage present in the part and the state of the damage. For example, FIG. 2(b) shows an example of damage degree information according to the determination result of the damage degree of the part corresponding to each partial region divided from the inspection image. As described above, a variety of technologies based on a trained model constructed based on machine learning and image analysis have been put to practical use for the process related to the determination of the damage degree of the part corresponding to each partial region divided from the inspection image, so a detailed description will be omitted.

In S705, the information processing device 100 applies a correction process to the damage level determination result in S704 based on the information on progression and redundancy acquired in S702 and S703. For example, FIG. 3A (b) shows an example of a correction table related to the correction process based on progression and redundancy. Also, FIGS. 3B (c) and (d) show an example of damage level information according to the application result of the correction process based on progression and redundancy. Also, based on the execution result of the process of S705, a damage level table is created in which damage level information before the aggregation process is applied, as exemplified as the damage level table 525 in FIG. 5D. In the following description, in order to make the characteristics of the information processing device 100 according to this embodiment easier to understand, it is assumed that the damage level table 525 shown in FIG. 5D is created as a damage level table in which damage level information before the aggregation process is applied is stored.

In S706, the information processing device 100 executes processing related to displaying a drawing of the target structure, which serves as a reference position for presenting damage level information for each part of the target structure. The drawing is presented to the user by being displayed on a specified output destination such as the output device 106.

In S707, the information processing apparatus 100 determines whether or not to perform an aggregated display (that is, whether or not to aggregate the damage level information and then display the aggregated damage level information).
If the information processing apparatus 100 determines in S707 that aggregated display is not to be performed, the process proceeds to S708.
In S708, the information processing device 100 superimposes and displays the damage level information to which the correction process was applied in S705 for each partial area in which the damage level was determined on the drawing of the structure displayed on the specified output destination in S706.

On the other hand, if the information processing apparatus 100 determines in S707 that an aggregated display is to be performed, the process proceeds to S710.
In S710, the information processing device 100 executes a process related to the aggregation of damage level information. The process of S710 will be described in detail later with reference to FIG. 7B. Based on the execution result of the process of S710, a damage level table is created in which the damage level information after the aggregation process is applied is stored, as exemplified as the damage level table 533 in FIG. 5D. In the following description, in order to make the characteristics of the information processing device 100 according to this embodiment easier to understand, it is assumed that the damage level table 533 shown in FIG. 5D is created as the damage level table in which the damage level information after the aggregation process is applied is stored.
In S711, the information processing device 100 superimposes and displays the damage level information after the aggregation process is applied in S710 for each partial area for which the damage level has been determined on the drawing of the structure displayed on the predetermined output destination in S706. In this case, for example, information such as the damage level taking into account the progression of damage and the redundancy boundary line of each part of the structure, as described with reference to Figures 6A and 6B, may be displayed.

In S709, the information processing device 100 determines whether or not to end the process related to the display of the damage level information. For example, the information processing device 100 may determine whether or not to end the process related to the display of the damage level information in response to an instruction from a user. In addition, as another example, the information processing device 100 may determine whether or not to end the process related to the display of the damage level information in response to whether or not a predetermined trigger is received. In this way, the information processing device 100 may determine whether or not to end the process related to the display of the damage level information in response to whether or not a predetermined condition is satisfied.
The information processing device 100 continues displaying the damage level information to a specified output destination (displaying the damage level information based on the processing of S708 or S711) unless it determines in S709 to end the processing related to the display of the damage level information.
Then, when it is determined in S709 that the processing related to the display of the damage level information is to be ended, the information processing device 100 ends the display of the damage level information to a predetermined output destination and then ends the series of processing steps shown in FIG. 7A.

Next, referring to FIG. 7B, an example of the process for aggregating damage level information shown as S710 in FIG. 7A will be described in detail.

In S721, the information processing device 100 creates an empty damage level table 533 in which damage level information after the aggregation process is applied is stored. Then, the information processing device 100 sets values for the INDEX 534 and the pattern information 538 in the damage level table 533 according to the range to which the aggregation process is applied.
In S722, the information processing device 100 acquires information on the redundancy of each part of the structure for each partial area divided from the inspection image from the damage level table 525 in which the damage level information before the aggregation process is applied is stored. Note that the information on the redundancy of each part of the structure is acquired based on the redundancy map exemplified in FIG. 3A(a), for example.

Next, based on the setting value of INDEX534, the information processing device 100 extracts information (e.g., damage level information, etc.) about each of the partial areas before aggregation that are contained in each partial area to be aggregated, and executes the processes of S723 to S728 based on the information.

The aggregation process executed from S724 to S727 will now be described below with a specific example. For example, in region 403 in FIG. 4, each of the 3×3 regions into which partial regions 405 are gathered is treated as a region to be aggregated (i.e., a new partial region after aggregation), and the processes from S724 to S727 are executed for each region to be aggregated.

Specifically, in S724, the information processing apparatus 100 acquires the maximum level of damage degree from among the damage degrees corresponding to each of the series of partial regions 405 included in the region to be subjected to the aggregation process.
In S725, the information processing device 100 acquires information regarding the presence or absence of progressive damage level based on damage level information corresponding to each of a series of partial regions 405 included in the region to be subjected to the aggregation process.
In the processing of S724 and S725, information regarding the damage and the presence or absence of progression set for the corresponding partial region in the damage level table 525 is referenced based on the setting value of INDEX 534 in the damage level table 533 generated in S721.

4, the maximum level of damage in partial region 409 is "5", and the corresponding damage level information is circled to indicate that the damage level is progressive. Therefore, from damage level table 525, the value of the damage level "5" is obtained as information on the damage level corresponding to partial region 409, and information indicating that the damage is progressive is obtained as information on whether the damage is progressive.
In the partial region 409, the damage level information with the highest level of damage also indicates progression, and the information on the damage level and the information on the presence or absence of damage progression are obtained from the same damage level information. On the other hand, the information on the damage level and the information on the presence or absence of damage progression may each be obtained from different damage level information.

In S726, the information processing device 100 acquires information regarding the number of partial areas (partial areas before aggregation) for which damage level information is set within the area to be aggregated (new partial area after aggregation), and information regarding the continuity of the arrangement of those partial areas.
4, the number of partial regions 405 for which damage level information is set is "3", and these partial regions 405 are consecutive. Therefore, "3" is obtained as information regarding the number of partial regions 405 for which damage level information is set, and information indicating that there is continuity is obtained as information regarding the continuity of the arrangement of these partial regions 405.

Then, in S727, the information processing device 100 executes an aggregation process (i.e., a process related to the aggregation of damage level information) for the information acquired in each of S724 to S726 based on the control table 505 shown in FIG. 5B.

4, the damage level value input to the aggregation process is "5", and the damage is progressive, so condition 508 for when damage in a region is progressive is applied as a condition related to the progressiveness of the damage. Also, since partial region 409 is included in a low redundancy region, condition 506 for when the region is low redundancy is applied as a condition related to the redundancy.
In addition, when the area to be subjected to the aggregation process contains both areas with high redundancy and areas with low redundancy, it is preferable, for example, to apply the aggregation process to the area to be subjected to the aggregation process as an area with low redundancy.
In addition, since the partial region 409 has continuity with respect to the arrangement of the partial regions 405 for which damage level information is set, condition 510 in which the damage density is dense or the damage has continuity is applied as a condition related to the damage density and continuity.
As a result, in the case of partial region 409, an increase or decrease value of "+1" is applied to the damage level after the aggregation process is applied, but in this embodiment, the maximum value of the damage level is set to "5." Therefore, in damage level information 601 shown in Fig. 6A, "5" is set as the value of the damage level after the aggregation process is applied. Also, in partial region 409, it is determined that damage is progressing, so a circle is placed around the value of the damage level as flag information indicating the progress of damage.

4 (partial region after aggregation), the value of the damage level input to the aggregation process is "1" and does not have damage progression, so condition 509 for a case where damage in a region does not have progression is applied as a condition related to damage progression. Also, since partial region 410 is included in a high redundancy region, condition 507 for a region with high redundancy is applied as a condition related to redundancy. In addition, partial region 410 has one partial region 405 with damage level information set and the damage density is "sparse", so condition 511 for a case where the damage density is sparse or there is a single damage is applied as a condition related to the damage density and continuity.
As a result, in the case of partial region 410, an increment/decrement value of "-1" is applied to the damage level after the aggregation process is applied, and the damage level after the increment/decrement value is applied becomes "0." As a result, partial region 410 is treated as an area for which no damage level information is set, like partial region 602 shown in FIG. 6 .

When the increase or decrease value applied based on the control table 505 shown in FIG. 5B becomes "±0", the damage level value (the level value at which the damage level is at its maximum) input to the aggregation process and information regarding the presence or absence of progression will be inherited by the partial area after the aggregation process.

In this manner, the information processing device 100 repeatedly executes the processes of S723 to S728 for each area to be aggregated, and when the execution of the processes for the series of areas is completed, ends the series of processes shown in FIG. 7B. That is, in the case of the example shown in FIG. 4, the processes of S723 to S728 are executed for each application unit of the aggregation process for the partial areas 405 for which damage level information has been set, out of the series of partial areas 405 included in the area 403, and the processing result shown in FIG. 6A is obtained.

As described above, in the information processing device 100 according to this embodiment, when presenting the damage level of a structure to a user, the damage level information is presented to the user at a coarser granularity by applying an aggregation process to the damage level information determined at a finer granularity based on the inspection image. By applying such control, it becomes possible to allow the user to visually recognize the area to be noted in a more suitable manner. Therefore, for example, when a user investigates the soundness of a structure based on the damage level information, it becomes possible to draw the user's attention to the damage level that is important in the investigation of the soundness, including the potential damage level. In addition, since it is possible to suppress the presentation of information to the user about damage levels with low priority in the investigation of the soundness of the structure, it is expected to have the effect of guiding the user to pay attention to damage levels with higher priority.

In this embodiment, the description focuses on a case where an area of 3x3 size is applied as the application unit of the aggregation process, but the application unit of the aggregation process is not necessarily limited. As a specific example, an area larger or smaller than a 3x3 size area may be applied as the application unit of the aggregation process.

In the present embodiment, as shown in FIG. 6A, an example of the case where the damage level is expressed by a numerical value has been described, but the presenting of the damage level is not necessarily limited to this embodiment. As a specific example, as shown in FIG. 6B, in addition to the numerical display, the damage level may be displayed by the color (background color) of the area where the damage level information is displayed. By applying such control, the user can intuitively determine the damage level by the color of the area where the numerical value is displayed, without having to focus on the numerical value indicating the damage level, and it is also possible to expect the effect of further improving the visibility of the damage level. In addition, by applying a color according to the damage level to the area after the aggregation process is applied, it is possible to intuitively recognize that the presented damage level information is the damage level information after the aggregation process is applied. Of course, in the process of S708, when the damage level information is presented, the color (background color) of the partial area where the damage level information is presented may be controlled.

The manner in which the presence or absence of damage progression is presented is not necessarily limited to the method of circling the numerical value indicating the level of damage. As a specific example, the presence or absence of damage progression may be presented by presenting the numerical value indicating the level of damage in bold or italics. As another example, the presence or absence of damage progression may be presented by adding a subscript of a different character type to the numerical value indicating the level of damage.

In addition, in this embodiment, an example of a case where the damage level is determined for each part of the structure corresponding to a partial area divided from the inspection image of the target structure has been described, but the method for determining the damage level is not necessarily limited to this. As a specific example, damage occurring in the structure may be detected individually, and the damage level may be determined for each detected damage. Even in such a case, by setting a partial area wider than the partial areas corresponding to each individual damage as the area to which the aggregation process is applied, it becomes possible to apply the aggregation process in the same manner as in the above-mentioned embodiment.

The conditions for applying the aggregation process are not necessarily limited to the example shown as control table 505 in FIG. 5B. As a specific example, more detailed conditions may be applied to the conditions related to the progression of damage, the redundancy of each part of the structure, and the density and continuity of damage. Furthermore, the threshold applied to determine each condition is not limited to one threshold, and multiple thresholds may be applied.

<Modification>
Below, a modification of the information processing device 100 according to this embodiment will be described.

(Variation 1)
First, the information processing device 100 according to the first modification will be described with reference to Fig. 8A, Fig. 8B, Fig. 9, Fig. 10A, and Fig. 10B. In the above-described embodiment, an example of a case where damage level information for each partial region divided from an inspection image and damage level information after aggregation processing are selectively switched and presented is described. In this modification, an example of a mechanism that makes it possible to present damage level information by gradually switching the application range of the aggregation processing by gradually applying the aggregation processing of the damage level information is described.

Fig. 8A(a) shows an example of a presentation mode of damage level information for each part of the structure 401, which corresponds to the state after the aggregation process of the damage level information has been performed in the above-mentioned embodiment. In contrast, Fig. 8A(b) shows an example of a presentation mode of damage level information when another aggregation process is applied from the state shown in Fig. 8A(a). The partial area 803 surrounded by a thick frame corresponds to the area to which further aggregation process is applied (in other words, the application unit of further aggregation process). Moreover, damage level information 804 corresponds to damage level information according to the application result of the aggregation process targeted at the partial area 803.
Similarly, Fig. 8B(c) shows an example of the presentation mode of the damage level information when another aggregation process is applied from the state shown in Fig. 8A(b). Also, Fig. 8B(d) shows an example of the presentation mode of the damage level information when another aggregation process is applied from the state shown in Fig. 8B(c).

When further aggregation processing is performed from the state shown in FIG. 8A(b) to the state shown in FIG. 8B(c), a 3×2 pattern and a 2×2 pattern are applied, rather than the 3×3 pattern that was the target of the previously executed aggregation processing. Therefore, as the operation pattern of the aggregation processing in this case, an operation pattern other than the operation pattern based on the control table 505 shown in FIG. 5B is applied. This is also the case when further aggregation processing is performed from the state shown in FIG. 8B(c) to the state shown in FIG. 8B(d).

Next, referring to FIG. 9, an example of a UI applied in this modified example will be described, focusing in particular on a UI for presenting damage level information while applying aggregation processing in stages. A display screen 901 according to this modified example includes a display area 902, a slider 908, and buttons 906 and 907.

The display area 902 is an area where a drawing of a structure and damage level information are displayed.
The slider 908 is an interface for receiving from the user a threshold value for the level of damage level used to control the damage level information to be displayed in the display area 902 .
The button 906 is an interface for receiving an instruction from the user to apply further aggregation processing of the damage level information. For example, when the button 906 is pressed, another aggregation processing of the damage level information is applied, and the result is reflected in the display area 902.
The button 907 is an interface for receiving an instruction from the user to cancel the further aggregation process of the damage level information that has already been applied. For example, when the button 907 is pressed, the further aggregation process of the damage level information that has already been applied is canceled by one step, and the result is reflected in the display area 902.

The display screen 901 may also include buttons 903, 904, and 905. The button 904 is an interface for receiving an instruction from the user to enlarge the structure drawing and damage level information to be displayed in the display area 902 in the display area 902. The button 905 is an interface for receiving an instruction from the user to reduce the structure drawing and damage level information to be displayed in the display area 902 in the display area 902. The button 903 is an interface for receiving an instruction from the user regarding scrolling within the display area 902 of the structure drawing and damage level information to be displayed in the display area 902. By using the button 903, for example, when an enlarged display is performed, it is also possible to adjust the area displayed in the display area 902 by scrolling.

The various UIs shown in FIG. 9(a) may be realized by a hardware UI such as a physical button, or may be realized by a software UI (e.g., a UI displayed on a screen) that can be operated by an operating device such as a mouse or a touch panel.

FIG. 9(b) shows an example of a drawing and damage level information of the structure 401 displayed in the display area 902. In the example shown in FIG. 9(b), the damage level information to be displayed is limited to damage level information with a damage level of 3 or more, due to a threshold value specified for the damage level.

Next, an example of the process of the information processing device 100 according to this modification will be described with reference to Fig. 10A and Fig. 10B. First, Fig. 10A is a flowchart showing an example of the process related to the display of damage level information.
The processes of S1001 to S1005 are substantially similar to the processes of S701 to S705 in Fig. 7A, and therefore detailed description thereof will be omitted. By executing the process of S1005, for example, damage level information is acquired in which a correction process based on the progression of damage and redundancy of each part of the structure is applied to the damage level determination results for each partial area 405 in the area 403 as shown in Fig. 4.

In S1006, the information processing apparatus 100 executes processing for creating a damage level table for managing damage level information aggregated for each hierarchical level, such as the damage level table 533 illustrated in FIG. 5D.
The process of S1006 will now be described in more detail with reference to Fig. 10B, which is a flowchart showing an example of a process related to the creation of a damage level table.

In S1021, the information processing device 100 creates an empty damage level table for recording damage level information corresponding to each area for each aggregation hierarchy when the aggregation process is applied repeatedly in stages based on the damage level information for each partial area before the aggregation process is applied.
In S1022, the information processing device 100 writes the damage level information to which correction processing based on progression and redundancy has been applied in S1005 into the damage level table created in S1021 as damage level information before further aggregation processing is applied.

In S1023, the information processing device 100 determines the area to which the aggregation process is applied next. For example, when applying the repair process illustrated in Fig. 5A, the information processing device determines a 3 x 3 area as the target of the aggregation process. Note that the area to which the aggregation process is applied next is determined within the range in which the aggregation process can be applied.
For example, in the state shown in Fig. 8A(a), it is possible to secure a 3 x 3 area as a target for the next aggregation process. Therefore, in the state shown in Fig. 8A(a), the 3 x 3 area is determined as the area to which the next aggregation process is to be applied, similar to the previously applied aggregation process.
On the other hand, in the state shown in Fig. 8A(b), it is difficult to secure a 3 x 3 area as the target for the next aggregation process. Therefore, in the state shown in Fig. 8A(b), the area to which the next aggregation process is applied is determined within a range that can be secured. Specifically, a 3 x 2 area is determined as the area to which the next aggregation process is applied for the central part, and a 2 x 2 area is determined as the area to which the next aggregation process is applied for both ends.

In S1024, the information processing device 100 determines the operation pattern of the aggregation process for the area to which the aggregation process is next applied, which was determined in S1023. For example, if the area to which the aggregation process is next applied is a 3x3 area, the operation pattern of the aggregation process may be determined based on the control table 505 shown in FIG. 5B. On the other hand, if the area to which the aggregation process is next applied is an area of a pattern other than a 3x3 area (for example, a 3x2 area or a 2x2 area), the conditions for determining the operation pattern of the aggregation process may be changed as appropriate. As a specific example, the thresholds for determining the density and continuity of damage shown in FIG. 5B may be changed depending on the number of partial areas for which damage level information is set in the area to which the aggregation process is next applied.

The repeated processing of S1025 to S1027 may be configured to refer to the input damage level from the INDEX of the area where the aggregation processing has been performed. This makes it possible to apply the aggregation processing shown in S1026 to each area determined in S1023, regardless of the number of areas to be aggregated. In other words, by sequentially aggregating the information shown as pattern information 538 in FIG. 5D, it becomes possible to collect damage level information for the areas to which the aggregation processing is applied, and by switching the corresponding operation pattern, it becomes possible to execute the repeated processing.

In S1028, the information processing apparatus 100 writes the damage level information after the aggregation process has been applied by the repeated processes of S1025 to S1027 into the damage level table created in S1021.
In the damage level table in which the damage level information is written for each hierarchical level in S1028, a number for each hierarchical level is added to the INDEX used to refer to each piece of damage level information, as exemplified as the setting value 536 of the index 534 in Fig. 5D. That is, among the numbers exemplified as the setting value 536, the first number is assigned "1" if it is the result of the first aggregation process, and thereafter, a number incremented by "1" is assigned for each repetition of the aggregation process.

In S1029, the information processing device 100 determines whether or not the result of the aggregation process extends to an area indicating the entire structure (in other words, whether or not the result of the aggregation process has converged to one area).
If the information processing device 100 determines in S1029 that the result of the aggregation process does not extend to the area showing the entire structure, the information processing device 100 advances the process to S1023. In this case, the processes from S1023 onwards are executed again.
On the other hand, when the information processing device 100 determines in S1029 that the result of the aggregation process has reached the area showing the entire structure, the series of processes shown in Fig. 10B is terminated. In this case, information (e.g., damage level information) is written in the damage level table for each of the series of hierarchical levels related to the aggregation process.

Now, reference is again made to FIG. 10A.
In S1007, the information processing device 100 displays a drawing of the target structure on a predetermined display screen. As a specific example, in the case of the example shown in Fig. 9, the information processing device 100 displays the drawing of the target structure in a display area 902 of a display screen 901.

In S1008, the information processing device 100 selects a hierarchical level related to the aggregation process for which damage level information is to be displayed, and identifies a damage level table corresponding to the selected hierarchical level. Note that there are no particular limitations on the configuration or method for selecting a hierarchical level for which damage level information is to be displayed. As a specific example, the information processing device 100 may be configured to be able to select any hierarchical level at any time. As another example, the information processing device 100 may select a hierarchical level for which damage level information is to be displayed by sequentially switching between hierarchical levels in response to a desired trigger (e.g., an instruction from the user via button 906 or 907 shown in FIG. 9).

In S1009, the information processing device 100 reads in a threshold value for the damage level to be displayed. As a specific example, the information processing device 100 may read in a threshold value selected by the user via the slider 908 shown in FIG. 9. In the example shown in FIG. 9, the threshold value can be selected from "1" to "5" via the slider 908, and damage level information with a damage level below the selected threshold value is excluded from being displayed. In this case, when "1" is selected as the threshold value, damage level information with damage levels of 1 to 5, i.e., all damage level information, is displayed.

In S1010, the information processing device 100 extracts damage level information whose damage level is equal to or greater than the threshold selected in S1009 from the damage level information corresponding to the hierarchical level selected in S1008, and displays the damage level information on a specified display screen. As a specific example, in the case of the example shown in FIG. 9, the information processing device 100 displays the damage level information in the display area 902 of the display screen 901 so that the damage level information is superimposed on the drawing of the structure displayed in the display area 902. For example, FIG. 9(b) shows an example of a display mode of damage level information when, in the state shown in FIG. 8A(b), damage level information whose damage level is 3 or more is specified as the damage level information to be displayed via the slider 908.

In S1011, the information processing device 100 determines whether or not to end the process relating to the display of damage level information.
When it is determined in S1011 that the process related to the display of the damage level information is not to be ended, the information processing apparatus 100 advances the process to S1008. In this case, the process from S1008 onward is executed again.
Then, when the information processing device 100 determines in S1011 to end the process related to the display of the damage level information, it ends the display of the damage level information to a predetermined output destination and then ends the series of processes shown in FIG. 10A.

As described above, the information processing device 100 according to this modified example allows the user to refer to the damage level information according to the damage level determination results based on the inspection image as information aggregated at the desired granularity by appropriately selecting the hierarchy related to the aggregation process. In addition, by appropriately selecting the granularity related to the display of the damage level information (in other words, the range of each area), it becomes possible to check the damage level of each part of the target structure (for example, the pier parts and deck parts of a bridge) in a more suitable manner.

In addition, by limiting the damage degree information to be displayed according to, for example, the level of damage degree, it is possible to efficiently check the damage degree by limiting it to, for example, areas with a high degree of damage.
Furthermore, as shown in FIG. 8D, it is also possible to present damage level information for the entire structure, and this damage level information can be used to determine the soundness of the structure.
Furthermore, even in a situation where there are multiple structures to be inspected, by appropriately selecting the granularity of the display of the damage level information (in other words, the hierarchy related to the aggregation process), it is possible to focus on inspecting the parts that are expected to be particularly affected.As a specific example, it is possible to identify structures that are estimated to be more damaged based on the damage level information for each structure, and for those structures, it is possible to gradually identify the parts that are expected to be particularly affected while appropriately lowering the hierarchy related to the aggregation process.

(Variation 2)
Next, the information processing device 100 according to the modified example 2 will be described with reference to FIG. 11, FIG. 12A, and FIG. 12B. In the above-described embodiment and modified example 1, an example of a mechanism that enables the granularity of the display of the damage degree information to be changed by aggregating the damage degree information in each of a plurality of partial regions has been described. In this modified example, an example of a configuration for presenting information related to the repair of damage present in a part corresponding to the partial region upon receiving a designation of at least a part of a series of partial regions (for example, partial regions before or after aggregation) related to the presentation of the damage degree information will be described. As a specific example, the information processing device 100 according to this modified example tally up the type of repair material for repairing the damage present in the part corresponding to the selected partial region, the amount of the repair material required for the repair, and the like, and presents information according to the result of the tally as information related to the repair of the damage.

FIG. 11 shows an example of a display screen used to present damage level information in the information processing device 100 according to this modification. In the example shown in FIG. 11, damage level information after aggregation processing is performed is presented for each area to which the aggregation processing is applied. For example, an area 1101 corresponds to an area to be aggregated, and the level of damage level is determined to be "5" based on the result of the aggregation processing of the damage level information. In addition, the notification information 1102 is a display area used to present information related to repair of damage existing in the selected area. In the example shown in FIG. 11, when an area 1101 is selected, the type of repair material for repairing the damage existing in the area 1101, the amount of the repair material required for the repair, etc. are aggregated, and the notification information 1102 in which information corresponding to the result of the aggregation is presented is displayed.
The notification information 1102 corresponds to an example of "third information" related to repair of damage present in the target structure portion.

Next, an example of the process of the information processing device 100 according to this modification will be described with reference to Fig. 12A and Fig. 12B. First, Fig. 12A is a flowchart showing an example of the process related to the display of damage level information.
The processes of S1201 to S1210 are substantially similar to the processes of S1001 to S1010 in Fig. 10A, and therefore detailed description thereof will be omitted. By executing the process of S1010, a display screen related to the presentation of damage level information according to the determination result of the damage level in each part of the target structure, as shown in Fig. 9, is displayed on a predetermined output destination.
Then, in S1211, the information processing device 100 executes a process for displaying information related to the repair of the damage, for example, a process for displaying information related to an estimate of repair materials.

Here, the process of S1211 will be described in more detail with reference to FIG. 12B. FIG. 12B is a flowchart showing an example of a process related to displaying information related to the repair of damage, and in particular shows a process for displaying information related to the repair of the damage, which is information related to the estimated amount of repair material.

In S1221, the information processing device 100 determines whether or not a desired area (e.g., a partial area before or after aggregation) in which damage level information is displayed has been selected. As a specific example, the information processing device 100 may receive an instruction from a user to select an area in which damage level information is displayed via a display screen 901 shown in FIG. 9A.
If the information processing device 100 determines in S1221 that the desired area in which the damage level information is displayed has not been selected, the information processing device 100 ends the series of processes shown in FIG. 12B.
On the other hand, if the information processing device 100 determines in S1221 that a desired area in which damage level information is displayed has been selected, the information processing device 100 advances the process to S1222.

In S1222, the information processing device 100 acquires a series of damage level information before the aggregation process is performed for the area selected in S1221. In this manner, the information processing device 100 individually collects damage level information corresponding to each of the damages detected based on the inspection image in S1201.
In S1223, the information processing device 100 acquires information on the design of the part of the structure corresponding to the area selected in S1221 (hereinafter, also referred to as "design drawing information"). The design drawing information may include information on the characteristics of the part, such as the name and type of the structural material for each part of the target structure.

In the repeated processing of S1224 to S1231, the processing is repeatedly executed for each damage corresponding to the individual damage level information obtained in S1222.

Specifically, in S1225, the information processing apparatus 100 detects the type of damage to be detected. Examples of the type of damage include cracks, peeling, water leakage, and floats.
In S1226, the information processing device 100 detects the size and scale of the target damage according to the type of the damage detected in S1225. As a specific example, when the target damage is a crack, the information processing device 100 detects the length and width of the crack, and the presence or absence of each of branches, intersections, and tortoiseshell shapes, etc., as the size and scale of the damage.

In S1227, the information processing device 100 acquires information on the characteristics of the structural member in which the target damage has occurred from the design drawing information acquired in S1223.
In S1228, the information processing device 100 selects the type of repair material for repairing the target damage according to the type of the damage, the size of the damage, and the type of structural material specified in S1225 to S1227. Note that information regarding the type of damage, the type of repair agent corresponding to the type of structural material, and the amount of the repair material required for each unit of the size of the damage may be stored in advance as a correspondence table.
In S1229, the information processing device 100 calculates the amount of repair material required when the repair material selected in S1228 is used to repair the damage, based on the size and scale of the target damage.
In S1230, the information processing device 100 records information regarding the type of repair material selected in S1228 and information regarding the amount of repair material calculated in S1229 in association with the target damage.

In this manner, the information processing device 100 repeatedly executes the processes of S1225 to S1230 for each of the series of damages. As a result, for each of the series of damages that occurred in the area corresponding to the region selected in S1221, information regarding the type and amount of repair material used to repair the damage is recorded.

Then, in S1232, the information processing device 100 displays, at a specified output destination, information regarding the type and total amount of repair material required to repair the series of damage that has occurred in the area corresponding to the area selected in S1221. As a result, for example, as shown as notification information 1102 in FIG. 11, information regarding the type and amount of repair material required to repair the series of damage that has occurred in the area corresponding to the selected area is displayed as a guide.

The damage for which damage level information is presented on the display screen shown in FIG. 11 is detected solely based on the analysis results of the inspection image corresponding to the imaging results of the side of the structure. Therefore, in order to present more accurate information related to the repair of the damage (e.g., information on the type and amount of repair material, etc.), the damage level determination results based on other inspection images obtained by imaging the structure from other directions may also be used. As a specific example, if the structure is a bridge, the damage level determination results based on inspection images of the deck plate taken from the top or bottom of the bridge, or inspection images of the bridge taken from the back side, etc. may be used in conjunction with obtaining information related to the repair of the damage.

Now, reference is made again to FIG. 12A.
In S1212, the information processing device 100 determines whether or not to end the process relating to the display of the damage level information.
When the information processing device 100 determines in S1212 that the process related to the display of the damage level information is not to be ended, the process proceeds to S1208. In this case, the process from S1208 onwards is executed again.
Then, when it is determined in S1212 that the processing related to the display of the damage level information is to be ended, the information processing device 100 ends the display of the damage level information to a predetermined output destination and then ends the series of processing steps shown in FIG. 12A.

As described above, with the information processing device 100 of this modified example, a user can select any area in which damage level information is presented at any hierarchical level related to the aggregation process, and thereby view information related to repairing damage that has occurred in the part corresponding to that area.
As described above, the information related to the repair of damage is displayed as a result of aggregation based on the damage level information of each of the series of damages that occurred in the part corresponding to the selected area. Therefore, for example, a user can view information related to the repair of damage that occurred in the part corresponding to the selected area at any granularity by appropriately switching the hierarchy related to the aggregation process and then selecting a desired area. This allows, for example, a user to order repair materials related to the repair of damage that occurred in the part corresponding to the desired area, and to order the repair materials collectively based on the presented information, without the cumbersome task of aggregating the type and amount of repair materials for each damage.

<Other embodiments>
The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a recording medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

REFERENCE SIGNS LIST 100 Information processing device 111 Damage level determination unit 112 Correction processing unit 113 Aggregation processing unit 114 Storage unit 115 Display control unit

Claims (13)

an aggregation means for aggregating, for each second partial region larger than the first partial region, first information relating to a damage degree of a portion of the structure corresponding to the first partial region, which is associated with the first partial region divided from an image of the target structure, and for each second partial region larger than the first partial region, the first information corresponding to each of the first partial regions included in the second partial region, as second information based on at least one of a density and a continuity of damage in the second partial region;
a recording means for recording the second information corresponding to the second partial area and the first information corresponding to each of the plurality of first partial areas included in the second partial area in association with each other;
An information processing device comprising: The information processing device according to claim 1, characterized in that the recording means assigns the first information for each of the first partial regions and the second information for each of the second partial regions to different hierarchical levels, thereby recording information regarding the degree of damage to the portion of the structure for each hierarchical level according to the granularity of division of the captured image. The information processing device according to claim 2, characterized in that the series of hierarchies includes at least a hierarchical level corresponding to an area including the entire structure. The information processing device according to claim 2 or 3, characterized in that it comprises a display control means for displaying, on a predetermined output destination, partial areas divided from the captured image at a granularity corresponding to the specified hierarchical level and information regarding the degree of damage of the part of the structure corresponding to the partial areas in association with the captured image. The information processing device according to claim 4, characterized in that the display control means selectively switches between the second information corresponding to the second partial region and the first information corresponding to each of the plurality of first partial regions included in the second partial region, and displays the information on the output destination in association with the corresponding region in the captured image. The information processing device according to claim 4 or 5, wherein the display control means suppresses display of part of the information on the damage level of the part of the structure corresponding to each of a series of partial regions divided from the captured image at a granularity corresponding to the specified hierarchical level, in accordance with the specified level of damage level. a counting means for counting the first information for each of the first partial regions included in the second partial region,
the display control means displays, on the output destination, third information related to repair of damage present in a portion of the structure corresponding to the second partial area according to a result of the compilation;
7. The information processing device according to claim 4. The counting means, based on a result of the counting, identifies a type of repair material related to repair of damage present in a portion of the structure corresponding to the second partial region and calculates an amount of the repair material;
The third information includes information on the type of the identified repair material and information corresponding to a calculation result of the amount of the repair material.
The information processing device according to claim 7. The information processing device according to any one of claims 1 to 8, further comprising a correction means for correcting information on the degree of damage of a portion of the structure corresponding to a partial region divided from the captured image, based on at least one of redundancy according to the allowable damage in that portion and progression of the damage in that portion. The correction means is
correcting the information about the degree of damage so that the degree of damage becomes lower as the redundancy becomes higher;
The information on the damage level is corrected so that the damage level becomes higher as the redundancy becomes lower.
The information processing device according to claim 9. The correction means is
correcting the information on the degree of damage so that the degree of damage is higher the more progressive the condition is,
The information on the degree of damage is corrected so that the lower the progression, the lower the degree of damage.
The information processing device according to claim 9 or 10. An information processing method executed by an information processing device,
an aggregation step of aggregating, for each second partial region larger than the first partial region, first information relating to a damage degree of a portion of the structure corresponding to the first partial region, which is associated with the first partial region divided from the captured image of the target structure, into second information based on at least one of a density and a continuity of damage in the second partial region, and the first information corresponding to each of the first partial regions included in the second partial region;
a recording step of recording the second information corresponding to the second partial area and the first information corresponding to each of the plurality of first partial areas included in the second partial area in association with each other;
13. An information processing method comprising: A program for causing a computer to function as each of the means of an information processing device according to any one of claims 1 to 11.

Images