JP7429133B2 - Learning method, learning device, building landscape information generation method, generation device, computer program for building anomaly detection model - Google Patents

Description

The present invention relates to a technique for obtaining useful information by inputting BIM information of a building into a generative model using GAN.

In recent years, BIM (Building Information Modeling) information, which is three-dimensional information about buildings, has been used as CAD data for buildings (see Patent Document 1).

JP 2020-4278 Publication

This kind of BIM information includes various information such as shape information indicating the shape of the elements that make up each part of the building and attribute information indicating the attributes of the elements, so BIM information can be used for various useful purposes. It is possible to obtain information.
For example, it is conceivable to use a machine-learned abnormality detection model to detect abnormal locations based on image information obtained by capturing images of target locations in buildings and BIM information.
Alternatively, it is conceivable to use a machine-learned generative model to obtain landscape information of buildings in various environments based on BIM information.
The present invention has been made in view of the above circumstances, and its purpose is to improve the accuracy of detecting abnormalities using an abnormality detection model, and to improve the detection of abnormalities in buildings, which is advantageous in reliably detecting abnormalities. The purpose of the present invention is to provide a model learning method, a learning device, and a computer program. Another object of the present invention is to provide a method, a device, and a computer program for generating architectural landscape information that are advantageous in accurately generating architectural landscape information that reflects environmental conditions.

In order to achieve the above-mentioned object, the present invention provides a learning method for an anomaly detection model that detects abnormal locations based on image information obtained by capturing images of a building, the method comprising: A first image information, which is normal image information of a part of the building to be detected, based on BIM (Building Information Modeling) information, which is three-dimensional information of the building, including shape information and attribute information indicating attributes of the elements. a step of generating image information; a step of obtaining second image information obtained by actually capturing an image of the detection target location of the building; and performing image processing on at least one of the first image information and the second image information. a step of generating third image information to which an abnormal location is newly added by the method; a step of learning the third image information by a generative model using GAN (Generative Adversarial Networks); and a step of learning with the third image information. generating fourth image information as image information for padding from the first image information using the generated generation model; and generating the abnormality detection model using the second image information and the fourth image information as training data. The method is characterized in that it includes a step of performing learning.
Further, the present invention is characterized in that the abnormal location includes at least one of cracks, peeling, and stains occurring in the building.
Further, the present invention is a computer program that causes a computer to execute the learning method for the building anomaly detection model described above.
The present invention also provides a learning device for an anomaly detection model that detects an abnormal location based on image information obtained by capturing an image of a building, comprising: an imaging unit that captures an image of the building and generates the image information; The detection target of the building is determined from BIM (Building Information Modeling) information, which is three-dimensional information of the building, including shape information indicating the shape of elements constituting each part of the object, and attribute information indicating the attributes of the elements. a first image information generation unit that generates first image information that is normal image information of a portion of the building; and second image information that obtains second image information obtained by actually capturing an image of the detection target location of the building by the imaging unit. a third image information generating section that generates third image information to which an abnormal location is newly added by performing image processing on at least one of the first image information and the second image information; a fourth image information generation unit configured with a generative model using GAN (Generative Adversarial Networks), which learns the third image information and generates fourth image information as image information for padding from the first image information; and an anomaly detection model learning unit that learns the anomaly detection model using the second image information and the fourth image information.
The present invention also provides a method for generating building landscape information, which includes shape information indicating the shapes of elements constituting each part of the building, and attribute information indicating attributes of the elements. A step of preparing BIM (Building Information Modeling) information, a step of inputting environmental conditions surrounding the building, and a step of learning the environmental conditions by a generative model using GAN (Generative Adversarial Networks). generating building landscape information indicating an external image of the building as seen from the outside or an internal image of the building as seen from the inside from the BIM information using a generative model that has learned the environmental conditions; It is characterized by
Further, the present invention is characterized in that the environmental conditions include at least one of the time of day, the season of the year, and the weather.
Further, the present invention is a computer program that causes a computer to execute the above-described method for generating building landscape information.
The present invention also provides a building landscape information generation device, which provides three-dimensional information of the building, including shape information indicating the shape of elements constituting each part of the building, and attribute information indicating attributes of the elements. A recording unit that records BIM (Building Information Modeling) information, an input unit that inputs environmental conditions surrounding the building, and a generative model that uses GAN (Generative Adversarial Networks) to learn the environmental conditions and A building landscape information generation unit that generates building landscape information indicating an external image of the building viewed from the outside or an internal image of the building viewed from the inside, from BIM information. .

According to the present invention, first image information, which is normal image information of the part of the building to be detected, is generated from BIM information, which is three-dimensional information of the building, and the detection target part of the building is actually imaged. Obtaining the second image information, and performing image processing on at least one of the first image information and the second image information, third image information with newly added abnormal areas is generated, and GAN is used. The third image information is learned by the generative model, and the learned generative model generates fourth image information as image information for padding from the first image information, and the second image information and the fourth image information are used as training data. The anomaly detection model was trained using this method.
Therefore, when training an anomaly detection model, even if only a small amount of second image information can be prepared, a large number of fourth image information as padded data can be easily generated, and the second image information and fourth image information can be used as training data. Since the anomaly detection model can be trained as This is advantageous for reliably detecting abnormal locations.
Further, according to the present invention, BIM information, which is three-dimensional information of a building, including shape information indicating the shape of elements constituting each part of the building and attribute information indicating the attributes of the elements, is prepared, and the GAN is The generative model used learns the environmental conditions surrounding the building, and the generative model that has learned the environmental conditions shows an external image of the building as seen from the outside or an internal image of the building as seen from the inside from the BIM information. Architectural landscape information is now generated.
Therefore, it goes without saying that building landscape information can be generated accurately based on the shape information and attribute information of BIM information, and since a generation model that has learned environmental conditions is used, the obtained building landscape information reflects the environmental conditions. It becomes what it is.
Therefore, when public relations materials (catalogs) for buildings are created using building landscape information, images of buildings that reflect various environmental conditions can be presented, making the content of public relations materials diverse and appealing to customers. This will be advantageous in improving the advertising effect for

1 is a block diagram of a learning device for a building anomaly detection model according to a first embodiment; FIG. FIG. 3 is a schematic diagram showing the flow of information in the learning method for the building anomaly detection model according to the first embodiment. 2 is a flowchart illustrating a learning method for a building anomaly detection model according to the first embodiment. FIG. 2 is a block diagram of a building landscape information generation device according to a second embodiment. 7 is a flowchart showing a method of generating building landscape information according to a second embodiment.

(First embodiment)
Next, a learning device for a building anomaly detection model according to an embodiment of the present invention will be described together with a learning method and a computer program.
In addition, in this specification, a building may be either a building under construction or a building after construction.
As shown in FIG. 1, a learning device (hereinafter simply referred to as the learning device) 10 for a building anomaly detection model includes an imaging section 12, a BIM information generating section 14, and a computer 16. It is communicably connected to a robot 18, which will be described later.

The imaging unit 12, BIM information generation unit 14, and computer 16 are connected via a wired or wireless communication line so that they can exchange data with each other.
The BIM information generation unit 14 is configured of, for example, a computer used in building design work.
The BIM information generation unit 14 generates BIM (Building Information), which is three-dimensional information (three-dimensional CAD information) of a building, including shape information indicating the shape of elements constituting each part of the building, and attribute information indicating attributes of the elements. Information Modeling) generates information and supplies it to the computer 16.
Here, the elements constituting each part of a building include, for example, a roof, an outer wall, an inner wall, a column, a ceiling, a floor, fittings (windows, doors, doors), etc., which constitute each part of various conventionally known buildings. is an element.
Shape information is information indicating the shape of the element.
Attribute information is information indicating dimensions, materials, colors, etc. of the element, and in other words, it is information expressed visually.

The computer 16 includes a CPU, a ROM that stores and stores control programs, a RAM that serves as an operating area for the control programs, an interface section that interfaces with peripheral circuits, and the like.
The computer 16 has a first image information generation section 16A, a second image information generation section 16B, a third image information generation section 16C, a fourth image information generation section 16D, and abnormality detection model learning by the CPU executing the control program. It functions as part 16E.

The first image information generation unit 16A generates first image information D1, which is normal image information of a portion of the building to be detected, from the BIM information.
Note that the first image information D1 to fourth image information D4 described below may be three-dimensional image information or two-dimensional image information.
The detection target portion of the building is a portion that is visible from the outside or the inside of the building, in other words, a portion that can be imaged by the imaging unit 12.
The first image information D1 is generated from BIM information, and is therefore image information in a state where there is no abnormality in the part to be detected, for example, there is no damage such as cracks or peeling, or stains due to aging deterioration. This is image information of the part whose condition is to be detected.

The second image information generation unit 16B generates second image information D2 obtained by actually capturing an image of the detection target location of the building. In this embodiment, the second image information generation unit 16B This is accepted as image information D2. It is preferable that the second image information D2 is generated at a stage when no abnormality has yet occurred in the detection target location of the building.

The third image information generation unit 16C generates third image information D3 to which an abnormal part is newly added by performing image processing on at least one of the first image information D1 and the second image information D2. be.
That is, the third image information D3 is information to which an abnormal location artificially generated by image processing is newly added. In other words, it is image information to which noise is artificially added.

The fourth image information generation unit 16D is configured with a generative model using GAN (Generative Adversarial Networks), learns the third image information D3, and generates image information for padding from the first image information D1. The fourth image information D4 is generated.
That is, using a generation model that has learned the third image to which the abnormality area has been newly added, fourth image information D4 having the abnormality area for padding is newly generated from the first image information D1 having no abnormality area.
Therefore, the more the fourth image information generation unit 16D learns the third image information D3, the more it can generate various kinds of fourth image information D4 having various abnormalities. In other words, It is designed to be able to generate the fourth image information D4 without spending much time and effort.

The anomaly detection model learning section 16E uses both the second image information D2 and the fourth image information D4 as teacher data DT to learn an anomaly detection model 22 of the robot 18, which will be described later.
That is, the second image information D2 is an actual image of the detection target location of the building, and therefore has a limited amount of information.
On the other hand, since the fourth image information D4 is generated by a generative model using GAN, it is easy to generate a large amount of fourth image information D4. The larger the amount of information of the teacher data DT for learning the abnormality detection model 22 of the robot 18, which will be described later, can be increased. In other words, it is advantageous in ensuring both the quality and quantity of the teacher data DT without taking much time and effort.

Next, the robot 18 autonomously moves outside or inside the building, and detects abnormalities in the building while patrolling along a predetermined route.
The robot 18 includes a robot side imaging section 20, an abnormality detection model 22, a moving section 24, a map database 26, a positioning section 28, a movement control section 30, a notification information generation section 32, a recording section 34, a communication section 36, etc. has been done.

The robot-side imaging unit 20 captures an image of a building while the robot 18 is patrolling, and generates image information of a target location on the building.

The anomaly detection model 22 detects anomalies based on the image information of the target part of the building captured by the robot-side imaging unit 20. The accuracy of abnormality detection can be improved.

The moving unit 24 moves the robot 18, and includes, for example, a drive wheel rotated by a motor, a turning wheel rotated by a motor, etc., and moves the robot 18 along the floor surface. It is something.
The moving unit 24 is not limited to a configuration including a drive wheel and a turning wheel, and may be configured as the moving unit 24 of various conventionally known robots 18, such as a configuration used in a quadruped walking robot 18.

The map database 26 records map information on the inside and outside of buildings and predetermined tour routes on the map information.

The positioning unit 28 measures the current position of the robot 18.
In this embodiment, positioning beacons (not shown) are installed in advance along the patrol route.
In the map database 26, the location where the beacon is installed and the identification information of the beacon are recorded in association with each other.
The positioning unit 28 performs positioning of the current position by specifying from the map database 26 the position linked to the identification information received from the beacon.
Note that the positioning unit 28 may use various conventionally known positioning methods, such as one that positions the current position based on a positioning signal received from a positioning satellite such as GPS.
However, using a beacon is advantageous in accurately determining the current position even in cases where the positioning signal from the positioning satellite is blocked by buildings.

The movement control unit 30 controls the movement unit 24 to autonomously move along the tour route based on the current position determined by the positioning unit 28 and map information in the map database 26 .

When the abnormality detection model 22 detects an abnormality, the notification information generation unit 32 generates abnormality detection image information obtained by adding an abnormality display mark indicating the abnormality to the image information captured by the robot-side imaging unit 20; This is to generate notification information that is associated with position information indicating the current position of the robot 18 at the time when the abnormality is detected, and to record it in the recording unit 34.
Note that the notification information may include map information in a range including the tour route among the map information in the map database 26.
The recording unit 34 records and holds broadcast information.
The communication unit 36 transmits the notification information recorded in the recording unit 34 to a terminal such as a tablet terminal or a personal computer, and thereby allows the terminal to process the notification information as appropriate to confirm abnormalities. can be done.

Next, the operation of the learning device 10 will be explained with reference to the schematic diagram showing the flow of information in FIG. 2 and the flowchart in FIG. 3.
First, the first image information generation unit 16A generates first image information D1, which is normal image information of a part of the building to be detected, from BIM information, which is three-dimensional information of the building including shape information and attribute information. is generated (step S10).
Next, the second image information generation unit 16B obtains second image information D2 obtained by actually capturing an image of the detection target location of the building (step S12). In this embodiment, the image information of the building imaged by the imaging section 12 is supplied in advance to the second image information generation section 16B as the second image information D2.
Next, the third image information generation section 16C generates at least one of the first image information D1 generated by the first image information generation section 16A and the second image information D2 generated by the second image information generation section 16B. By performing image processing, third image information D3 to which abnormal locations are newly added is generated (step S14).
Next, the fourth image information generation unit 16D learns the third image information D3 using a generation model using GAN (step S16).
Next, the generation model (fourth image information generation unit 16D) that has learned the third image information D3 generates fourth image information D4 as image information for padding from the first image information D1 (step S18).
Next, the anomaly detection model learning unit 16E uses the second image information D2 and the fourth image information D4 as teacher data DT to learn the anomaly detection model 22 of the robot 18 (step S20), and performs a series of processes. finish.
By repeatedly performing such processing, the learning effect of the abnormality detection model 22 is improved.

Note that once the anomaly detection model 22 of the robot 18 has been trained, the robot 18 is used to patrol the outside or inside of the building, and the anomaly detection model 22 is trained based on the image information captured by the robot side imaging unit 20. Detects abnormalities in buildings and generates notification information. Workers inspect, repair, and repair abnormalities in buildings based on the notification information.
It should be noted that the anomaly detection model 22 does not need to be provided in the robot 18 from the beginning; instead, the anomaly detection model 22 separated from the robot 18 may be trained by the learning device 10, and then the anomaly detection model 22 may be mounted on the robot 18. Optional.

As explained above, according to the present embodiment, the first image information D1, which is normal image information of the part of the building to be detected, is generated from the BIM information, which is the three-dimensional information of the building, and A new abnormal location is added by obtaining second image information D2 obtained by actually capturing an image of the detection target location of the object, and performing image processing on at least one of the first image information D1 and the second image information D2. The third image information D3 is generated, the third image information D3 is learned by a generative model using GAN, and the fourth image information D4 as image information for padding is generated from the first image information D1 by the learned generative model. The abnormality detection model 22 is trained using the second image information D2 and the fourth image information D4 as teacher data DT.
Therefore, when learning the anomaly detection model 22, even if only a small number of second image information D2 can be prepared, a large number of fourth image information D4 as padded data can be easily generated, and the second image information D2 and the fourth image Since the anomaly detection model 22 can be trained using the information D4 as the teacher data DT, it is advantageous to improve the learning (machine learning) effect of the anomaly detection model 22 without spending time and effort. This is advantageous in increasing the accuracy of detecting an abnormal location by the 22 and reliably detecting an abnormal location.
Furthermore, in this embodiment, the abnormal location includes at least one of cracks, peeling, and stains occurring in the building, which is advantageous in reliably detecting these cracks, peelings, and stains.

(Second embodiment)
Next, a landscape information generation device for a building according to a second embodiment will be described together with a landscape information generation method and a computer program.
A building landscape information generation device (hereinafter referred to as a landscape information generation device) generates building landscape information with high utility value by inputting BIM information of a building into a generation model using GAN.
As shown in FIG. 4, the landscape information generation device 40 is configured by a computer 42. As shown in FIG.
Also connected to the computer 42 are an input device 44 (keyboard, mouse), an output device 46 (display, printer), and a recording device 48 (external recording medium such as a hard disk and a memory card).
That is, the computer 42 includes a CPU, a ROM that stores and stores control programs, a RAM that serves as an operating area for the control programs, an interface section that interfaces with peripheral circuits, and the like.
The computer 42 functions as a BIM information recording section 42A, an input section 42B, and a building landscape information generation section 42C when the CPU executes a control program.
The BIM information recording unit 42A records BIM information, which is three-dimensional information of the building, including shape information indicating the shape of the elements constituting each part of the building, and attribute information indicating the attributes of the elements. BIM information generated through the design process is recorded.
As described in the first embodiment, BIM information includes shape information indicating the shape of elements constituting each part of a building and attribute information indicating attributes of the elements.
In the second embodiment, the elements constituting each part of the building include a roof, an outer wall, a window, a door, and the like that constitute an external image of the building viewed from the outside. Also included are ceilings, inner walls, floors, windows, doors, doors, etc. that constitute an internal image of a building viewed from inside.
Shape information is information indicating the shape of the element.
Attribute information is information indicating the dimensions, material, color, etc. of the element.

The input unit 42B receives (inputs) environmental conditions surrounding the building.
The environmental conditions include at least one of the following: time of day, season of year, and weather.
If the time of the day is specified as the environmental condition, the input unit 42B inputs, for example, the presence or absence of sunlight, the angle of sunlight, It is converted into specific conditions such as color temperature and spectral distribution.
If a season of the year is specified as the environmental condition, the input unit 42B converts it into specific conditions such as the angle of sunlight, color temperature, and spectral distribution according to the four seasons.
If the weather is specified as the environmental condition, the input unit 42B converts it into specific conditions such as light intensity, color temperature, and spectral distribution according to the weather such as clear, sunny, slightly cloudy, cloudy, rainy, and snowy. accept.
Furthermore, environmental conditions are not limited to the conditions related to sunlight mentioned above, but include, for example, the amount of light, color temperature, spectral distribution, direction of light irradiation, etc. of lighting devices installed outside or inside the building. It may include.

The building landscape information generation unit 42C learns the various environmental conditions described above using a generative model using GAN (Generative Adversarial Networks), and generates an external image of the building as seen from the outside or an external image of the building from the BIM information. This is to generate building landscape information that shows an internal image as seen from the inside. Which building scenery information, an external image or an internal image, is to be generated may be set in advance in the building scenery information generation section 42C.
Note that the environmental conditions for generating the building landscape information are separately designated to the building landscape information generation section 42C.

Next, the operation of the landscape information generation device 40 will be explained with reference to the flowchart shown in FIG.
First, BIM information is recorded in the BIM information recording section 42A (step S50).
BIM information may be supplied from an external computer via a communication line (network) or via an external recording medium.
Next, the environmental conditions are received by the input unit 42B via the input device 44 (step S52).

Next, the building landscape information generation unit 42C learns environmental conditions using a generative model using GAN (step S54).
Next, using the generative model that has learned the environmental conditions, building landscape information indicating an external image of the building viewed from the outside or an internal image of the building viewed from the inside is generated from the BIM information (step S56).
At this time, the environmental conditions for generating the building scenery information are specified to the building scenery information generating section 42C via the input device 44, for example.
Next, the generated building landscape information is output (step S58).
Specifically, it is visually displayed on a display, printed out on a paper medium by a printer, recorded on an external recording medium, or supplied to another terminal (computer 42) via a communication line. Ru.

According to the second embodiment, BIM information, which is three-dimensional information of a building, including shape information indicating the shape of elements constituting each part of the building and attribute information indicating attributes of the elements, is prepared, A generative model using GAN learns the environmental conditions surrounding the building, and the generative model that has learned the environmental conditions creates an external image of the building as seen from the outside or an internal image of the building as seen from the inside from BIM information. The building landscape information that shows the information is now generated.
Therefore, it goes without saying that building landscape information can be generated accurately based on the shape information and attribute information of BIM information, and since a generation model that has learned environmental conditions is used, the obtained building landscape information reflects the environmental conditions. It becomes what it is.
For example, it is possible to easily obtain architectural landscape information that corresponds to time zones such as morning, noon, and night, seasonal changes, and weather changes such as sunny, cloudy, and rainy.
Therefore, when public relations materials (catalogs) for buildings are created using building landscape information, images of buildings that reflect various environmental conditions can be presented, making the content of public relations materials diverse and appealing to customers. This will be advantageous in improving the advertising effect for

10 Learning device 12 Imaging unit 14 BIM information generation unit 16 Computer 16A First image information generation unit 16B Second image information generation unit 16C Third image information generation unit 16D Fourth image information generation unit 16E Anomaly detection model learning unit 18 Robot 40 Landscape information generation device 42 Computer 42A BIM information recording section 42B Input section 42C Building landscape information generation section 44 Input device 46 Output device 48 Recording device

Claims (4)

A learning method for an anomaly detection model that detects anomalies based on image information of a building, the method comprising:
Detection of the building from BIM (Building Information Modeling) information, which is three-dimensional information of the building, including shape information indicating the shape of elements constituting each part of the building, and attribute information indicating attributes of the elements. generating first image information that is normal image information of the target portion;
obtaining second image information that is an actual image of the detection target location of the building;
performing image processing on at least one of the first image information and the second image information to generate third image information to which an abnormal location is newly added;
learning the third image information by a generative model using GAN (Generative Adversarial Networks);
generating fourth image information as image information for padding from the first image information using the generative model learned using the third image information;
learning the anomaly detection model using the second image information and the fourth image information as training data;
A learning method for a building anomaly detection model characterized by including the following. The abnormal location includes at least one of cracks, peeling, and staining occurring in the building.
2. A learning method for a building anomaly detection model according to claim 1. A computer program that causes a computer to execute the learning method for a building anomaly detection model according to claim 1 or 2. A learning device for an anomaly detection model that detects anomalies based on image information of a building,
an imaging unit that images the building and generates the image information;
Detection of the building from BIM (Building Information Modeling) information, which is three-dimensional information of the building, including shape information indicating the shape of elements constituting each part of the building, and attribute information indicating attributes of the elements. a first image information generation unit that generates first image information that is normal image information of the target portion;
a second image information generation unit that obtains second image information obtained by actually capturing an image of the detection target location of the building by the imaging unit;
a third image information generation unit that generates third image information to which an abnormal location is newly added by performing image processing on at least one of the first image information and the second image information;
a fourth image information generation unit configured with a generative model using GAN (Generative Adversarial Networks), which learns the third image information and generates fourth image information as image information for padding from the first image information; and,
an anomaly detection model learning unit that learns the anomaly detection model using the second image information and the fourth image information;
A learning device for a building anomaly detection model, comprising:

Images