JP7824909B2 - Building Management Systems and Programs - Google Patents

Description

The present invention relates to a building management system and program.

For example, Patent Document 1 discloses an invention for a work management support device aimed at efficiently performing work related to structures such as social infrastructure (particularly repair work and inspection work). The work management support device has an imaging means capable of acquiring real-world images of a target area including the structure, and a support information generation means that acquires three-dimensional shape information corresponding to the target area and generates virtual support information indicating that virtual work has been performed over time on the three-dimensional shape information. The real-world images and virtual support information are synchronized in real time and can be displayed on a display means. This invention is said to be able to visualize the progress of work, facilitating consensus building among multiple parties, for example, on the target area, work content, and estimates of work-related quantities for work such as repairing, renovating, and replenishing structures.

For example, Patent Document 2 discloses an invention for an interior construction management system for interior construction work such as restoration. The interior construction management system comprises: a drawing registration means for registering drawing information for each property provided by the client; a photo registration means for registering photographic information based on actual photographs taken of the interior of each property; an estimate creation means for determining restoration work and setting the dimensions of the area for each restoration work based on the drawings and/or photographs registered by the drawing registration means and the photo registration means; an estimate creation means for creating an estimate that displays at least a list of products, quantities, and prices of items selected as necessary for each restoration work based on the set dimensional information, deleting any undesired products, and creating a new estimate; and a schedule creation means for creating a restoration work schedule based on the selected products and dimensional information. This invention is said to enable construction personnel to place orders, accept orders, and work in a focused and efficient manner, while simultaneously clarifying and reducing costs.

Japanese Patent Publication No. 2022-155553 Japanese Patent Publication No. 2020-155124

The work management support device described in Patent Document 1 synchronizes virtual support information and real-world images in real time, making it effective for understanding work progress. However, while the invention described in this document includes the idea of generating objects from 3D shape information as virtual support information, it does not include the idea of associating these objects with BIM (Building Information Modeling) data. In other words, the 3D shapes captured and acquired by the work management support device described in this document are diverse and not standardized, and the objects generated are also diverse, making organization when calculating construction estimates and other data cumbersome and hindering the accuracy of estimates. Furthermore, managing a wide variety of objects places a heavy processing load on the computer system, and there is the disadvantage that expensive hardware resources must be provided to provide a good system usage environment.

Furthermore, the interior construction management system in Patent Document 2 allows construction personnel to place orders, receive orders, and perform work in a centralized and efficient manner, while simultaneously clarifying and reducing costs. However, with the invention described in this document, the dimensions of the work area are set from registered drawings and photographs, which means that creating the drawings is time-consuming and the dimensions read from the photographs may not be accurate. Furthermore, there is no concept of treating products, etc. as objects and associating BIM data with those objects. Therefore, as with the business management support device in Patent Document 1, organizing the calculation of construction estimates, etc. becomes cumbersome, which can hinder the accuracy of the estimates.

The object of the present invention is to provide a building management system that measures and acquires three-dimensional data of the interior of a building with simple operations, acquires the walls, windows, doors, housing equipment, etc. within the interior as objects to which individual attribute information can be assigned, and then assigns BIM data, which serves as attribute information, to the objects. This allows building management tasks such as creating estimates to be performed quickly and easily using inexpensive devices, without using expensive computer resources.

Note that the BIM data in this invention does not refer to the broader BIM data used in all construction information modeling methods, but rather refers to simplified three-dimensional data and attribute information limited to data related to building maintenance management, renovations, etc. used in the building management system of this invention, and these are collectively referred to as BIM data in this specification. Examples of attribute information for BIM data in this invention include the common code of the corresponding object, company code, large/medium/subcategory, manufacturer name, product name, product code, size, material unit price, construction unit price, unit, remarks, various codes, and associated repair work.

In order to solve the above problem, a first aspect of the present invention provides a building management system having a mobile information terminal equipped with a 3D data generation function that captures an image of the interior of a room in a building and generates 3D data of the interior of the room by measuring the distance from the imaging point to the imaged object, an object generation function that generates, from the 3D data, at least one object selected from wall objects corresponding to each of the walls of the room, fixture objects corresponding to each of one or more fixtures installed in the room, and household equipment objects corresponding to each of one or more household equipment placed in the room, and a communication function that transmits building information including multiple wall objects for at least one of the rooms, a communication unit that receives the building information, a white model generation unit that combines the multiple wall objects included in the building information received by the communication unit to generate a white model of the building including at least one of the rooms, and a BIM data assignment unit that assigns BIM data to at least one target object selected from the objects of the white model, the fixture objects, and the household equipment objects.

The system may further include a white model shaping unit that shapes the white model by applying corrections to at least one of the wall objects that make up the white model.

The system may further include an object image generation unit that generates a two-dimensional or three-dimensional object image representing at least one object selected from the white model object, the fixture object, and the housing equipment object, and the communication unit may transmit data of the object image and list data of all or part of the BIM data to the mobile information terminal or other information processing device. In this case, the communication unit may receive a request from the mobile information terminal or other information processing device that includes designation of at least one object represented in the object image and BIM data corresponding to the designated object, and in response to the request, the BIM data allocation unit may change or newly assign the BIM data of the designated object to the BIM data included in the request.

The system may further include an estimate generation unit that references all or part of the BIM data and generates an estimate of the repair and renovation costs of the building, and the communication unit may transmit the estimate data to the mobile information terminal or other information processing device.

The system may further include an inference unit that receives at least one input object selected from the white model object, the fixture object, and the housing equipment object, and outputs, as a result of inference, one or more BIM data candidates corresponding to the input object. The inference unit may perform inference using a supervised trained neural network trained using the input object and correct answer information corresponding to the input object, or an unsupervised trained neural network trained to appropriately classify and output BIM data corresponding to the input object. In this case, the BIM data allocation unit may automatically assign to the assignment target object the BIM data candidate with the highest probability among the BIM data candidates output by the inference unit as a result of inference using the assignment target object as input to the mobile information terminal or other information processing device used by the user. Alternatively, the system may further include a communication unit that transmits to the mobile information terminal or other information processing device used by the user the BIM data candidate output by the inference unit as a result of inference using the assignment target object as input. The BIM data allocation unit may assign to the assignment target object the BIM data candidate corresponding to the user's selection operation among the BIM data candidates transmitted by the communication unit.

The system may further include a grade registration unit that registers a grade for at least one registered object selected from the white model object, the fixture object, and the housing equipment object, and the BIM data assignment unit may assign BIM data corresponding to the grade registered in the grade registration unit to the registered object.

The system may further include a comment information storage unit that stores comment information related to at least one selected object selected from the white model objects, the fixture objects, and the home appliance objects, linked to the selected object. In this case, the comment information storage unit may further store position information indicating the position of a focus point in the selected object, and may further include an object image generation unit that generates a two-dimensional or three-dimensional object image representing one or more objects selected from the white model objects, the fixture objects, and the home appliance objects, including at least the selected object, and a focus point image generation unit that generates a focus point image indicating the focus point of the selected object so as to be superimposed on the object image, and the communication unit may transmit data of the object image with the focus point image superimposed and the comment information to the mobile information terminal or other information processing device.

The comment information storage unit may further store image data or electronic data related to the comment information, in which case the communication unit transmits the image data or electronic data, in addition to the image data and comment information of the object, to the mobile information terminal or other information processing device.

Furthermore, the building information may include captured image data of the interior of the room at a position corresponding to the point of interest, the comment information storage unit may further store the captured image data in association with the point of interest, and the communication unit may transmit the captured image data in addition to the comment information and data of the object image on which the point of interest image is superimposed.

The comment information storage unit may store imaging information indicating the shooting position and shooting direction when the captured image data was acquired, and the focus point image generation unit may generate the focus point image and display the shooting position and shooting direction superimposed on the object image.

The system may further include a drawing/perspective generation unit that uses the BIM data of the target objects assigned by the BIM data assignment unit to generate a floor plan, three-dimensional drawing, or perspective drawing of the building composed of the target objects.

In a second aspect of the present invention, a program is provided for causing a computer to implement the following functions from a mobile information terminal: a 3D data generation function that captures an image of the interior of a room in a building and generates 3D data of the interior of the room by measuring the distance from the imaging point to the imaged object; an object generation function that generates, from the 3D data, at least one object selected from wall objects corresponding to each of the walls of the room, fixture objects corresponding to each of one or more fixtures installed in the room, and household equipment objects corresponding to each of one or more household equipment installed in the room; and a communication function that transmits building information including a plurality of the wall objects for at least one of the rooms; a communication function that receives the building information from the mobile information terminal; a white model generation function that combines the wall objects included in the building information received by the communication unit to generate a white model of the building including at least one of the rooms; and a BIM data assignment function that assigns BIM data to at least one target object selected from the objects of the white model, the fixture objects, and the household equipment objects.

Note that the above summary of the invention does not list all of the necessary features of the present invention. Subcombinations of these features may also constitute inventions.

1 is a conceptual diagram showing an example of a network system 10 that uses a building management system 100. FIG. 1 is a functional block diagram showing an example of a building management system 100, a mobile information terminal 200, and an information processing device 300. FIG. FIG. 2 is a functional block diagram illustrating an example of a white model shaping unit 106. FIG. 2 is a flow diagram showing an example of the operation of the building management system 100. FIG. 10 is a flow diagram showing an example of a white model shaping process 512. FIG. 10 is a flow diagram illustrating an example of a BIM data allocation process 518. FIG. 10 is a flow diagram illustrating another example of the BIM data allocation process 518. FIG. 10 is a flow chart showing another example of the operation of the building management system 100.

The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention as claimed. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

Figure 1 is a conceptual diagram showing an example of a network system 10 that uses a building management system 100. The network system 10 includes a building management system 100, a mobile information terminal 200, an information processing device 300, and a network 400, and the building management system 100 and the mobile information terminal 200 are connected to each other via the network 400, and the building management system 100 and the information processing device 300 are connected to each other via the network 400.

The building management system 100 is an information processing device 300 that functions as a server, and the mobile information terminal 200 is a portable or mobile information processing device 300 such as a smartphone or pad computer. The information processing device 300 is a general information processing device 300 such as a desktop personal computer or laptop personal computer. The mobile information terminal 200 and information processing device 300 function as clients that issue requests to the building management system 100, which serves as the server, and process responses to those requests from the server. The network 400 is a data communication means that connects the client mobile information terminals 200 and information processing devices 300 to the building management system 100, which serves as the server. Examples of the network 400 include a wireless LAN (Local Area Network) or wired LAN conforming to the Internet Protocol, a mobile phone network, Ethernet, a serial communication line, Bluetooth (registered trademark) wireless communication, etc.

The hardware configurations of the building management system 100, mobile information terminal 200, and information processing device 300 are all common, and include, for example, a motherboard, a CPU (Central Processing Unit) connected to a system bus on the motherboard, cache memory, main memory, a GPU (Graphics Processing Unit), graphics memory, mass storage devices such as HDDs (Hard Disk Drives) and SSDs (Solid State Drives), input devices such as keyboards and pointing devices, and output control devices such as displays and printers. By cooperating with this hardware through software, the building management system 100, mobile information terminal 200, and information processing device 300 can realize the functions described below.

Figure 2 is a functional block diagram showing an example of a building management system 100, a mobile information terminal 200, and an information processing device 300.

The building management system 100 has a communication unit 102, a white model generation unit 104, a white model shaping unit 106, a BIM data allocation unit 108, an inference unit 110, a grade registration unit 112, an object image generation unit 114, an estimate generation unit 116, a comment information storage unit 118, a focus point image generation unit 120, a memory unit 122, a control unit 124, and a drawing/perspective generation unit 126. The mobile information terminal 200 has a camera module 202, a LiDAR (Light Detection and Ranging) module 204, a 3D data generation unit 206, an object generation unit 208, a communication unit 210, a display input unit 212, a memory unit 214, and a control unit 216. The information processing device 300 has an input unit 302, an output unit 304, a display unit 306, a memory unit 308, a communication unit 310, and a control unit 312.

The camera module 202 captures images of the interior of a room in the building, and the LiDAR module 204 measures the distance from the imaging point to the imaged object. Based on the measurements by the LiDAR module 204, the 3D data generation unit 206 generates 3D data of the interior of the room. The object generation unit 208 generates objects within the room from the 3D data generated by the 3D data generation unit 206. Examples of objects generated include wall objects corresponding to each wall of the room, fixture objects corresponding to one or more fixtures installed in the room, and home appliance objects corresponding to one or more home appliances placed in the room. The object generation unit 208 generates at least one of these example objects. The communication unit 210 transmits and receives data. The communication unit 210 transmits building information including multiple wall objects for at least a single room generated by the object generation unit 208.

The display input unit 212 is, for example, a liquid crystal display equipped with a touch panel, and the memory unit 214 stores programs and data. The control unit 216 controls the camera module 202, LiDAR module 204, 3D data generation unit 206, object generation unit 208, communication unit 210, display input unit 212, and memory unit 214.

The input unit 302 is, for example, a pointing device such as a keyboard or mouse, and the output unit 304 is, for example, a printer. The display unit 306 is, for example, an LCD display, and the memory unit 308 is, for example, a mass storage device such as a HDD. Programs and data are recorded in the memory unit 308. The communication unit 310 sends and receives data. The control unit 312 controls the input unit 302, output unit 304, display unit 306, memory unit 308, and communication unit 310.

The communication unit 102 transmits and receives data to and from the mobile information terminal 200 or the information processing device 300. In particular, the communication unit 102 receives building information transmitted from the mobile information terminal 200. As described above, the building information includes multiple wall objects for at least a single room generated by the object generation unit 208. The communication unit 102 also transmits object image data and list data of all or part of the BIM data to the mobile information terminal 200 or the information processing device 300.

The white model generation unit 104 combines multiple wall objects included in the building information received by the communication unit 102 to generate a white model of a building that includes at least one room.

The white model shaping unit 106 shapes the white model by applying corrections to at least one of the wall objects that make up the white model. Figure 3 is a functional block diagram showing an example of the white model shaping unit 106. For example, the white model shaping unit 106 has the following correction functions: tilt correction 150, distortion correction 152, module pitch correction 154, unnecessary wall removal correction 156, small room removal correction 158, straightening correction 160, perimeter correction 162, and wall division correction 164.

Tilt correction 150 rotates the white model to correct tilt. Distortion correction 152 corrects two wall objects so that they are in a straight line parallel to the X or Y axis. Module pitch correction 154 moves each object so that it aligns with the grid lines. Unnecessary wall removal correction 156 removes unnecessary wall objects. Small room removal correction 158 removes wall objects that make up small rooms. Straightening correction 160 combines two wall objects into one. Periphery correction 162 adds missing peripheral walls. Wall division correction 164 divides wall objects.

The BIM data assignment unit 108 assigns BIM data to at least one target object selected from white model objects, fixture objects, and housing equipment objects.

The inference unit 110 can be used to assign BIM data. The inference unit 110 receives at least one input object selected from white model objects, fixture objects, and housing equipment objects, and outputs, as the result of inference, one or more BIM data candidates corresponding to the input object. The inference unit 110 performs inference using a supervised trained neural network that has been trained using the input object and correct answer information corresponding to the input object. Alternatively, the inference unit 110 performs inference using an unsupervised trained neural network that has been trained to appropriately classify and output BIM data corresponding to the input object.

As an example of using the inference unit 110, the BIM data allocation unit 108 can automatically assign to the allocation target object the BIM data with the highest probability among the BIM data candidates output by the inference unit 110 as a result of inference using the allocation target object as input.

Alternatively, as another example using the inference unit 110, the BIM data allocation unit 108 can allocate BIM data corresponding to the user's selection from among the BIM data candidates presented to the user as a result of the inference to the allocation target object. That is, the communication unit 102 transmits the inference results of the inference unit 110 to the mobile information terminal 200 or information processing device 300 used by the user, and the BIM data allocation unit 108 allocates BIM data corresponding to the user's selection operation from among the BIM data candidates transmitted by the communication unit 102 to the allocation target object.

The BIM data allocation unit 108 can also allocate BIM data to objects (registered objects) according to the grade registered in the grade registration unit 112. The grade registration unit 112 registers a grade for at least one registered object selected from white model objects, fixture objects, and housing equipment objects.

It is also possible to change BIM data that has already been assigned, or to assign new BIM data to an unassigned object. In this case, the communication unit 102 receives a request from the mobile information terminal 200 or the information processing device 300 that specifies at least one object and includes BIM data corresponding to the specified object, and in response to the request, the BIM data assignment unit 108 changes or assigns new BIM data for the specified object to the BIM data included in the request.

The object image generation unit 114 generates a two-dimensional or three-dimensional object image representing at least one object selected from a white model object, a fixture object, and a home appliance object. The communication unit 102 transmits data of the generated object image to the mobile information terminal 200 or the information processing device 300.

The estimate generation unit 116 references all or part of the BIM data to generate an estimate for the cost of repairing and renovating the building. The communication unit 102 transmits the generated estimate data to the mobile information terminal 200 or the information processing device 300.

The comment information storage unit 118 stores comment information related to at least one selected object selected from white model objects, fixture objects, and home appliance objects, by linking it to the selected object.

The comment information storage unit 118 can further store position information indicating the position of the focus point in the selected object. In this case, the focus point image generation unit 120 generates a focus point image indicating the focus point of the selected object so that it is superimposed on the object image generated by the object image generation unit 114. Data on the object image with the superimposed focus point image and comment information are transmitted by the communication unit 102 to the mobile information terminal 200 or the information processing device 300.

The comment information storage unit 118 can further store image data or electronic data related to the comment information. In this case, the communication unit 102 transmits the image data or electronic data of the object, as well as the comment information, to the mobile information terminal 200 or the information processing device 300.

Furthermore, if the building information received by the communication unit 102 includes captured image data of the interior of a room at a position corresponding to the point of interest, the comment information storage unit 118 stores the captured image data in association with the point of interest, and the communication unit 102 transmits the captured image data to the mobile information terminal 200 or the information processing device 300, in addition to the data of the object image on which the point of interest image is superimposed and the comment information.

The comment information storage unit 118 can store imaging information indicating the shooting position and shooting direction when the captured image data linked to the point of interest was acquired. In this case, the point of interest image generation unit 120 generates the point of interest image and also generates the shooting position and shooting direction so that they are superimposed on the object image.

The drawing/perspective generation unit 126 uses the BIM data of the target objects assigned by the BIM data assignment unit 108 to generate floor plans, three-dimensional drawings, or perspective drawings of the building made up of the target objects.

The memory unit 122 is a mass storage device such as a HDD, and stores programs and data. In particular, the memory unit 122 stores BIM data and object information. The control unit 124 controls the communication unit 102, white model generation unit 104, white model shaping unit 106, BIM data allocation unit 108, inference unit 110, grade registration unit 112, object image generation unit 114, estimate generation unit 116, comment information storage unit 118, focus point image generation unit 120, memory unit 122, and drawing/perspective generation unit 126.

Figure 4 is a flow diagram showing an example of the operation of the building management system 100. First, the camera module 202 of the mobile information terminal 200 captures an image of the interior of the building, and the LiDAR module 204 measures the distance to objects present in the room (step 500). From the distance measurement results obtained by the LiDAR module 204, the three-dimensional data generation unit 206 generates three-dimensional data (3D data) of the interior (step 502). Furthermore, from the generated 3D data, the object generation unit 208 generates objects for the interior (step 504).

The generated object has category information as its attributes, including the category that indicates the category into which it falls, as well as the object's identification information (UUID: Universally Unique Identifier), location information, size information such as height and width, and identification information of related objects if they are related to other objects. Examples of categories include walls, doors, openings, windows, bathtubs, beds, chairs, dishwashers, fireplaces, ovens, refrigerators, sinks, sofas, stairs, storage, stoves, desks, televisions, toilets, and washer/dryer. In this specification, objects that belong to the building structure category, such as walls and wall openings, are referred to as wall objects, objects that belong to the fixture category, such as windows and doors, are referred to as fixture objects, and objects that belong to the home appliance category, such as bathtubs, beds, chairs, dishwashers, and fireplaces, are referred to as home appliance objects.

The generated object (including category information) is transmitted to the building management system 100 as building information (step 506), and the communication unit 102 of the building management system 100 receives the building information (step 508). Note that the building information may include image data captured in step 500 or separately captured image data. The generated object may be transmitted to the building management system 100 in real time in synchronization with the generation of the object, or the generated object may be temporarily recorded in the memory unit 214 and then transferred from the memory unit 214 to the building management system 100.

The objects received by the communication unit 102 in step 508 are input to the white model generation unit 104, which generates a white model (step 510). By generating a white model, it is possible to place fittings objects corresponding to fittings and household equipment objects corresponding to household equipment in the generated white model, making it possible to simulate the interior structure that makes up a room in a real building.

The generation of a white model is performed, for example, according to the following algorithm. That is, the system references the wall objects included in the received object, and uses the object identification information (UUID), position information, and size information included in the category information of the wall object to construct vertex information for the wall object from the wall object's position information and width information included in the size information, and determines whether the vertices of each wall object are connected. Connected wall objects are connected in order, and if the start and end points of the vertices of a connected series of wall objects match, the space enclosed by that series of wall objects is recognized as a room. This determination and recognition process is performed for all wall objects, recognizing single or multiple rooms, and wall objects not determined to constitute a room are recognized as walls. In this way, a white model consisting of single or multiple rooms and walls is constructed.

However, the white model generated in step 510 may contain distorted walls that should be straight, may include unnecessary walls, may form rooms that do not exist in reality, or may lack walls that should be there. These defects are noise caused by improperly generated wall objects, and deviate from the actual room structure. If these deviations remain, the image simulating the room may display distorted or missing walls, or non-existent walls, and the fixtures and fixtures and fixtures placed in the room may appear unnatural, which is inconvenient. Furthermore, if wall objects that should be there are missing or unnecessary wall objects remain, the BIM data that will be assigned later will not be assigned appropriately. Therefore, the white model is shaped to eliminate any deviations between the actual room structure and the white model (step 512).

Figure 5 is a flow diagram showing an example of the white model shaping process. Shaping of the white model is processed by the white model shaping unit 106 in the following order: tilt correction (step 540), which rotates the white model to correct tilt; distortion correction (step 542), which corrects two wall objects so that they are straight lines parallel to the X-axis or Y-axis; module pitch correction (step 544), which moves each wall object along a grid line; unnecessary wall deletion correction (step 546), which deletes unnecessary wall objects; small room deletion correction (step 548), which deletes wall objects that make up small rooms; straightening correction (step 550), which combines two wall objects into one; perimeter correction (step 552), which complements missing outer walls; and wall division correction (step 554), which divides wall objects. It is not necessary to perform all of the correction processes from step 540 to step 554; any of the correction processes may be performed as needed.

By shaping the white model, the discrepancy between the actual room structure and the white model is eliminated, and BIM data can be appropriately assigned to the wall objects that make up the white model.

Next, the grade registration unit 112 of the building management system 100 generates a grade registration request and sends it to the client side (step 514). Note that after the mobile information terminal 200 sends the building information to the building management system 100 in step 506, the client-side processing may continue to be performed by the mobile information terminal 200, or, after appropriate login processing, an information processing device 300 other than the mobile information terminal 200 may take over processing as a client. Therefore, the grade registration request in step 514 is sent to the mobile information terminal 200 or information processing device 300 operating as a client.

The mobile information terminal 200 or information processing device 300 that receives the grade registration request performs the grade registration operation (step 516), and the BIM data allocation unit 108 of the building management system 100 that received the grade registration allocates the BIM data corresponding to the registered grade to the wall objects, fittings objects, and housing equipment objects that make up the white model (step 518). Note that if no grade is registered, a pre-set default grade may be applied.

In a broad sense, BIM data refers to data used in modeling methods for information about buildings. However, the BIM data in this invention does not refer to the broad BIM data used in all modeling methods for building information, but rather refers to simplified three-dimensional data and attribute information limited to data related to building maintenance and management, renovations, etc. used in the building management system 100, and these are collectively referred to as BIM data. Examples of attribute information for BIM data in this invention include the common code for the corresponding object, company code, large/medium/small classification, manufacturer name, product name, product code, size, material unit price, construction unit price, unit, remarks, various codes, and associated repair work.

The assignment of BIM data in step 518 does not need to be performed for all wall objects, fixture objects, and home appliance objects; BIM data may be assigned only to some of the wall objects, fixture objects, or home appliance objects. It is also possible to skip step 518 without assigning BIM data to all objects.

The BIM data allocation in step 518 may be performed using the inference results from the inference unit 110. Figure 6 is a flow diagram showing an example of the BIM data allocation process (step 518). First, the BIM data allocation unit 108 identifies a target object that is the target of the BIM data allocation process (step 560) and inputs the target object into the trained model of the inference unit 110 (step 562). The inference unit 110 infers BIM data candidates for the input target object (step 564) and passes the inferred BIM data candidates to the BIM data allocation unit 108. The BIM data allocation unit 108 assigns the BIM data that shows the highest probability among the BIM data candidates to the target object (step 566). The BIM data allocation unit 108 determines whether there are any remaining target objects (step 568). If it determines that there are remaining target objects, it repeats the process from step 560; if it determines that there are no remaining target objects, it terminates the process (step 570). By performing this type of processing, the BIM data that is inferred to be the most appropriate for the target object can be automatically assigned, eliminating the need for user operation for assignment and improving user convenience.

The BIM data allocation in step 518 may also be based on a user selection made with reference to the inference results from the inference unit 110. Figure 7 is a flow chart showing another example of the BIM data allocation process (step 518). Similar to the process in Figure 6, the target object is identified (step 560), input into the trained model (step 562), and BIM data candidates are inferred (step 564). The inferred BIM data candidates are then transmitted to the mobile information terminal 200 or information processing device 300 (step 572). The mobile information terminal 200 or information processing device 300 that has received the BIM data candidates displays the BIM data candidates (step 574), and the user performs a selection operation to select BIM data to be assigned from the displayed BIM data candidates (step 576). The BIM data allocation unit 108, which has received information on the selection results, assigns the selected BIM data selected by the user to the target object (step 578). The BIM data allocation unit 108 determines whether there are any remaining target objects (step 568), and if it determines that there are, repeats the processing from step 560, or if it determines that there are no remaining objects, terminates the processing (step 570). By performing this processing, BIM data according to the user's selection can be allocated from BIM data candidates inferred to be appropriate for the target object, thereby assisting the user in allocating BIM data and improving user convenience.

Next, the object image generation unit of the building management system 100 generates object images for all or some of the wall objects, fixture objects, and home appliance objects (step 520). The object images are three-dimensional or two-dimensional images, and if BIM data is assigned to the object, an image such as a product image associated with the BIM data is generated as the image of the object.

The generated object image is transmitted to the mobile information terminal 200 or information processing device 300 along with the BIM data list data (step 522). Upon receiving the object image, the mobile information terminal 200 or information processing device 300 displays the object image along with the BIM data list data on the display input unit 212 or display unit 306 (step 524). The user of the mobile information terminal 200 or information processing device 300 references the displayed object image and performs operations to add or modify the BIM data (step 526). The results of the addition or modification operations in step 526 are transmitted to the building management system 100, and the BIM data allocation unit 108 adds or reassigns new BIM data for the corresponding object to reflect the transmitted operation results (step 528). The process then returns to step 522 and repeats.

When the user completes the addition and modification operations in step 526 and wishes to request an estimate, the mobile information terminal 200 or information processing device 300 responds to the user's estimate request operation by requesting an estimate from the building management system 100 (step 530). The estimate generation unit 116 of the building management system 100, which receives the estimate request, references the BIM data to generate an estimate and transmits the generated estimate data to the requesting mobile information terminal 200 or information processing device 300 (step 532). The mobile information terminal 200 or information processing device 300, which receives the estimate data, displays it on the display input unit 212 or display unit 306, or outputs it to the output unit 304 (step 534).

In addition to generating the estimate in step 532, the building management system 100 can generate a floor plan, three-dimensional drawing, or perspective drawing (hereinafter referred to as "drawings, etc.") of the building using the drawing/perspective generation unit 126, and can transmit the generated drawings, etc. to the mobile information terminal 200 or information processing device 300 together with or separately from the estimate. In this case, the mobile information terminal 200 or information processing device 300 that receives the drawings, etc. outputs the drawings, etc. after outputting the estimate in step 534 (step 536).

The building management system 100 described above generates a white model of a building's rooms using a mobile information terminal 200 equipped with an object generation function, thereby simplifying tasks such as on-site dimension measurements and drawing up drawings of existing buildings during building renovation work, and significantly reducing work time.

Furthermore, with the building management system 100 described above, the generated white model is shaped and simplified BIM data is assigned to each object, such as walls, fixtures, and household appliances, making it possible to easily create estimates by referencing the product unit prices, etc., in the BIM data.

Furthermore, the building management system 100 described above allows for interactive additions and changes to BIM data, making it easy to add or amend estimates, significantly improving work efficiency. In addition, the ability to interactively add and change BIM data makes it easy to input, manage history, and share repair information in building maintenance management, improving work efficiency.

Furthermore, in the above-mentioned building management system 100, the BIM data of the present invention is limited to simplified BIM data, which reduces hardware resources and keeps the system usage costs low.

The building management system 100 described above can be used for more than just creating estimates; it can also be used for property management, as shown in Figure 8. That is, when the mobile information terminal 200 or information processing device 300 issues a property management request (step 580) and displays (step 584) the object image received from the building management system 100 in response to the request (step 582), the user of the mobile information terminal 200 or information processing device 300 specifies the position of a focus point on the displayed object, enters a comment, and performs operations to select a captured image (step 586), and transmits the position of the focus point, comment, and captured image to the building management system 100 (step 588). The building management system 100 receives the position of the focus point, comment, and captured image (step 590), and the comment information storage unit 118 stores them. The comment information storage unit 118 generates an object image by superimposing a point of interest image (e.g., a pin image) on the point of interest position of the object related to the received point of interest position, comment, and captured image, and transmits this to the mobile information terminal 200 or information processing device 300 (step 592). The mobile information terminal 200 or information processing device 300 displays the received object image with the point of interest image together with the comment (step 594). This allows, for example, when stains or scratches are discovered on a wall of a property, the stain or scratch can be recorded as a point of interest, and the point of interest can be managed by recording and updating comments and images.

The comment information storage unit 118 can store information on the shooting position and shooting direction (imaging information) in addition to the position of the point of interest, comments, and captured images, and can superimpose the shooting position and shooting direction on an object image with the image of the point of interest. The comment information storage unit 118 can also store image data as attribute information for the corresponding selected object (BIM data), and can also store electronic data that can be viewed in the same format as when printed on paper. This image data and electronic data can be displayed and printed out in the same way as comments.

The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

The above-described embodiment can also be understood as a program for running a computer. That is, it can also be understood as a program for causing a computer to implement the following: a 3D data generation function for capturing an image of the interior of a room in a building and measuring the distance from the imaging point to the imaged object to generate 3D data of the interior of the room; an object generation function for generating, from the 3D data, at least one object selected from wall objects corresponding to each of the room's walls, fixture objects corresponding to each of one or more fixtures installed in the room, and household equipment objects corresponding to each of one or more household equipment installed in the room; and a communication function for transmitting building information including multiple wall objects for at least a single room. From a mobile information terminal 200 equipped with the following functions: a communication function for receiving building information; a white model generation function for combining multiple wall objects included in the building information received by the communication unit 102 to generate a white model of a building including at least a single room; and a BIM data assignment function for assigning BIM data to at least one target object selected from the white model objects, fixture objects, and household equipment objects.

The order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings, is not specifically stated as "before," "prior to," or the like, and it should be noted that processes can be performed in any order, unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using terms such as "first," "next," etc. for convenience, this does not mean that the processes must be performed in that order.

10...Network system, 100...Building management system, 102...Communication unit, 104...White model generation unit, 106...White model shaping unit, 108...BIM data allocation unit, 110...Inference unit, 112...Grade registration unit, 114...Object image generation unit, 116...Estimate generation unit, 118...Comment information storage unit, 120...Focus point image generation unit, 122...Memory unit, 124...Control unit, 126...Drawing/perspective generation unit, 150...Tilt correction, 152...Distortion correction, 154...Module pitch correction, 156...Incorrect Main wall removal correction, 158...small room removal correction, 160...straightening correction, 162...perimeter correction, 164...wall division correction, 200...mobile information terminal, 202...camera module, 204...LiDAR module, 206...3D data generation unit, 208...object generation unit, 210...communication unit, 212...display input unit, 214...storage unit, 216...control unit, 300...information processing device, 302...input unit, 304...output unit, 306...display unit, 308...storage unit, 310...communication unit, 312...control unit, 400...network.

Claims (16)

a communication unit that receives the building information from a mobile information terminal that includes a three-dimensional data generation function that captures an image of the interior of a room of a building and generates three-dimensional data of the interior of the room by measuring the distance from the imaging point to the object being imaged, an object generation function that generates, from the three-dimensional data, at least one object selected from a wall object corresponding to each wall of the room, a fixture object corresponding to each of a single or multiple fixtures installed in the room, and a household equipment object corresponding to each of a single or multiple household equipment devices arranged in the room, and a communication function that transmits building information including a plurality of the wall objects for at least one of the rooms;
a white model generation unit that generates a white model of the building including at least one of the rooms by combining a plurality of the wall objects included in the building information received by the communication unit;
a BIM data allocation unit that allocates BIM data to at least one target object selected from the white model object, the fixture object, and the housing equipment object. The building management system according to claim 1 , further comprising a white model shaping unit that shapes the white model by applying correction to at least one of the wall objects that make up the white model. an object image generation unit that generates a two-dimensional or three-dimensional object image representing at least one object selected from the white model object, the fixture object, and the home equipment object;
The building management system according to claim 1 , wherein the communication unit transmits data of the object images and list data of all or part of the BIM data to the mobile information terminal or other information processing device. The communication unit receives a request from the mobile information terminal or other information processing device, the request including designation of at least one object represented in the object image and BIM data corresponding to the designated object;
The building management system according to claim 3 , wherein in response to the request, the BIM data allocation unit changes or newly allocates the BIM data of the specified object to the BIM data included in the request. The system further includes an estimate generating unit that generates an estimate of repair and renovation costs for the building by referring to all or part of the BIM data,
The building management system according to claim 1 , wherein the communication unit transmits the estimate data to the mobile information terminal or another information processing device. An inference unit that receives at least one input object selected from the white model object, the fixture object, and the housing equipment object, and outputs, as a result of inference, one or more BIM data candidates according to the input object,
The building management system of claim 1, wherein the inference unit performs inference using a supervised trained neural network trained using the input object and correct answer information corresponding to the input object, or an unsupervised trained neural network trained to appropriately classify and output BIM data corresponding to the input object. The building management system according to claim 6, wherein the BIM data allocation unit automatically allocates to the target object the BIM data with the highest probability among the BIM data candidates output by the inference unit as a result of inference using the target object as an input. Further, a communication unit is provided that transmits BIM data candidates output by the inference unit as a result of inference using the target object as input to the mobile information terminal or other information processing device used by the user,
The building management system according to claim 6, wherein the BIM data allocation unit allocates, to the target object, BIM data corresponding to a selection operation by the user from among the BIM data candidates transmitted by the communication unit. a grade registration unit that registers a grade for at least one registered object selected from the white model object, the fixture object, and the home equipment object;
The building management system according to claim 1 , wherein the BIM data allocation unit allocates BIM data corresponding to the grade registered in the grade registration unit to the registered object. The building management system according to claim 1, further comprising a comment information storage unit that stores comment information related to at least one selected object selected from the white model object, the fixture object, and the home appliance object, by linking the comment information to the selected object. the comment information storage unit further stores position information indicating a position of a point of interest in the selected object;
an object image generation unit that generates a two-dimensional or three-dimensional object image representing one or more objects selected from the objects of the white model, the fixture objects, and the home equipment objects, the selected objects including at least the selected object;
a focus point image generating unit that generates a focus point image indicating the focus point of the selected object so as to be displayed superimposed on the object image,
The building management system according to claim 10 , wherein the communication unit transmits data of the object image on which the point-of-interest image is superimposed and the comment information to the mobile information terminal or another information processing device. the comment information storage unit further stores image data or electronic data related to the comment information;
The building management system according to claim 11 , wherein the communication unit transmits the image data or the electronic data of the object in addition to the image data and comment information of the object to the mobile information terminal or another information processing device. the building information includes captured image data of the interior of the room at a position corresponding to the point of interest,
the comment information storage unit further stores the captured image data in association with the point of interest;
The building management system according to claim 11 , wherein the communication unit transmits the captured image data in addition to data of the object image on which the point-of-interest image is superimposed and the comment information. the comment information storage unit stores imaging information indicating a shooting position and a shooting direction when the captured image data was acquired;
The building management system according to claim 13 , wherein the focus point image generating unit generates the focus point image and generates the image so that the shooting position and the shooting direction are displayed superimposed on the object image. The building management system according to claim 1, further comprising a drawing/perspective generation unit that uses the BIM data of the target object assigned by the BIM data assignment unit to generate a plan view, a three-dimensional view, or a perspective view of the building composed of the target object. a receiving function for receiving the building information from a mobile information terminal having a three-dimensional data generating function for capturing an image of the interior of a room of a building and generating three-dimensional data of the interior of the room by measuring the distance from the imaging point to the object being imaged; an object generating function for generating, from the three-dimensional data, at least one object selected from a wall object corresponding to each wall of the room, a fixture object corresponding to each of a single or multiple fixtures installed in the room, and a household equipment object corresponding to each of a single or multiple household equipment devices arranged in the room; and a transmitting function for transmitting building information including a plurality of the wall objects for at least one of the rooms;
a white model generating function that generates a white model of the building including at least one of the rooms by combining a plurality of the wall objects included in the building information received by the receiving function ;
A BIM data assignment function that assigns BIM data to at least one target object selected from the white model object, the building fixture object, and the housing equipment object;
A program to make the above happen on a computer.