JP7499475B2 - Inspection support device, inspection support system, inspection support network system, inspection support method, and program - Google Patents

Description

The present disclosure relates to an inspection support device, an inspection support system, an inspection support network system, an inspection support method, and a program.

The inspection system disclosed in Patent Document 1 has been proposed as a system to support construction inspection work in building construction and other work projects.

The inspection system in Patent Document 1 includes a photography point input unit for inputting photography points of the inspection object, and a photography unit capable of taking images of the inspection object as a subject, and stores inspection result data in a storage unit that associates the inspection object, inspection results, photography points, and inspection photos. With this inspection system, an inspector can input photography points, take inspection photos, and associate the inspection photos with photography points on-site.

Even if the inspection system of Patent Document 1 detects an abnormality in an inspection object from an image, it is time-consuming to match the abnormality to a drawing of the inspection object. In other words, the inspection system of Patent Document 1 requires time and effort to match the captured image of the inspection object (target space) to a drawing of the inspection object (space drawing).

JP 2011-8764 A

The object of the present disclosure is to provide an inspection support device, an inspection support system, an inspection support network system, an inspection support method, and a program that can easily map an image captured of at least a portion of a target space that is the subject of inspection into a spatial drawing of the target space.

An inspection support device according to an aspect of the present disclosure records a state of a target space that is a target of inspection. The inspection support device includes a drawing acquisition unit, an image acquisition unit, an area setting unit, a data storage unit, and a small area determination unit . The drawing acquisition unit acquires data of a space drawing that is a drawing of the target space. The image acquisition unit acquires data of a captured image obtained by capturing an actual shooting area that is at least a part of the target space. The area setting unit sets an area in the space drawing that corresponds to the actual shooting area as a drawing shooting area. The data storage unit stores each data of the space drawing, the captured image, and the drawing shooting area in association with each other. The small area determination unit determines whether or not a small area that satisfies a predetermined determination condition exists in the actual shooting area, and sets the determination result as small area information. The data storage unit stores the small area information.

An inspection support system according to one aspect of the present disclosure comprises the above-mentioned inspection support device and a display unit that displays the spatial drawing and the captured image.

A test support network system according to one aspect of the present disclosure includes the test support device described above and a server device. The server device includes a server-side data storage unit that stores the data received from the test support device, and a data calculation unit that performs calculation processing using the data.

An inspection support method according to an aspect of the present disclosure is executed by an inspection support device that records a state of a target space that is an object of inspection. The inspection support method includes a drawing acquisition step, an image acquisition step, an area setting step, a data storage step, and a small area determination step . The drawing acquisition step acquires data of a space drawing that is a drawing of the target space by the inspection support device . The image acquisition step acquires data of a captured image of an actual shooting area that is at least a part of the target space by the inspection support device . The area setting step sets an area corresponding to the actual shooting area in the space drawing as a drawing shooting area by the inspection support device . The data storage step stores each data of the space drawing, the captured image, and the drawing shooting area in association with each other by the inspection support device. The small area determination step determines whether or not a small area that satisfies a predetermined determination condition exists in the actual shooting area by the inspection support device, and sets the determination result as small area information. The data storage step stores the small area information.

A program relating to one aspect of the present disclosure causes a computer system to execute the above-mentioned inspection support method.

An inspection support device according to one aspect of the present disclosure records the state of a target space that is the subject of an inspection. The inspection support device includes a drawing acquisition unit, an image acquisition unit, an area setting unit, a small area determination unit, and a data storage unit. The drawing acquisition unit acquires data of a space drawing that is a drawing of the target space. The image acquisition unit acquires data of a captured image that captures an actual shooting area that is at least a part of the target space. The area setting unit sets an area in the space drawing that corresponds to the actual shooting area as a drawing shooting area. The small area determination unit inputs the captured image data into a learning model that is created in advance by machine learning to extract an area in which a small area that satisfies a predetermined judgment condition is captured from the captured image, thereby determining whether the small area exists in the actual shooting area, and sets the judgment result as small area information. The data storage unit stores each data of the space drawing, the captured image, the drawing shooting area, and the small area information in association with each other.

FIG. 1 is a block diagram showing an examination support system including an examination support device according to an embodiment. FIG. 2 is a plan view showing a space diagram in the above-mentioned inspection support device. FIG. 3 is a perspective view for explaining a user's examination using the above-mentioned examination support device. FIG. 4 is another perspective view for explaining a user's examination using the above-mentioned examination support device. FIG. 5 is a diagram showing an image captured by the above-mentioned inspection support device. FIG. 6 is a diagram showing a display screen on which an image captured by the above-mentioned inspection support device is displayed. FIG. 7 is a diagram showing a display screen on which a marker image is displayed by the above-mentioned examination support device. FIG. 8 is a diagram showing a display screen on which an area setting image is displayed by the above-mentioned inspection support device. FIG. 9 is a diagram showing a display screen on which an image detected within a drawing by the inspection support device is displayed. FIG. 10 is a diagram showing a display screen on which a related information setting image is displayed by the above-mentioned examination support device. FIG. 11 is a diagram for explaining a process of changing a marker image by the above-mentioned examination support device. FIG. 12 is a diagram for explaining a process of moving an in-image detection area in a marker image by the above-mentioned inspection support device. FIG. 13 is a diagram for explaining a process of moving an in-image detection area in a marker image by the above-mentioned inspection support device. FIG. 14 is a diagram for explaining a process of deleting an in-image detection region in a marker image by the above-mentioned inspection support device. FIG. 15 is a diagram for explaining a process of deleting an in-image detection region in a marker image by the above-mentioned inspection support device. FIG. 16 is a diagram for explaining a process of changing a spatial drawing by the inspection support device. FIG. 17 is a diagram for explaining another process of changing a spatial drawing by the inspection support device. FIG. 18 is a block diagram showing a configuration of an inspection support network system including the inspection support device according to the present invention. FIG. 19 is a diagram showing form data by the above-mentioned inspection support network system. FIG. 20 is a flowchart showing an inspection support method by the inspection support device.

The following embodiments generally relate to an inspection support device, an inspection support system, an inspection support network system, an inspection support method, and a program. More specifically, the following embodiments relate to an inspection support device, an inspection support system, an inspection support network system, an inspection support method, and a program that record the state of a target space that is the subject of inspection (inspection target). The target space is an interior or exterior space of a building (e.g., an apartment building, a detached house, an office building, a hospital, a factory, a commercial facility, etc.) that is under construction or has been constructed. However, the building is not limited to a specific building. In addition, the embodiment described below is merely an example of an embodiment of the present disclosure. The present disclosure is not limited to the following embodiments, and various modifications are possible depending on the design, etc., as long as the effects of the present disclosure can be achieved.

(1) Overview of the Inspection Support System FIG. 1 shows the configuration of an inspection support system 1 equipped with an inspection support device 2. The inspection support system 1 and the inspection support device 2 record the state of a target space TS1 that is the target of inspection. The target space TS1 is an internal or external space of a building under construction or after construction, and an inspector inspects the target space TS1 for the presence or absence of defective parts (abnormal parts). Defective parts are parts that are considered to be construction defects, missing parts, defects, or defects in the target space TS1, such as scratches, stains, and cracks on the surfaces of walls, floors, and ceilings, installation errors of components, and equipment defects. Note that the building may be an apartment building, a detached house, an office building, a hospital, a factory, a commercial facility, or the like, but is not limited to a specific building.

The inspection support system 1 is composed of a tablet terminal 10 having a computer system, and includes an inspection support device 2 and a display unit 4a. It is preferable that the inspection support system 1 further includes an imaging unit 3. A user such as an inspector can use the inspection support system 1 in the target space TS1 by carrying the tablet terminal 10.

In the computer system of the tablet terminal 10, a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) reads and executes a program of the inspection support method stored in the memory, thereby realizing some or all of the functions of the inspection support device 2. The computer system has a processor that operates according to a program as its main hardware configuration. The type of processor is not important as long as it can realize the function by executing the program. The processor is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or an LSI (large scale integration). Here, it is called an IC or an LSI, but the name changes depending on the degree of integration, and it may be called a system LSI, VLSI (very large scale integration), or ULSI (ultra large scale integration). A field programmable gate array (FPGA) that is programmed after the manufacture of the LSI, or a reconfigurable logic device that can reconfigure the connection relationship inside the LSI or set up the circuit partition inside the LSI, can also be used for the same purpose. Multiple electronic circuits may be integrated into one chip or provided on multiple chips. The multiple chips may be integrated into one device, or may be provided in multiple devices.

The inspection support device 2 includes a drawing acquisition unit 2d, an image acquisition unit 2e, an area setting unit 2f, and a data storage unit 2g. The drawing acquisition unit 2d acquires data of a spatial drawing D1 (see FIG. 2), which is a drawing of the target space TS1. The image acquisition unit 2e acquires data of a captured image G1 (see FIG. 5), which is captured with at least a portion of the target space TS1 as the actual shooting area R1 (see FIG. 4). The area setting unit 2f sets the area in the spatial drawing D1 corresponding to the actual shooting area R1 as the drawing shooting area V1 (see FIG. 8). The data storage unit 2g associates and stores the data of the spatial drawing D1, the captured image G1, and the drawing shooting area V1.

The inspection support system 1 further includes an imaging unit 3. The imaging unit 3 functions as a camera that captures color or monochrome images (still images). The imaging unit 3 outputs data of the captured image G1 to the inspection support device 2.

The test support system 1 further includes a touch panel display 4. The touch panel display 4 functions as a display unit 4a and an operation unit 4b. The display unit 4a is a display using an LCD (Liquid Crystal Display) or an OLED (Organic Light Emitting Diode), etc. The operation unit 4b has a function of detecting an operation by a user's finger or pen applied to the surface of the display unit 4a, and outputting the detection result as operation information to the test support device 2. Examples of user operations detected by the operation unit 4b include tapping, double tapping, long tapping, dragging, swiping, flicking, pinching in, and pinching out.

In the above-mentioned inspection support system 1 and inspection support device 2, the data storage unit 2g stores the spatial drawing D1, the captured image G1, and the drawing capture area V1 in association with each other. As a result, the inspection support system 1 and the inspection support device 2 can easily store the captured image G1, which captures at least a portion of the target space TS1 that is the subject of the inspection, in the spatial drawing D1 of the target space TS1 by referring to the data storage unit 2g. Furthermore, the inspection support system 1 and the inspection support device 2 can record the state of the target space TS1 using the data of the spatial drawing D1, the captured image G1, and the drawing capture area V1.

1, the inspection support device 2 includes a drawing acquisition unit 2d, an image acquisition unit 2e, a region setting unit 2f, and a data storage unit 2g. It is preferable that the inspection support device 2 further includes a communication unit 2a, a display control unit 2b, an operation information acquisition unit 2c, a small region determination unit 2h, a related information acquisition unit 2i, an image change unit 2j, a position conversion unit 2k, a drawing change unit 2m, and a number assignment unit 2n.

The communication unit 2a is connected to a network NT1 including the Internet, communicates with devices on the network NT1, and has a communication interface function for transmitting and receiving signals. A server device 51 and an external terminal 52 are connected to the network NT1, and the communication unit 2a can communicate with the server device 51 and the external terminal 52.

The display control unit 2b controls the screen displayed on the display unit 4a.

The operation information acquisition unit 2c acquires operation information corresponding to a user operation on the operation unit 4b from the operation unit 4b. Each part of the examination support device 2 and the imaging unit 3 executes an operation corresponding to the operation information acquired by the operation information acquisition unit 2c.

(2.1) Data Acquisition The drawing acquisition unit 2d acquires data of a space drawing D1 as shown in FIG. 2 from the server device 51 (or another server device) via the communication unit 2a as drawing data of the target space TS1. The drawing acquisition unit 2d stores the acquired data of the space drawing D1 in the data storage unit 2g. In this embodiment, the target space TS1 is a dwelling unit of an apartment building, and the space drawing D1 is a floor plan of the dwelling unit. In the space drawing D1, the walls, columns, partitions, doors, windows, etc. of the target space TS1 are drawn at a predetermined scale, and each symbol of the home equipment (sink, bathtub, toilet, washbasin, outlet, etc.) is arranged so as to correspond to the actual position. The space drawing D1 is a CAD (Computer Aided Design) drawing created by a design company or the like, and is uploaded to the server device 51 (or another server device) from a CAD terminal or the like. Note that FIG. 2 shows an example of a space drawing.

As shown in FIG. 3, user H1 inspects the target space TS1 and applies masking tape 9 to defective areas in the target space TS1. In FIG. 3, user H1 inspects the wall W1 of the target space TS1, and if there are defects such as scratches, stains, and cracks on the wall W1, he applies rectangular masking tape 9 to the defective areas. At this time, the color and application method of the masking tape 9 may be different for each type of defect (scratches, stains, cracks, etc.). Note that user H1 preferably folds the end of the masking tape 9 before applying it to the wall W1 to make it easier to peel off the masking tape 9 after inspection.

As shown in FIG. 4, the user H1 sticks masking tape 9 on each of the confirmed defective parts of the wall W1, and then photographs the wall W1 from the front using the imaging unit 3. If the imaging range in the target space TS1 by the imaging unit 3 is the actual imaging area R1, the actual imaging area R1 includes the wall W1, and masking tape 9 is attached to each of six places on the surface of the wall W1. Hereinafter, when distinguishing between the masking tapes 9 at each of the six places, each of the masking tapes 9 at each of the six places will be referred to as masking tape 91 to 96. The imaging unit 3 generates data of a photographed image G1 (see FIG. 5) obtained by photographing the wall W1 in the actual imaging area R1 from the front, and stores the data of the photographed image G1 in the data storage unit 2g in association with the data of the space drawing D1. In the photographed image G1, as shown in FIG. 5, areas Q91 to Q96 in which masking tapes 91 to 96 are respectively photographed exist within the area Q (W1) in which the wall W1 is photographed. When capturing images using the imaging unit 3, it is preferable to install a lighting fixture on the wall W1 to irradiate the wall with light for capturing images.

The data storage unit 2g includes a storage device that stores data such as the spatial drawing D1 and the captured image G1. It is preferable that the data storage unit 2g has a rewritable non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read Only Memory) or a flash memory as the storage device.

The display control unit 2b can display each data including the spatial drawing D1 and the captured image G1 stored in the data storage unit 2g on the display unit 4a individually or together with two or more data.

(2.2) Small Area Determination The user H1 causes the display unit 4a to display the photographed screen B1 (see FIG. 6) including the photographed image G1, and causes the small area determination unit 2h to execute a small area determination process. As the small area determination process, the small area determination unit 2h determines whether or not a small area that satisfies a predetermined determination condition exists in the actual photographed area R1, and sets the determination result as small area information. The small area in this embodiment is the above-mentioned masking tape 9. That is, as the small area determination process, the small area determination unit 2h determines whether or not the masking tape 9 exists in the actual photographed area R1. The data storage unit 2g stores the small area information, which is the determination result of the small area determination unit 2h, in association with the photographed image G1.

Specifically, it is preferable that the small area determination unit 2h includes in the determination condition that an area having a predetermined characteristic exists in the photographed image G1. In this case, the small area determination unit 2h performs a pattern recognition process on the photographed image G1 to determine whether or not the characteristics of the area in which the masking tape 9 is shown are included in the photographed image G1. In one example of the pattern recognition process, an algorithm for extracting the area in which the masking tape 9 is shown from the photographed image is created in advance as a learning model using machine learning such as deep learning as an input of the data of the photographed image. Then, the small area determination unit 2h performs the small area determination process using the learning model created by machine learning. As shown in FIG. 6, the small area determination unit 2h of this embodiment detects areas Q91 to Q96 in which the masking tapes 91 to 96 are shown in the photographed image G1, and each of the areas Q91 to Q96 is regarded as an in-image detection area. Hereinafter, each of the areas Q91 to Q96 extracted by the small area determination unit 2h is referred to as an in-image detection area Q91 to Q96. It should be noted that the small area determination unit 2h may extract an area in which the masking tape 9 appears from the captured image without using a learning model based on machine learning.

Then, when the small region determination unit 2h detects the in-image detection regions Q91-Q96 from the captured image G1, the display control unit 2b performs marker display processing. As shown in Fig. 7, the display control unit 2b executes image processing to place markers M91-M96 in the in-image detection regions Q91-Q96 of the captured image G1, respectively. The markers M91-M96 in Fig. 7 are rectangular frames (rectangular frames) that surround each of the in-image detection regions Q91-Q96. Note that the shape of the marker is not limited to a rectangular frame, and may be other shapes such as a ring or a triangle.

Furthermore, the numbering unit 2n assigns numbers to the small areas of the masking tape 9 (91-96). Specifically, the numbering unit 2n assigns numbers "1" to "6" to the in-image detection areas Q91-Q96 that correspond to the masking tape 9 (91-96), respectively. The numbering unit 2n stores information related to the assigned numbers as number information in the data storage unit 2g, corresponding to the in-image detection areas Q91-Q96 or markers M91-M96 of the captured image G1, respectively. The display control unit 2b adds a display of each number near each of the in-image detection areas Q91-Q96 in the captured image G1. Note that in FIG. 7, each number assigned by the numbering unit 2n is represented by a circled number.

As a result, a marker image G11 is created in which the numbers of the in-image detection areas Q91 to Q96 (the numbers of the masking tape small areas 91 to 96) and the markers M91 to M96 are added to the photographed image G1. The marker image G11 is an image in which the markers M91 to M96 and the numbers of the in-image detection areas Q91 to Q96 are arranged in the photographed image G1.

Furthermore, when the number assignment unit 2n receives correction information regarding the correction of numbers, it is preferable to correct the numbers assigned to the in-image detection areas Q91 to Q96 based on the correction information. The correction information is generated by the operation of the user H1 on the operation unit 4b, and the numbers can be corrected by the operation of the user H1. For example, the same number may be assigned to multiple in-image detection areas (multiple in-image detection areas are linked to one number). For example, if there is dirt in three places on the wall W1 and the three small areas are close to each other, the same number is assigned to the three in-image detection areas corresponding to the three small areas.

The small area determination unit 2h also detects the positions of each of the in-image detection areas Q91-Q96 in the captured image G1, and creates position information representing the positions of each of the in-image detection areas Q91-Q96 using the coordinates of the captured image G1. The small area determination unit 2h includes the position information of each of the in-image detection areas Q91-Q96 in the small area information. The coordinates of the captured image G1 are represented by mutually orthogonal X and Y coordinates. Therefore, the position information of the in-image detection area Q91 is represented by (X1, Y1). The position information of the in-image detection area Q92 is represented by (X2, Y2). The position information of the in-image detection area Q93 is represented by (X3, Y3). The position information of the in-image detection area Q94 is represented by (X4, Y4). The position information of the in-image detection area Q95 is represented by (X5, Y5). The position information of the in-image detection area Q96 is represented by (X6, Y6).

The display control unit 2b can generate a detection number notification image G2 and a position information notification image G3, and display the detection screen B2 shown in Figure 7 on the display unit 4a. The detection screen B2 is a screen in which the detection number notification image G2 and the position information notification image G3 are superimposed on the marker image G11. The detection number notification image G2 displays the number of in-image detection areas Q91 to Q96 (6 points in Figure 7) detected by the small area determination unit 2h. The position information notification image G3 displays the position information of each of the in-image detection areas Q91 to Q96.

The position information of each of the in-image detection areas Q91 to Q96 is not limited to position information defining one point in the captured image G1, but may be position information defining two points in the captured image G1 (for example, two diagonal vertices of the four vertices of a rectangle). Furthermore, the position information of each of the in-image detection areas Q91 to Q96 may be information defining the coordinates of the center of a circle and the radius of the circle, or information defining the coordinates of the center of a rectangle and the diagonal length of the rectangle.

Furthermore, if the color, application method, etc. of the masking tape 9 differs for each type of defect (scratches, stains, cracks, etc.), the small area determination unit 2h distinguishes the color, application method, etc. of the masking tape 9 by pattern recognition processing. In this case, the small area determination unit 2h can distinguish the type of defect in the intra-image detection areas Q91 to Q96 based on the color, application method, etc. of the masking tape 9. The display control unit 2b varies at least one of the shape and color of the marker for each type of defect.

(2.3) Setting of Drawing Shooting Area After the small area determination process, the area setting unit 2f executes a process of setting the drawing shooting area V1 in response to an operation by the user H1. The drawing shooting area V1 is an area in the spatial drawing D1 that corresponds to the actual shooting area R1.

When the setting process of the drawing shooting area V1 is started, the display control unit 2b displays the area setting screen B3 of FIG. 8 on the display unit 4a. The space drawing D1 is arranged in the left half of the area setting screen B3, and the marker image G11 is arranged in the right half of the area setting screen B3. The user H1 operates the operation unit 4b to set the drawing shooting area V1 (see FIG. 8) in the space drawing D1 of the area setting screen B3. The user H1 touches the space drawing D1 of the area setting screen B3 with a finger or a pen to set the range of the drawing shooting area V1 as the drawing area V11. The shape of the drawing area V11 may be any of a rectangle, a trapezoid, a line segment, a circle, and the like. The user H1 also touches the space drawing D1 of the area setting screen B3 with a finger or a pen to set the shooting direction of the drawing shooting area V1 as an arrow V12. In this embodiment, the drawing area V11 is an area corresponding to a part of the wall W1 in the spatial drawing D1, and the arrow V12 indicates the direction in which the wall W1 is viewed from the front.

The area setting unit 2f sets the data of the drawing area V11 and the arrow V12 as data of the drawing capture area V1, and stores the data of the drawing capture area V1 in the data storage unit 2g in association with each data of the spatial drawing D1 and the captured image G1. At this time, the data storage unit 2g stores data of the range and capture direction of the drawing capture area V1 in the spatial drawing D1 as data of the drawing capture area V1. Therefore, the inspection support device 2 can easily associate the captured image G1, which captures at least a portion of the target space TS1 that is the subject of inspection, with the spatial drawing D1 of the target space TS1.

In addition, if the drawing capture area V1 does not completely match the actual capture area R1, the area setting unit 2f may perform a trimming process to cut out a part of the marker image G11. That is, by performing a trimming process on the marker image G11, the drawing capture area V1 can be made to match the actual capture area R1 in a pseudo manner. The trimming process may be performed by the user H1 operating the operation unit 4b, or may be performed automatically by comparing the drawing capture area V1 and the marker image G11.

(2.4) Position Conversion The position conversion unit 2k determines positions in the spatial drawing D1 corresponding to the in-image detection areas Q91-Q96 as in-drawing detection positions P91-P96 (see FIG. 9) based on the drawing shooting area V1 and the in-image detection areas Q91-Q96.

When the position conversion unit 2k determines the in-drawing detection positions P91 to P96, it displays the in-drawing detection screen B4 of FIG. 9 on the display unit 4a. The spatial drawing D1 is arranged in the left half of the in-drawing detection screen B4, and the marker image G11 is arranged in the right half of the in-drawing detection screen B4. The position conversion unit 2k determines the in-drawing detection positions P91 to P96 by converting the position information (positions in the coordinate system of the photographed image G1) of the in-image detection areas Q91 to Q96 in the marker image G11 into the coordinate system of the spatial drawing D1. The numbers of the in-image detection areas Q91 to Q96 assigned by the number assignment unit 2n are also assigned to the in-drawing detection positions P91 to P96, respectively. In the spatial drawing D1 of the in-drawing detection screen B4, the numbers of the in-drawing detection positions P91 to P96 are associated with the positions of the in-drawing detection positions P91 to P96. In FIG. 9, each number assigned by the number assignment unit 2n is represented by a circled number.

(2.5) Related Information The related information acquisition unit 2i acquires information on the masking tape 9 (small area) as related information. The data storage unit 2g stores the related information in association with the small area information.

Specifically, the related information acquisition unit 2i sets related information for each number assigned by the number assignment unit 2n. Each number assigned by the number assignment unit 2n corresponds to an in-image detection area Q91-Q96 and an in-drawing detection position P91-P96, respectively. That is, each number assigned by the number assignment unit 2n corresponds to a masking tape 91-96 (small area) present in the actual shooting area R1, respectively. Thus, the related information acquisition unit 2i can set related information for each masking tape 9 (small area) by setting related information for each number assigned by the number assignment unit 2n.

Figure 10 shows the related information setting screen B5 for setting related information for the small area (masking tape 92) numbered "2". When the user H1 performs an operation to select the in-image detection area Q92 of the marker image G11 or the in-drawing detection position P92 of the spatial drawing D1, the related information acquisition unit 2i displays the related information setting screen B5 on the display unit 4a. The spatial drawing D1 is arranged in the left half of the related information setting screen B5. The marker image G11 is arranged in the upper part of the right half of the related information setting screen B5, and the related information input area G5 is arranged in the lower part of the right half of the related information setting screen B5. The related information input area G5 has the following items: "Room" for inputting the name of the room to which the actual shooting area R1 belongs, "Part" for inputting the part where the masking tape 92 (small area) is located, "Content" for inputting the content of the defect, "Person in charge" for inputting the name of the company or person in charge of repairing the defect, and "Response" for inputting a specific response to resolve the defect. The user H1 inputs information into the related information input field G5 by operating the operation unit 4b or by voice input using a microphone of the tablet terminal 10. The related information acquisition unit 2i regards the content input into the related information input field G5 as related information for the small area numbered "2" (masking tape 92) and stores it in the data storage unit 2g in association with the number "2."

(2.6) Image Modification The image modification unit 2j has a function of modifying the marker image displayed on the display unit 4a. The image modification unit 2j reflects the modification content in the small area information (including position information).

The display unit 4a in FIG. 11 displays a marker screen B6a including a marker image G11a. The marker image G11a is an image in which markers M91 and M92 are added to a captured image G1a having in-image detection areas Q91 and Q92 in an area Q (W1) in which the wall W1 is captured. The marker M91 surrounds the in-image detection area Q91, and the marker M92 surrounds the in-image detection area Q92. The image change unit 2j can add an in-image setting area Qa to the area Q (W1) by the user H1 operating the operation unit 4b (e.g., long tap or double tap). The in-image setting area Qa of the area Q (W1) corresponds to a place where the masking tape 9 is not attached on the wall W1 of the actual shooting area R1. That is, the image change unit 2j can add an in-image setting area Qa corresponding to the in-image detection area to a place in the marker image G11a where the small area determination unit 2h has not detected the in-image detection area. The numbering unit 2n assigns a new number to the set region Qa in the image and updates the number information. Therefore, even if the masking tape 9 is forgotten to be attached or the small region determination unit 2h makes a detection error, the inspection support device 2 can correct the error by an operation of the user H1.

At this time, it is preferable that the small area determination unit 2h includes in the determination conditions that the in-image setting area Qa has been set in the captured image G1a by the user H1. In this case, the small area determination unit 2h includes in the small area information the position information of the in-image setting area Qa in the captured image G1a. Therefore, the inspection support device 2 can also reflect the information of the added in-image setting area Qa in the small area information.

The display unit 4a in FIG. 12 displays a marker screen B6b including a marker image G11b. The marker image G11b is an image in which markers M91 and M92 are added to the captured image G1b. In the captured image G1b, there are areas Q91 and Q92 in which masking tapes 91 and 92 are respectively captured, and an area Qb in which an outlet is captured. However, the small area determination unit 2h does not detect the area Q91 in which the masking tape 91 is captured as an in-image detection area, and erroneously detects the area Qb in which the outlet is captured as an in-image detection area. In other words, the small area determination unit 2h erroneously determines that the area Qb in which the outlet is captured is an area in which the masking tape 9 is captured. As a result, in the marker image G11b in FIG. 12, the area Qb is surrounded by the marker M91 as an in-image detection area, and the area Q92 is surrounded by the marker M92 as an in-image detection area, but the area Q91 is not surrounded by a marker.

Therefore, as shown in FIG. 13, the image change unit 2j can move the marker M91 to a position specified by the user H1 by the user H1 operating the operation unit 4b (e.g., a drag operation OP1).

In Figure 13, marker M91 has moved from a position surrounding area Qb to a position surrounding area Q91. As a result, area Q91 becomes the in-image detection area, and area Qb is excluded from the in-image detection area. In marker image G11b in Figure 13, area Q91 is surrounded by marker M91 as the in-image detection area, and area Q92 is surrounded by marker M92 as the in-image detection area, but area Qb is not surrounded by a marker.

In this way, the image modification unit 2j can change the position of the intra-image detection area by moving the marker to a location in the marker image G11b where the small area determination unit 2h has not detected the intra-image detection area. That is, the image modification unit 2j changes the position of the intra-image detection area in the captured image G1b through the operation of the user H1. Therefore, even if a detection error occurs in the small area determination unit 2h, the inspection support device 2 can correct it through the operation of the user H1.

The number assignment unit 2n also transfers the number assigned to the area Qb to the area Q91 in the marker M91. That is, the area Q91 is assigned the number of the in-image detection area, and the area Qb is not assigned the number of the in-image detection area.

Next, the display unit 4a in FIG. 14 displays a marker screen B6c including a marker image G11c. Marker image G11c is an image in which markers M91 and M92 have been added to the captured image G1c. In the captured image G1c, there is an area Q92 in which masking tape 92 is captured, and an area Qb in which an electrical outlet is captured. However, the small area determination unit 2h erroneously detects the area Qb in which the electrical outlet is captured as an area detected within the image. In other words, the small area determination unit 2h erroneously determines that the area Qb in which the electrical outlet is captured is an area in which masking tape 9 is captured. As a result, in the marker image G11c in FIG. 14, area Qb is surrounded by marker M91 as an area detected within the image.

15, the image changing unit 2j can delete the marker M91 by the user H1 operating the operation unit 4b (e.g., long tapping or double tapping the marker M91). As a result, the area Qb is excluded from the detection area in the image.

In this way, the image modification unit 2j deletes the in-image detection area in the captured image G1b by the operation of the user H1. Therefore, even if a detection error occurs in the small area determination unit 2h, the inspection support device 2 can correct it by the operation of the user H1.

In addition, in addition to electrical outlets, other elements such as switches, wallpaper patterns, shadows, etc. may also be subject to erroneous judgment by the small area judgment unit 2h.

(2.7) Drawing modification The drawing modification unit 2m has a function of modifying the detected position in the drawing within the spatial drawing D1 displayed on the display unit 4a. The drawing modification unit 2m reflects the modification content in the small area information (including position information).

The display unit 4a in FIG. 16 displays the in-drawing detection screen B4a. In the spatial drawing D1 of the in-drawing detection screen B4a, the numbers of the in-drawing detection positions P91 to P96 are associated with the in-drawing detection positions P91 to P96. However, the position of the in-drawing detection position P91 is shifted from the actual application position of the masking tape 91. Therefore, the image change unit 2j can move the in-drawing detection position P91 to a position specified by the user H1 by operating the operation unit 4b by the user H1 (e.g., dragging). In FIG. 16, the in-drawing detection position P91 after the movement is represented as the in-drawing detection position Pa. Therefore, even if a conversion error occurs in the position conversion unit 2k, the inspection support device 2 can correct it by the operation of the user H1.

Also, the display unit 4a in FIG. 17 displays the in-drawing detection screen B4b. The image change unit 2j can add a new in-drawing detection position P97 to the spatial drawing D1 of the in-drawing detection screen B4b by the user H1 operating the operation unit 4b (e.g., long tap or double tap). The number assignment unit 2n assigns a new number to the in-drawing detection position P97 and updates the number information. Therefore, even if the masking tape 9 is forgotten to be applied or a conversion error occurs in the position conversion unit 2k, the inspection support device 2 can correct it by the operation of the user H1.

The small area determination unit 2h preferably includes in the determination conditions that the user H1 has set the in-drawing detection position P97 in the spatial drawing D1. In this case, the small area determination unit 2h includes in the small area information the position information of the in-drawing detection position P97 in the spatial drawing D1. Therefore, the inspection support device 2 can also reflect the information of the in-drawing detection position P97 in the small area information.

(3) Inspection Support Network System The inspection support network system 5 will now be described with reference to FIG.

The inspection support network system 5 includes an inspection support device 2 and a server device 51. It is preferable that the inspection support network system 5 further includes an external terminal 52. The inspection support device 2, the server device 51, and the external terminal 52 are configured to be able to communicate with each other via a network NT1 including the Internet.

The communication unit 2a of the inspection support device 2 transmits each of the data of the spatial drawing D1, the captured image G1, the small area information (including location information), the number information, the related information, and the drawing capture area V1, which are stored in the data storage unit 2g and correspond to each other, to the server device 51.

The server device 51 includes a server-side data storage unit 51a and a data calculation unit 51b. The server device 51 acquires each data from the inspection support device 2. The server device 51 stores each data acquired from the inspection support device 2 in the server-side data storage unit 51a. The data calculation unit 51b creates the inspection results of the target space TS1 as report data F1 in a predetermined format based on each data stored in the server-side data storage unit 51a. As shown in FIG. 19, the report data F1 describes the room and part in the target space TS1 where the defective part exists, the content of the defect, the person in charge of repairing the defect, and specific responses, in association with the number assigned by the number assignment unit 2n. The server device 51 transmits the report data F1 to the external terminal 52 together with the data of the in-drawing detection screen B4.

The external terminal 52 is a personal computer, tablet terminal, smartphone, or the like managed by the construction company, design company, or the like contracted to construct the target space TS1. The external terminal 52 notifies the person in charge at the construction company, design company, or the like of the inspection results of the target space TS1 by displaying on its screen the form data F1 and the drawing detection screen B4, etc., received from the server device 51. The person in charge repairs the defective parts under his/her responsibility based on the notified form data F1 and the drawing detection screen B4, etc.

(4) Inspection Support Method The inspection support method by the inspection support device 2 described above can be summarized as shown in the flowchart of FIG.

The inspection support method includes a drawing acquisition step S1, an image acquisition step S2, an area setting step S3, and a data retention step S4. The drawing acquisition step S1 acquires data of a spatial drawing D1, which is a drawing of the target space TS1. The image acquisition step S2 acquires data of a captured image G1 obtained by capturing an actual shooting area R1, which is at least a part of the target space TS1. The area setting step S3 sets an area in the spatial drawing D1 that corresponds to the actual shooting area R1 as the drawing shooting area V1. The data retention step S4 associates and retains the data of the spatial drawing D1, the captured image G1, and the drawing shooting area V1.

(5) Modification The above-mentioned marker image G11 (see FIG. 7) is composed of one image data. However, the display control unit 2b may create a sub-image in which the markers M91 to M96 are arranged at positions corresponding to the in-image detection areas Q91 to Q96, respectively, and display the sub-image on the display unit 4a by superimposing the sub-image on the photographed image G1. In this case, the marker image is composed of two image data: the data of the photographed image G1 and the data of the sub-image.

The small area may be something other than the masking tape 9. The small area determination unit 2h may recognize, for example, an area where the masking tape 9 is not applied, such as a scratch, stain, or crack, as a small area.

In addition, the configuration of the examination assistance system 1 may be a configuration in which the examination assistance device 2, imaging unit 3, display unit 4a, and operation unit 4b are distributed across two or more devices, like a personal computer.

In addition, at least a part of the testing support device 2 may be realized using cloud computing technology.

(6) Summary The inspection support device (2) of the first aspect according to the embodiment described above is an inspection support device that records the state of the target space (TS1) that is the target of the inspection. The inspection support device (2) includes a drawing acquisition unit (2d), an image acquisition unit (2e), an area setting unit (2f), and a data storage unit (2g). The drawing acquisition unit (2d) acquires data of a space drawing (D1) that is a drawing of the target space (TS1). The image acquisition unit (2e) acquires data of captured images (G1, G1a, G1b, G1c) that capture an actual shooting area (R1) that is at least a part of the target space (TS1). The area setting unit (2f) sets an area corresponding to the actual shooting area (R1) in the space drawing (D1) as a drawing shooting area (V1). The data storage unit (2g) stores the spatial drawing (D1), the photographed images (G1, G1a, G1b, G1c), and the drawing photographed area (V1) in association with each other.

The above-mentioned inspection support device (2) can easily associate captured images (G1, G1a, G1b, G1c) of at least a portion of the target space (TS1) that is the subject of inspection with a spatial drawing (D1) of the target space (TS1).

In the second aspect of the inspection support device (2) according to the above-mentioned embodiment, in the first aspect, it is preferable that the area setting unit (2f) sets an area within the spatial drawing (D1) set by the user (H1) as the drawing capture area (V1).

The above-mentioned inspection support device (2) can easily associate the captured images (G1, G1a, G1b, G1c) with the spatial drawing (D1) of the target space (TS1) through operation by the user (H1).

In the inspection support device (2) of the third aspect of the above-mentioned embodiment, in the first or second aspect, it is preferable that the data holding unit (2g) holds data on the range and shooting direction of the drawing shooting area (V1) in the spatial drawing (D1) as data on the drawing shooting area (V1).

The above-mentioned inspection support device (2) can easily match the captured images (G1, G1a, G1b, G1c) to the spatial drawing (D1).

It is preferable that the examination support device (2) of the fourth aspect of the above-mentioned embodiment further includes a small area determination unit (2h) in any one of the first to third aspects. The small area determination unit (2h) determines whether or not a small area (9) that satisfies a predetermined determination condition exists in the actual shooting area (R1), and the determination result is small area information. The data storage unit (2g) stores the small area information.

The above-mentioned inspection support device (2) can determine whether or not there are any defective areas in the target space (TS1).

In the inspection support device (2) of the fifth aspect of the above-mentioned embodiment, in the fourth aspect, the small area determination unit (2h) is an algorithm that uses data of the captured images (G1, G1a, G1b, G1c) as input and extracts an area in which the small area (9) is captured from the captured images (G1, G1a, G1b, G1c), and it is preferable to determine whether or not a small area (9) exists in the actual captured area (R1) using a learning model created in advance by machine learning.

The above-mentioned inspection support device (2) can automatically detect defective areas in the target space (TS1).

In the sixth aspect of the inspection support device (2) according to the above-mentioned embodiment, in the fourth or fifth aspect, it is preferable that the small area determination unit (2h) includes in the determination conditions the presence of an area having predetermined characteristics in the captured image (G1, G1a, G1b, G1c).

The above-mentioned inspection support device (2) can automatically detect defective areas in the target space (TS1).

In the seventh aspect of the inspection support device (2) according to the above-mentioned embodiment, in any one of the fourth to sixth aspects, it is preferable that the small area determination unit (2h) includes in the determination conditions that an in-image set area (Qa) has been set in the captured image (G1a) by the user (H1).

The above-mentioned inspection support device (2) can correct the judgment results of defective areas in the target space (TS1) through operation by the user (H1).

In the eighth aspect of the inspection support device (2) according to the above-mentioned embodiment, in any one of the fourth to seventh aspects, it is preferable that the data storage unit (2g) stores small area information in association with the captured images (G1, G1a, G1b, G1c).

The above-mentioned inspection support device (2) can add information on defective areas to the captured images (G1, G1a, G1b, G1c).

In the ninth aspect of the test support device (2) according to the above-mentioned embodiment, in any one of the fourth to eighth aspects, it is preferable that the test support device (2) further includes a related information acquisition unit (2i) that acquires information about the small region (9) as related information. The data storage unit (2g) stores the related information in association with the small region information.

The above-mentioned inspection support device (2) can reflect information regarding defective areas in the inspection results.

In the inspection support device (2) of the 10th aspect of the above-mentioned embodiment, in any one of the 4th to 9th aspects, it is preferable that the small area determination unit (2h) determines an area in the captured image (G1, G1a, G1b, G1c) corresponding to the small area (9) as an intra-image detection area (Q91 to Q96), and includes position information of the intra-image detection area (Q91 to Q96) in the small area information.

The above-mentioned inspection support device (2) can reflect the location information of the defective area in the inspection results.

In the inspection support device (2) of the 11th aspect of the above-mentioned embodiment, in the 10th aspect, it is preferable that the position information of the detection area (Q91 to Q96) within the image is expressed by the coordinates of the captured image (G1, G1a, G1b, G1c).

The above-mentioned inspection support device (2) can reflect the location information of the defective area in the captured images (G1, G1a, G1b, G1c).

It is preferable that the inspection support device (2) of the 12th aspect of the above-mentioned embodiment further includes, in the 10th or 11th aspect, an image modification unit (2j) which, by operation of a user (H1), performs at least one of changing the position of the in-image detection area in the captured image (G1b) and deleting the in-image detection area in the captured image (G1c).

The above-mentioned inspection support device (2) can be corrected by the user's (H1) operation even if a detection error occurs in the small area determination unit (2h).

The inspection support device (2) of the thirteenth aspect according to the above-mentioned embodiment, in any one of the tenth to twelfth aspects, preferably further includes a display control unit (2b). The display control unit (2b) creates one marker image (G11, G11a, G11b, G11c) in which a marker (M91 to M96) is arranged in the intra-image detection area (Q91 to Q96) of the captured image (G1), and displays the marker image (G11, G11a, G11b, G11c) on the display unit (4a).

The above-mentioned inspection support device (2) can create marker images (G11, G11a, G11b, G11c) that indicate defective areas with markers (M91 to M96).

It is preferable that the inspection support device (2) of the 14th aspect of the above-mentioned embodiment further includes a display control unit (2b) in any one of the 10th to 12th aspects. The display control unit (2b) creates a sub-image in which markers (M91 to M96) are arranged at positions corresponding to the detection areas (Q91 to Q96) in the image, and displays the sub-image on the display unit (4a) by superimposing it on the captured image (G1, G1a, G1b, G1c).

The above-mentioned inspection support device (2) can create marker images (G11, G11a, G11b, G11c) that indicate defective areas with markers (M91 to M96).

It is preferable that the inspection support device (2) of the 15th aspect according to the above-mentioned embodiment further includes a position conversion unit (2k) in any one of the 10th to 14th aspects. The position conversion unit (2k) determines positions in the spatial drawing (D1) corresponding to the intra-image detection areas (Q91 to Q96) as intra-drawing detection positions (P91 to P96) based on the drawing capture area (V1) and the intra-image detection areas (Q91 to Q96).

The above-mentioned inspection support device (2) can reflect defective areas in the spatial drawing (D1).

In the 16th aspect of the inspection support device (2) according to the above-mentioned embodiment, in the 15th aspect, it is preferable that the inspection support device (2) further includes a drawing modification unit (2m) that modifies the detection positions (P91 to P96) in the spatial drawing (D1). The drawing modification unit (2m) moves any one of the detection positions (P91 to P96) in the drawing selected by the user (H1) to a position in the spatial drawing (D1) specified by the user (H1).

The above-mentioned inspection support device (2) can be corrected by the user (H1) through operation even if a conversion error occurs in the position conversion unit (2k).

It is preferable that the inspection support device (2) of the 17th aspect of the above-mentioned embodiment further includes a number assignment unit (2n) that assigns numbers to at least one of the detection areas in the image (Q91 to Q96) and the detection positions in the drawing (P91 to P96) in the 15th or 16th aspects.

The above-mentioned inspection support device (2) can easily distinguish between multiple defective areas.

In the 18th aspect of the inspection support device (2) according to the above-mentioned embodiment, in the 17th aspect, when the number assignment unit (2n) receives correction information regarding the correction of the number, it is preferable to correct the number to be assigned to the small area (9) based on the correction information.

The above-mentioned inspection support device (2) can flexibly associate defective areas with numbers.

In the inspection support device (2) of the 19th aspect of the above-mentioned embodiment, in any one of the 15th to 18th aspects, it is preferable that the small area determination unit (2h) includes in the determination conditions that the detection position (P91 to P96) within the drawing has been set by the user (H1).

The above-mentioned inspection support device (2) can correct the judgment results of defective areas in the target space (TS1) through operation by the user (H1).

It is preferable that the test support device (2) of the 20th aspect of the above-mentioned embodiment further includes a communication unit (2a) capable of communicating with the server device (51) in any one of the first to nineteenth aspects.

The above-mentioned testing support device (2) can reduce the resources required for data processing.

The inspection support system (1) of the 21st aspect of the above-mentioned embodiment comprises an inspection support device (2) of any one of the 1st to 20th aspects, and a display unit (4a) that displays a spatial drawing (D1) and captured images (G1, G1a, G1b, G1c).

The above-mentioned inspection support system (1) can easily associate captured images (G1, G1a, G1b, G1c) of at least a portion of the target space (TS1) that is the subject of inspection with a spatial drawing (D1) of the target space (TS1).

The test support network system (5) of the 22nd aspect according to the above embodiment includes the test support device (2) of the 20th aspect and a server device (51). The server device (51) includes a server-side data storage unit (51a) that stores data received from the test support device (2) and a data calculation unit (51b) that performs calculation processing using the data.

The above-mentioned inspection support network system (5) can easily associate captured images (G1, G1a, G1b, G1c) of at least a portion of the target space (TS1) that is the subject of inspection with a spatial drawing (D1) of the target space (TS1).

In the 23rd aspect of the inspection support network system (5) relating to the above-mentioned embodiment, in the 22nd aspect, it is preferable that the data calculation unit (51b) creates the results of the inspection based on the data held by the server-side data holding unit (51a).

The above-mentioned testing support network system (5) can reduce the resources of the testing support device (2) required for data processing.

The inspection support method of the 24th aspect of the embodiment described above is an inspection support method that records the state of the target space (TS1) that is the subject of the inspection. The inspection support method includes a drawing acquisition step (S1), an image acquisition step (S2), an area setting step (S3), and a data retention step (S4). The drawing acquisition step (S1) acquires data of a space drawing (D1), which is a drawing of the target space (TS1). The image acquisition step (S2) acquires data of captured images (G1, G1a, G1b, G1c) captured of an actual shooting area (R1), which is at least a part of the target space (TS1). The area setting step (S3) sets an area in the space drawing (D1) that corresponds to the actual shooting area (R1) as the drawing shooting area (V1). The data retention step (S4) associates and retains each data of the space drawing (D1), the captured images (G1, G1a, G1b, G1c), and the drawing shooting area (V1).

The above-mentioned inspection support method can easily associate captured images (G1, G1a, G1b, G1c) of at least a portion of the target space (TS1) that is the subject of inspection with a spatial drawing (D1) of the target space (TS1).

The program of the 25th aspect of the above-mentioned embodiment causes a computer system to execute the inspection support method of the 24th aspect.

The above-mentioned program can easily associate captured images (G1, G1a, G1b, G1c) of at least a portion of the target space (TS1) that is the subject of inspection with a spatial drawing (D1) of the target space (TS1).

The inspection support device (2) of the 26th aspect according to the embodiment described above is an inspection support device that records the state of the target space (TS1) that is the subject of the inspection. The inspection support device (2) includes a drawing acquisition unit (2d), an image acquisition unit (2e), an area setting unit (2f), a small area determination unit (2h), and a data storage unit (2g). The drawing acquisition unit (2d) acquires data of a space drawing (D1) that is a drawing of the target space (TS1). The image acquisition unit (2e) acquires data of captured images (G1, G1a, G1b, G1c) that capture an actual shooting area (R1) that is at least a part of the target space (TS1). The area setting unit (2f) sets an area in the space drawing (D1) that corresponds to the actual shooting area (R1) as the drawing shooting area (V1). The small area determination unit (2h) inputs the data of the photographed images (G1, G1a, G1b, G1c) into a learning model created in advance by machine learning to extract an area in which a small area (9) that satisfies a predetermined determination condition is captured from the photographed images (G1, G1a, G1b, G1c), thereby determining whether or not a small area (9) exists in the actual photographed area (R1), and sets the determination result as small area information. The data storage unit (2g) stores the spatial drawing (D1), the photographed images (G1, G1a, G1b, G1c), the drawing photographed area (V1), and the data of the small area information in association with each other.

The above-mentioned inspection support device (2) can easily associate captured images (G1, G1a, G1b, G1c) of at least a portion of the target space (TS1) that is the subject of inspection with a spatial drawing (D1) of the target space (TS1).

REFERENCE SIGNS LIST 1 Examination support system 2 Examination support device 2a Communication unit 2b Display control unit 2d Drawing acquisition unit 2e Image acquisition unit 2f Area setting unit 2g Data storage unit 2h Small area determination unit 2i Related information acquisition unit 2j Image change unit 2k Position conversion unit 2m Drawing change unit 2n Number assignment unit 4a Display unit 5 Examination support network system 51 Server device 51a Server-side data storage unit 51b Data calculation unit 9 Masking tape (small area)
TS1 Target space D1 Space drawing R1 Actual shooting area G1, G1a, G1b, G1c Shooting image G11, G11a, G11b, G11c Marker image V1 Drawing shooting area H1 User Qa Set area in image Q91-Q96 Detection area in image M91-M96 Marker P91-P96 Detection position in drawing S1 Drawing acquisition step S2 Image acquisition step S3 Area setting step S4 Data storage step

Claims (25)

An inspection support device that records a state of a target space that is an object of inspection,
A drawing acquisition unit that acquires data of a space drawing, which is a drawing of the target space;
an image acquisition unit that acquires data of a captured image obtained by capturing an actual capture area that is at least a part of the target space;
an area setting unit that sets an area corresponding to the actual photography area in the spatial drawing as a drawing photography area;
a data storage unit that stores data of the spatial drawing, the photographed image, and the drawing photographing area in association with each other;
a small area determination unit that determines whether or not a small area that satisfies a predetermined determination condition exists in the actual imaging area, and sets the determination result as small area information ;
The data storage unit stores the small region information.
Inspection support equipment. The inspection support device according to claim 1 , wherein the area setting unit sets an area in the spatial drawing set by a user as the drawing photographing area. The inspection support device according to claim 1 or 2, wherein the data storage unit stores data on a range and an imaging direction of the drawing imaging area on the spatial drawing as the data on the drawing imaging area. The small area determination unit is an algorithm that extracts an area in which the small area is captured from the captured image using data of the captured image as an input, and determines whether the small area is present in the actual captured area using a learning model that is created in advance by machine learning.
4. The inspection support device according to claim 1. The small region determination unit includes in the determination conditions that a region having a predetermined characteristic exists in the captured image.
5. The inspection support device according to claim 1. The small area determination unit includes in the determination conditions that a set area within the image has been set in the captured image by a user.
6. The inspection support device according to claim 1. The data storage unit stores the small area information in association with the captured image.
7. The inspection support device according to claim 1. a related information acquisition unit that acquires information about the small region as related information;
The data storage unit stores the related information in association with the small area information.
8. The inspection support device according to claim 1. The small region determination unit determines a region in the captured image corresponding to the small region as an in-image detection region, and includes position information of the in-image detection region in the small region information.
9. The inspection support device according to claim 1. The position information of the detection area in the image is expressed by the coordinates of the photographed image.
The inspection support device according to claim 9. The image change unit changes the position of the detection area in the captured image and/or deletes the detection area in the captured image in response to a user operation.
The inspection support device according to claim 9 or 10. a display control unit that creates a marker image by arranging a marker in the detection area in the captured image and displays the marker image on a display unit;
12. The inspection support device according to claim 9. a display control unit that creates a sub-image in which a marker is arranged at a position corresponding to the detection area in the image, and displays the sub-image on the display unit while superimposing the sub-image on the captured image.
12. The inspection support device according to claim 9. a position conversion unit that obtains a position in the spatial drawing that corresponds to the in-image detection area based on the drawing capture area and the in-image detection area as an in-drawing detection position.
14. The inspection support device according to claim 9. A drawing change unit that changes the detected position in the drawing of the space,
The drawing change unit moves any one of the detected positions in the drawing selected by the user to a position in the spatial drawing designated by the user.
The inspection support device according to claim 14. The image detection unit may further include a number assignment unit that assigns a number to at least one of the detection area in the image and the detection position in the drawing.
16. The inspection support device according to claim 14 or 15. When the numbering unit receives correction information regarding the correction of the number, the numbering unit corrects the number to be assigned to the small area based on the correction information.
The inspection support device according to claim 16. The small area determination unit includes in the determination conditions that the detection position within the drawing has been set by a user.
18. The inspection support device according to claim 14. Further comprising a communication unit capable of communicating with the server device.
19. The inspection support device according to claim 1. An inspection support device according to any one of claims 1 to 19,
a display unit for displaying the spatial drawing and the captured image;
Inspection support system. The inspection support device according to claim 19,
The server device,
The server device includes:
a server-side data storage unit that stores the data received from the examination support device;
a data calculation unit that performs calculation processing using the data;
Inspection support network system. The data calculation unit creates a result of the inspection based on the data stored in the server-side data storage unit.
22. The inspection support network system according to claim 21. 1. An inspection support method executed by an inspection support device that records a state of a target space that is an object of inspection, comprising:
a drawing acquisition step of acquiring data of a space drawing, which is a drawing of the target space, by the inspection support device;
an image acquiring step of acquiring data of an image captured by the inspection support device, the image captured being an actual imaging area that is at least a part of the target space;
an area setting step of setting an area corresponding to the actual photographing area in the spatial drawing as a drawing photographing area by the inspection support device;
a data storage step of storing, by the inspection support device, data on the spatial drawing, the captured image, and the drawing capture area in association with each other;
a small area determination step of determining, by the examination support device, whether or not a small area satisfying a predetermined determination condition exists in the actual imaging area, and setting the determination result as small area information;
The data storage step stores the small area information.
Inspection support methods. A computer system is caused to execute the inspection support method of claim 23.
Program. An inspection support device that records the state of a target space that is the subject of inspection.
A drawing acquisition unit that acquires data of a space drawing, which is a drawing of the target space;
an image acquisition unit that acquires data of a captured image obtained by capturing an actual capture area that is at least a part of the target space;
an area setting unit that sets an area corresponding to the actual photography area in the spatial drawing as a drawing photography area;
a small area determination unit that inputs data of the captured image into a learning model that is created in advance by machine learning in order to extract an area in which a small area that satisfies a predetermined determination condition is captured from the captured image, thereby determining whether or not the small area exists in the actual captured area, and sets the determination result as small area information;
a data storage unit that stores the spatial drawing, the photographed image, the drawing photographing area, and the small area information in association with each other.
Inspection support equipment.

Images