JP7740965B2 - Inspection system, inspection method, and program - Google Patents

Description

This disclosure relates to an inspection system, an inspection method, and a program.

In Patent Document 1, at a construction site for a reinforced concrete structure, the diameter, pitch, number, etc. of the arranged rebars are inspected to confirm that they are arranged correctly before they are embedded in the poured concrete.

Specifically, the rebar inspection device of Patent Document 1 comprises a stereoscopic imaging unit, an arrangement plane identification unit, and an inspection unit. The stereoscopic imaging unit acquires positional information of the rebars in three-dimensional space by capturing images of the rebars arranged on a plane in three-dimensional space from two different points. The arrangement plane identification unit identifies the plane on which the rebars are arranged based on the positional information of the rebars acquired by the stereoscopic imaging unit. The inspection unit inspects the rebars, targeting the rebars arranged on the plane identified by the arrangement plane identification unit.

Japanese Patent Application Laid-Open No. 2019-2737

When multiple components are arranged in three-dimensional space, other components may appear in the image (captured image) captured by the stereo imaging unit (imaging unit) in addition to the component being inspected. In such cases, it may be difficult to distinguish the area in the captured image where the component being inspected appears (inspection area) from the area where other components appear. As a result, it is difficult to accurately set the inspection area.

The purpose of this disclosure is to provide an inspection system, inspection method, and program that allows for more accurate setting of the inspection range within a captured image.

An inspection system according to one aspect of the present disclosure includes an image acquisition unit, a display control unit, an inspection range setting unit, and an inspection unit. The image acquisition unit acquires captured image data, which is an image of a three-dimensional inspection object and includes three-dimensional position information. The display control unit displays the captured image on a screen. The inspection range setting unit sets an inspection range indicating the area in which the inspection object is captured in the captured image through a human operation. The inspection unit inspects the inspection object by performing image recognition processing on the inspection range. The inspection range setting unit sets, as the inspection range, an imaging area in which an image of a rectangular object is captured, the four vertices of which are specified by the human operation in the captured image displayed on the screen. One of the four points corresponds to a vertex common to three mutually perpendicular faces of the outline of the rectangular object. The remaining three of the four points correspond to the ends of three sides extending from the vertex common to the three faces.
An inspection system according to one aspect of the present disclosure includes an image acquisition unit, a display control unit, an inspection range setting unit, and an inspection unit. The image acquisition unit acquires data of an image of a three-dimensional inspection object, the captured image including three-dimensional position information. The display control unit displays the captured image on a screen. The inspection range setting unit sets an inspection range indicating the range in which the inspection object is captured in the captured image through a human operation. The inspection unit inspects the inspection object by performing image recognition processing on the inspection range. The inspection range setting unit sets the inspection range using information on the dimensions of the inspection object.
An inspection system according to one aspect of the present disclosure includes an image acquisition unit, a display control unit, an inspection range setting unit, an inspection unit, and a marker generation unit. The image acquisition unit acquires captured image data, which is an image of a three-dimensional inspection object and includes three-dimensional position information. The display control unit displays the captured image on a screen. The inspection range setting unit sets an inspection range indicating the range in which the inspection object is captured in the captured image through a human operation. The inspection unit inspects the inspection object by performing image recognition processing on the inspection range. The marker generation unit generates a marker image indicating the inspection range in the captured image. The display control unit displays the marker image on the screen by superimposing it on the captured image. The marker generation unit generates a three-dimensional model of the inspection range based on the three-dimensional position information included in the inspection range, and sets a projected image of the three-dimensional model as the marker image.
An inspection system according to one aspect of the present disclosure includes an image acquisition unit, a display control unit, an inspection range setting unit, an inspection unit, and a marker generation unit. The image acquisition unit acquires captured image data, which is an image of a three-dimensional inspection target and includes three-dimensional position information. The display control unit displays the captured image on a screen. The inspection range setting unit sets an inspection range indicating the range in which the inspection target is captured in the captured image through a human operation. The inspection unit inspects the inspection target by performing image recognition processing on the inspection range. The marker generation unit generates a marker image indicating the inspection range in the captured image. The display control unit displays the marker image on the screen by superimposing it on the captured image. The marker image is a projection image of a three-dimensional model. The marker generation unit displays the marker image on the screen by converting three-dimensional coordinates of the three-dimensional model into two-dimensional coordinates based on coordinate information indicating the position of the inspection target in a three-dimensional space in which the inspection target exists.
An inspection system according to one aspect of the present disclosure includes an image acquisition unit, a display control unit, an inspection range setting unit, an inspection unit, a marker generation unit, and a marker processing unit. The image acquisition unit acquires captured image data, which is an image of a three-dimensional inspection object and includes three-dimensional position information. The display control unit displays the captured image on a screen. The inspection range setting unit sets an inspection range indicating the range in which the inspection object is captured in the captured image through a human operation. The inspection unit inspects the inspection object by performing image recognition processing on the inspection range. The marker generation unit generates a marker image indicating the inspection range in the captured image. The display control unit displays the marker image on the screen by superimposing it on the captured image. The marker processing unit performs at least one of processing, including movement, rotation, enlargement, and reduction, on the marker image displayed on the screen in response to the human operation. The inspection range setting unit modifies the inspection range in response to the at least one processing performed on the marker image.

An inspection method according to one aspect of the present disclosure includes an image acquisition step, a display step, an inspection range setting step, and an inspection step. The image acquisition step is an image of a three-dimensional inspection object, and data of the captured image including three-dimensional position information is acquired. The display step displays the captured image on a screen. The inspection range setting step sets an inspection range indicating the range in which the inspection object is captured in the captured image through a human operation. The inspection step inspects the inspection object by performing image recognition processing on the inspection range. The inspection range setting step sets, as the inspection range, an imaging area obtained by imaging a rectangular object whose four vertices are specified by the human operation in the captured image displayed on the screen. One of the four points corresponds to a vertex common to three mutually intersecting faces of the outline of the rectangular object. The remaining three points correspond to the ends of three sides extending from the vertex common to the three faces.
An inspection method according to one aspect of the present disclosure includes an image acquisition step, a display step, an inspection range setting step, and an inspection step. The image acquisition step is an image of a three-dimensional inspection object, and data of the captured image including three-dimensional position information is acquired. The display step displays the captured image on a screen. The inspection range setting step sets an inspection range indicating the range in which the inspection object is captured in the captured image by human operation. The inspection step inspects the inspection object by performing image recognition processing on the inspection range. The inspection range setting step sets the inspection range using information on the dimensions of the inspection object.
An inspection method according to one aspect of the present disclosure includes an image acquisition step, a display step, an inspection range setting step, an inspection step, and a marker image setting step. The image acquisition step is an image of a three-dimensional inspection object, and data of the captured image including three-dimensional position information is acquired. The display step displays the captured image on a screen. The inspection range setting step sets an inspection range indicating the range in which the inspection object is captured in the captured image through human operation. The inspection step inspects the inspection object by performing image recognition processing on the inspection range. The marker image setting step generates a marker image indicating the inspection range in the captured image. The marker image setting step displays the marker image on the screen while superimposing it on the captured image. The marker image setting step generates a three-dimensional model of the inspection range based on the three-dimensional position information included in the inspection range, and a projected image of the three-dimensional model is used as the marker image.
An inspection method according to one aspect of the present disclosure includes an image acquisition step, a display step, an inspection range setting step, an inspection step, and a marker image setting step. The image acquisition step is an image of a three-dimensionally shaped inspection target, and data of the captured image including three-dimensional position information is acquired. The display step displays the captured image on a screen. The inspection range setting step sets an inspection range indicating the range in which the inspection target is captured in the captured image through human operation. The inspection step inspects the inspection target by performing image recognition processing on the inspection range. The marker image setting step generates a marker image indicating the inspection range in the captured image. The marker image setting step superimposes the marker image on the captured image and displays it on the screen. The marker image is a projection image of a three-dimensional model. The marker image setting step displays the marker image on the screen by converting three-dimensional coordinates of the three-dimensional model into two-dimensional coordinates based on coordinate information indicating the position of the inspection target in a three-dimensional space in which the inspection target exists.
An inspection method according to one aspect of the present disclosure includes an image acquisition step, a display step, an inspection range setting step, an inspection step, a marker image setting step, and a marker processing step. The image acquisition step is an image of a three-dimensional inspection object, and data of the captured image including three-dimensional position information is acquired. The display step displays the captured image on a screen. The inspection range setting step sets an inspection range indicating the range in which the inspection object is captured in the captured image through a human operation. The inspection step inspects the inspection object by performing image recognition processing on the inspection range. The marker image setting step generates a marker image indicating the inspection range in the captured image. The marker image setting step displays the marker image on the screen, superimposed on the captured image. The marker processing step performs at least one of processing, including movement, rotation, enlargement, and reduction, on the marker image displayed on the screen in response to the human operation. The inspection range setting step modifies the inspection range in response to the at least one processing performed on the marker image.

A program according to one aspect of the present disclosure causes a computer system to execute the above-described inspection method.

As described above, the present disclosure has the advantage of enabling more accurate setting of the inspection range within a captured image.

FIG. 1 is a block diagram showing the configuration of an inspection system according to an embodiment. FIG. 2 is a perspective view showing an example of use of the inspection system. FIG. 3 is a plan view showing an imaging unit of a tablet terminal constituting the above inspection system. FIG. 4 is a diagram showing an image captured by the inspection system. FIG. 5 is a diagram showing an inspection object designated in an image captured by the inspection system. FIG. 6 is a diagram showing an inspection range set in an image captured by the inspection system. FIG. 7 is a perspective view showing a three-dimensional model of the above inspection system. FIG. 8 is a diagram showing a marker image set on an image captured by the above inspection system. FIG. 9 is a flowchart showing an inspection method of the above inspection system. FIG. 10 is a diagram showing an inspection target designated in an image captured by the inspection system of the second modified example. FIG. 11 is a diagram showing an inspection range set in an image captured by the inspection system. FIG. 12 is a perspective view showing a three-dimensional model of the above inspection system. FIG. 13 is a diagram showing a marker image set on an image captured by the inspection system. FIG. 14 is a diagram showing an inspection target designated in an image captured by the inspection system according to the third and fourth modified examples. FIG. 15 is a block diagram showing the configuration of an inspection system according to the fifth modified example. FIG. 16 is a block diagram showing the configuration of another inspection system according to the fifth modified example.

The following embodiments generally relate to an inspection system, an inspection method, and a program. More specifically, the following embodiments relate to an inspection system, an inspection method, and a program that inspect an object by performing image recognition processing on a captured image.

The inspection system, inspection method, and program according to the embodiments will be described in detail below with reference to the drawings. However, the figures described in the following embodiments are schematic diagrams, and the ratios of the sizes and thicknesses of the components do not necessarily reflect the actual dimensional ratios.

Furthermore, the embodiment described below is merely an example of an embodiment of the present disclosure. The present disclosure is not limited to the following embodiment, and various modifications are possible depending on the design, etc., as long as the effects of the present disclosure can be achieved.

(1) Overview In the construction of a building, various inspections are performed at each stage. For example, components such as reinforcement, wallpaper, wiring fixtures, wiring, or piping are inspected to determine the placement, shape, and size of the inspection target. In addition, the building itself is inspected to determine the structure, placement, and shape of the inspection target. In such inspections, image recognition processing is performed on captured images of the inspection target to evaluate inspection items such as the placement, shape, size, and structure of the inspection target. Note that the construction of a building may be any work related to the construction of a building, and includes, for example, construction work, electrical work, water supply and drainage work, and civil engineering work.

In recent years, it has become easier to generate captured images that include three-dimensional position information rather than two-dimensional captured images. Therefore, the inspection system of this embodiment shown in Figure 1 inspects the inspection object 9 using captured images that include three-dimensional position information.

The inspection system 1 comprises an image acquisition unit 1b, a display control unit 1c, an inspection range setting unit 1e, and an inspection unit 1h. The image acquisition unit 1b is an image of a three-dimensional inspection object 9, and acquires captured image data including three-dimensional position information. The display control unit 1c displays the captured image on a screen D1. The inspection range setting unit 1e sets, through human operation, an inspection range that indicates the range in which the inspection object 9 appears in the captured image. The inspection unit 1h inspects the inspection object 9 by performing image recognition processing on the inspection range.

Therefore, since the inspection system 1 sets the inspection range in the captured image through human operation, it is possible to prevent structures other than the object of inspection from being included in the inspection range, and the inspection range in the captured image can be set more accurately.

(2) Details The inspection system 1 of this embodiment will be described in detail below.

(2.1) Inspection Object In this embodiment, the inspection object 9 is the reinforcement of columns, beams, walls, slabs, foundations, etc. of a building. Fig. 2 shows reinforcement 91, 92. The reinforcement 91, 92 exist in a three-dimensional space W1 inside a building under construction. The reinforcement 91, 92 are reinforcement of beams. The outline of the reinforcement 91 and the outline of the reinforcement 92 are rectangular.

Each of the reinforcement bars 91, 92 is made up of multiple reinforcing bars 9a arranged in a lattice pattern, and is a component with a three-dimensional shape. The reinforcement inspection involves individually determining whether the arrangement and arrangement of the multiple reinforcing bars 9a, as well as the size of the reinforcing bars 9a, for each of the reinforcement bars 91, 92, meets predetermined specifications.

In this embodiment, it is assumed that the rectangular outline of the reinforcement is composed of eight sides extending vertically and horizontally in the three-dimensional space W1.

(2.2) Inspection System The inspection system 1 preferably includes a computer system. The computer system executes a program to realize part or all of the inspection system 1. The computer system's main hardware component is a processor that operates according to the program. The processor can be of any type, as long as it can execute the program to realize its functions. The processor is composed of one or more electronic circuits, including a semiconductor integrated circuit (IC) or large-scale integration (LSI). While ICs and LSIs are used here, the names may vary depending on the degree of integration, and may be called system LSIs, very large-scale integration (VLSIs), or ultra large-scale integration (ULSIs). Field-programmable gate arrays (FPGAs), which are programmed after the LSI is manufactured, or reconfigurable logic devices that can reconfigure the connections within the LSI or set up circuit partitions within the LSI, can also be used for the same purpose. Multiple electronic circuits may be integrated on a single chip or provided on multiple chips. Multiple chips may be integrated into a single device or provided on multiple devices. The program is recorded on a non-transitory recording medium such as a computer system-readable ROM, an optical disk, a hard disk drive, etc. The program may be stored in the non-transitory recording medium in advance, or may be supplied to the non-transitory recording medium via a wide area communication network including the Internet.

The computer system is not limited to a single computer device, but may be realized by multiple computer devices linked together. Furthermore, the computer system may be constructed as a cloud computing system.

As shown in FIG. 1, the inspection system 1 of this embodiment includes an imaging unit 1a, an image acquisition unit 1b, a display control unit 1c, an operation unit 1d, an inspection range setting unit 1e, a marker generation unit 1f, a marker processing unit 1g, an inspection unit 1h, and a communication unit 1i. Preferably, the inspection system 1 further includes a screen D1. As shown in FIGS. 1 and 2, the inspection system 1 of this embodiment is an inspection device configured with a tablet terminal T1 that can be carried by a worker H1. The worker H1 carries the tablet terminal T1 and moves within a three-dimensional space W1. The worker H1 then uses the tablet terminal T1 to capture an image of the reinforcement bar that is the inspection target 9, and the inspection system 1 inspects the reinforcement bar that is the inspection target 9. In this embodiment, the worker H1 corresponds to the person who specifies the inspection range within the captured image.

(2.2.1) Imaging Unit As shown in FIG. 3 , the imaging unit 1a is a three-lens stereo camera having three lenses 101. The imaging unit 1a generates captured image data including three-dimensional position information based on image data captured by each of the three lenses 101. The three-dimensional position information is, for example, the three-dimensional coordinates (e.g., world coordinates) of an object included in the imaging range of the imaging unit 1a. That is, the imaging unit 1a generates three-dimensional image data as captured image data.

The imaging unit 1a captures an image of a three-dimensional space W1 containing a three-dimensional inspection object 9, and generates data for the captured image. In this embodiment, of the reinforcement bars 91 and 92, the reinforcement bar 91 is the inspection object 9. Therefore, the worker H1 operates the tablet terminal T1 so that the reinforcement bar 91 appears in the captured image. However, depending on the arrangement of the reinforcement bars 91 and 92 and the position of the worker H1, not only the reinforcement bar 91 but also the reinforcement bar 92 may appear in the captured image.

Then, the imaging unit 1a outputs the captured image data as real-time video data. In other words, the imaging unit 1a generates live view image data captured by the movable imaging unit 1a as captured image data.

(2.2.2) Image Acquisition Unit The image acquisition unit 1b acquires live view image data (captured image data) from the imaging unit 1a.

In this embodiment, the imaging unit 1a and image acquisition unit 1b are provided in the same device, the tablet terminal T1, but the imaging unit 1a may also be provided separately from the tablet terminal T1.

(2.2.3) Display Control Unit The display control unit 1c displays the captured image acquired by the image acquisition unit 1b on the screen D1. The screen D1 is, for example, a liquid crystal display or an organic EL display. The screen D1 in this embodiment is a display device provided in the tablet terminal T1.

Then, the display control unit 1c receives the live view image data acquired by the image acquisition unit 1b and displays the live view image on the screen D1.

Worker H1 carrying tablet terminal T1 can check the captured image by looking at screen D1. In this embodiment, the captured image is a live view image, so worker H1 can monitor the captured image in real time.

Figure 4 shows captured image G1 displayed on screen D1. Captured image G1 shows not only reinforcement 91, which is the inspection target 9, but also reinforcement 92, which is not the inspection target 9. In captured image G1, the area showing reinforcement 91 is designated Q91, and the area showing reinforcement 92 is designated Q92. The data for captured image G1 includes three-dimensional position information for area Q91 and three-dimensional position information for area Q92.

(2.2.4) Operation Unit The operation unit 1d is a touch sensor placed on top of the screen D1, and the screen D1 and the operation unit 1d form a touch panel. The worker H1 can specify any position in the captured image G1 on the screen D1 by touching the operation unit 1d, which is a touch sensor, with a finger or a touch pen. The any position in the captured image G1 is, for example, any point, line, or area in the captured image G1.

For example, worker H1 specifies at least one point within area Q91 in which reinforcement 91 is shown in captured image G1, or at least one point near area Q91.

In this embodiment, the outline of the reinforcement 91 is rectangular and has six sides. Then, worker H1 captures an image of the reinforcement 91 from an oblique angle so that at least one of the six sides constituting the rectangular outline of the reinforcement 91 is captured in area Q91 (in Figure 4, the reinforcement 91 is captured from an oblique angle so that three sides are captured). Then, as shown in Figure 5, worker H1 specifies four points P1-P4 in area Q91 containing the reinforcement 91, each of which corresponds to one of the four vertices of the outline of the rectangular inspection range R1 (see Figure 6). Point P1 corresponds to a vertex common to three mutually intersecting faces of the rectangular outline. Points P2-P4 correspond to the ends of three sides extending from a vertex common to the three mutually intersecting faces of the rectangle. In other words, by specifying four points P1-P4 in the captured image G1, the operator H1 can specify within the captured image G1 an area that corresponds to three of the six rectangular faces that make up the outline of the inspection range R1 (see Figure 6), which are perpendicular to each other.

In Figure 5, by specifying four points P1-P4 in area Q91, which shows the entire reinforcement 91, the entire reinforcement 91 is shown in inspection range R1 (see Figure 6). However, it is possible that area Q91 does not show the entire reinforcement 91, but only shows part of it. Even in such cases, as described above, by specifying four points P1-P4 in area Q91 that correspond to the four vertices of the outline of the rectangular inspection range R1 (see Figure 6), the area showing part of the reinforcement 91 can be set as the inspection range.

The operation unit 1d may also be a mouse, keyboard, etc.

(2.2.5) Inspection range setting unit The inspection range setting unit 1e sets an inspection range R1 shown in Fig. 6 within the captured image G1 based on the position specified within the captured image G1 by the worker H1. The inspection range R1 includes an area Q91 in the captured image G1 in which the reinforcement bars 91 to be inspected are captured.

The inspection range setting unit 1e sets an inspection range R1 within the captured image G1 based on the four points P1-P4 set in the captured image G1.

Specifically, the inspection range setting unit 1e can recognize the vertical and horizontal directions in the captured image G1 based on the detection results of motion sensors, such as an acceleration sensor, gyro sensor, or tilt sensor, built into the tablet terminal T1. Furthermore, the inspection range setting unit 1e can recognize the position and orientation of the tablet terminal T1 using at least one (or both) of the detection results of the motion sensor and changes in the captured image G1 (image frame differences). Therefore, the inspection range setting unit 1e sets, as the inspection range R1 (see FIG. 6), an imaged area in the captured image G1 that captures an image of a rectangle whose vertices are four points P1-P4. In this embodiment, it is assumed that the rectangular outline of the reinforcement is composed of eight sides extending vertically and horizontally in the three-dimensional space W1. Based on this assumption, the inspection range setting unit 1e estimates the imaged area that captures an image of a rectangle whose vertices are four points P1-P4.

Therefore, the inspection system 1 can distinguish the area Q91 containing the reinforcement 91 from the area Q92 containing the reinforcement 92, and set the area Q91 as the inspection area R1 without including the area Q92 in the inspection area. In other words, the inspection system 1 can more accurately set the inspection area R1 within the captured image G1.

(2.2.6) Marker Generator The marker generator 1f generates a marker image M1 (see FIG. 8) that indicates the inspection range R1 in the captured image G1.

In this embodiment, the marker generation unit 1f generates a rectangular three-dimensional model K1 (see Figure 7) that includes three mutually orthogonal faces with four points P1-P4 as vertices. Given the symmetry of the rectangle, if three mutually orthogonal faces of the six faces that make up the outer surface of the rectangle are known, it is possible to estimate the remaining three faces that make up the outer surface of the rectangle. Therefore, the marker generation unit 1f generates a rectangular three-dimensional model K1 in which the four points P1-P4 correspond to the four vertices. In other words, the marker generation unit 1f generates the shape of the three-dimensional model K1 as a rectangle that includes three mutually orthogonal faces specified by the four points P1-P4 in the captured image G1.

The marker generation unit 1f then generates the marker image M1 shown in FIG. 8 by perspectively projecting the three-dimensional model K1 onto the captured image G1. This marker image M1 is generated in the area where the three-dimensional model K1 is perspectively projected into the captured image G1, and becomes a three-dimensional graphic image in which the three-dimensional model K1 is displayed within the captured image G1. As a result, the display control unit 1c displays the marker image M1 on the screen D1, superimposed on the captured image G1, as shown in FIG. 8. The marker image M1 is an image that has a color, pattern, and design so that it can be distinguished from the captured image G1. The range of the marker image M1 in the captured image G1 is the same as the inspection range R1. In other words, the range of the marker image M1 in the captured image G1 includes the area Q91 in which the reinforcement bars 91 are visible.

Therefore, worker H1 can confirm the inspection range R1 within the captured image G1 by looking at the marker image M1 within the captured image G1 displayed on screen D1. As a result, the inspection system 1 can more accurately set the inspection range R1 within the captured image G1.

(2.2.7) Marker Processing Unit After setting the inspection range R1 and the marker image M1 in the captured image G1 as described above, the inspection system 1 links the subsequent captured image G1 with the marker image M1. That is, the range of the marker image M1 can be considered as the inspection range R1, and the inspection range R1 changes in the same way as the marker image M1, following the changes in the marker image M1.

In this embodiment, the marker processing unit 1g performs at least one of the following processes on the marker image M1 displayed on the screen D1: movement, rotation, enlargement, and reduction.

In this embodiment, the marker processing unit 1g performs processing such as movement, rotation, enlargement, and reduction on the marker image M1 displayed on the screen D1 in response to the operation of the operator H1 on the operation unit 1d.

Specifically, worker H1 operates the operation unit 1d, which is a touch sensor overlaid on screen D1, with a finger or a touch pen. Worker H1 performs a slide operation on the marker image M1 displayed on screen D1 to move or rotate the marker image M1. Worker H1 performs a pinch-out operation on the marker image M1 displayed on screen D1 to enlarge the marker image M1. Worker H1 performs a pinch-in operation on the marker image M1 displayed on screen D1 to reduce the marker image M1.

The marker processing unit 1g reflects the movement, rotation, enlargement, and reduction processes performed on the marker image M1 on the three-dimensional model K1, and performs correction processes for the movement of the three-dimensional model K1 relative to the projection surface, as well as the rotation, enlargement, and reduction of the three-dimensional model K1.

The inspection range setting unit 1e performs correction processes on the inspection range R1 (movement, rotation, enlargement, and reduction) in accordance with the movement, rotation, enlargement, and reduction processes performed on the marker image M1, in the same way as the processes performed on the marker image M1. In other words, the inspection range setting unit 1e corrects the inspection range R1 in accordance with the processes performed on the marker image M1.

Therefore, the inspection system 1 can correct the inspection range R1 by correcting the display area of the marker image M1. As a result, the inspection system 1 can more accurately set the inspection range R1 within the captured image G1.

(2.2.8) Inspection Unit The inspection unit 1h inspects the reinforcement 91, which is the inspection target 9, by performing image recognition processing on the inspection range R1 of the captured image G1.

Specifically, the inspection unit 1h extracts feature quantities of the inspection range R1 by performing binarization, filtering, edge extraction, and other processes on the inspection range R1 of the captured image G1. The inspection unit 1h also acquires design information from the design information storage unit 2a of the server device 2 via the communication unit 1i. The design information includes information such as the arrangement and assembly of multiple reinforcing bars 9a and the size of the reinforcing bars 9a as design specifications for the reinforcement 91. The inspection unit 1h then compares the feature quantities of the inspection range R1 with the design information to determine whether the reinforcement 91 satisfies the design specifications. The inspection unit 1h regards the determination result as the inspection result of the reinforcement 91 and generates inspection notification data as image data for notifying the inspection result of the reinforcement 91.

The display control unit 1c displays the inspection notification data for the reinforcement 91 on the screen D1. Therefore, the worker H1 can confirm the inspection results for the reinforcement 91 by looking at the inspection notification data displayed on the screen D1.

(2.2.9) Communication Unit The communication unit 1i is connected to a network NT1 including the Internet, communicates with devices on the network NT1, and has a function of a communication interface for sending and receiving signals. A server device 2 is connected to the network NT1, and the communication unit 1i can communicate with the server device 2 via the network NT1.

The server device 2 is managed, for example, by a contractor contracted to design or construct a building, and is equipped with a design information storage unit 2a. The design information storage unit 2a stores various information related to the specifications, design, and construction of the building. The information stored in the design information storage unit 2a also includes design information for reinforcement 91, 92.

Therefore, the inspection system 1 can obtain design information for reinforcement 91, 92, etc. from the server device 2 via the communication unit 1i.

(2.2.10) Linking of Captured Image and Marker Image As described above, the inspection system 1 inspects the reinforcement 91 by setting the inspection range R1 and the marker image M1 in the captured image G1 of the reinforcement 91.

After setting the inspection range R1 and marker image M1 in the captured image G1, the inspection system 1 links the subsequent captured image G1 with the marker image M1 and inspection range R1. As worker H1 moves within the three-dimensional space W1 while capturing images of the reinforcement 91 with the tablet terminal T1, the captured image G1 displayed on the screen D1 changes, and the area Q91 in which the reinforcement 91 appears in the captured image G1 also changes. As the tablet terminal T1 moves within the three-dimensional space W1, the positions and shapes of the marker image M1 and inspection range R1 in the captured image G1 also change.

Specifically, the captured image G1 includes three-dimensional position information. Therefore, the marker generation unit 1f uses the three-dimensional position information within the inspection range R1 (i.e., region Q91) of the captured image G1 to acquire three-dimensional coordinate information (e.g., world coordinates) as coordinate information indicating the position of the reinforcement 91 (inspection target 9) in the three-dimensional space W1. The marker generation unit 1f can also detect the movement distance and direction of the tablet terminal T1 based on at least one (or both) of the detection results of a motion sensor, such as an acceleration sensor, gyro sensor, or tilt sensor, built into the tablet terminal T1 and changes in the captured image G1 (image frame differences). Therefore, the marker generation unit 1f corrects the three-dimensional coordinates indicating the position of the reinforcement 91 in the three-dimensional space W1 based on the movement distance and movement direction of the tablet terminal T1. In other words, the marker generation unit 1f can detect the position of the reinforcement 91 relative to the imaging unit 1a even if the tablet terminal T1 moves.

The marker generation unit 1f then adjusts the projection distance and projection direction of the three-dimensional model K1 relative to the captured image G1 based on the three-dimensional coordinates indicating the position of the reinforcement 91, thereby causing the position and shape of the marker image M1 in the captured image G1 to follow changes in the inspection range R1 in the captured image G1.

In addition, the inspection range setting unit 1e changes the inspection range R1 in the same way as the marker image M1.

Therefore, the inspection system 1 can link the marker image M1 to the captured image G1 even if the captured image G1 is a live view image. Furthermore, the inspection system 1 can link the inspection range R1 to the captured image G1 even if the captured image G1 is a live view image.

(2.2.11) Inspection Method The inspection method by the inspection system 1 described above can be summarized as shown in the flowchart of Fig. 9. For example, the inspection method is performed by a computer system included in the inspection system 1 executing a program.

The inspection method comprises an image acquisition step S1, a display step S2, an inspection range setting step S3, and an inspection step S5. It is preferable that the inspection method further comprises a marker image setting step S4.

In image acquisition step S1, the image acquisition unit 1b acquires data for the captured image G1. The captured image G1 is an image of the inspection object 9 having a three-dimensional shape, and includes three-dimensional position information.

In display step S2, the display control unit 1c displays the captured image G1 on the screen D1.

In the inspection range setting step S3, the inspection range setting unit 1e sets an inspection range R1 indicating the range in which the inspection object 9 appears in the captured image G1, in response to an operation by the operator H1.

In the marker image setting step S4, the marker generation unit 1f generates a marker image M1 indicating the inspection range R1 in the captured image G1, and displays the marker image M1 on the screen D1 by superimposing it on the captured image G1.

In inspection step S5, the inspection unit 1h inspects the inspection object 9 by performing image recognition processing on the inspection range R1.

The above-described inspection method allows for more accurate setting of the inspection range R1 within the captured image G1.

(3) First Modification The marker generation unit 1f may input the captured image G1 to a learning model constructed by machine learning using the captured image and the range of the marker image in the captured image as training data, thereby obtaining the captured image G1 with the marker image M1 superimposed thereon from the learning model.

Specifically, a learning model is constructed by learning using a large amount of training data. It is preferable that the learning model be a neural network constructed using machine learning such as deep learning. The learning model may also use other algorithms such as a support vector machine.

The machine learning system that constructs the learning model may be either a configuration provided by the inspection system 1 or a configuration provided external to the inspection system 1.

(4) Second Modification The inspection range setting unit 1e may set the inspection range using information on the dimensions of the inspection object 9.

The following explanation will be given using the captured image G11 in Figure 10.

The captured image G11 displayed on screen D1 includes areas Q93 and Q94. The reinforcement shown in area Q93 is the reinforcement for the column. The reinforcement shown in area Q94 is the reinforcement for the foundation. The outline of the reinforcement for the foundation is formed in the shape of a horizontally extending rectangular plate, while the outline of the reinforcement for the column is formed in the shape of a rectangular body that is long in the vertical direction. In this modified example, worker H1 takes an image of the reinforcement for the column from the front, with the reinforcement for the column shown in area Q93 as the inspection object 9.

By operating tablet terminal T1, worker H1 downloads (pre-stores on tablet terminal T1) design information for the reinforcement of the column corresponding to area Q93 (the reinforcement that is the inspection target 9) from the design information storage unit 2a of server device 2 via communication unit 1i to tablet terminal T1 in advance. The design information includes reinforcement dimension information as the design specifications for the reinforcement of the column corresponding to area Q93.

Then, as shown in Figure 10, worker H1 specifies three points P11-P13 in area Q93, thereby specifying one face of the rectangular outline of the column reinforcement in area Q93. Points P11-P13 define the front surface of the rectangular outline of the reinforcement. In other words, by specifying three points P11-P13 in the captured image G11, worker H1 can specify an area in the captured image G11 that corresponds to one of the six rectangular faces that make up the outline of the reinforcement.

The inspection range setting unit 1e can recognize the vertical and horizontal directions in the captured image G11 based on the detection results of motion sensors, such as an acceleration sensor, gyro sensor, or tilt sensor, built into the tablet terminal T1. The inspection range setting unit 1e can also recognize the position and orientation of the tablet terminal T1 using at least one (or both) of the detection results of the motion sensor and changes in the captured image G1 (differences between image frames). Therefore, the inspection range setting unit 1e sets the inspection range R11 (see Figure 11) by comparing the three points P11-P13 specified in the captured image G11 with the reinforcement design information captured in area Q93. This inspection range R11 corresponds to the captured area in which a rectangular body including one surface defined by the three points P11-P13 is captured.

Therefore, the inspection system 1 can distinguish area Q93, which shows the reinforcement of the inspection target, from area Q94, which shows reinforcement that is not the inspection target 9, and set area Q93 as the inspection range R11 without including area Q94 in the inspection range. In other words, the inspection system 1 can more accurately set the inspection range R11 within the captured image G11.

In addition, the inspection system 1 can simplify the operations of worker H1 by using information on the dimensions of the reinforcement bars 93.

The marker generation unit 1f then generates a marker image M11 (see Figure 13) that indicates the inspection range R11 in the captured image G11.

In this embodiment, the marker generation unit 1f uses dimensional information about the reinforcement, which is the inspection target 9, to generate a rectangular three-dimensional model K11 (see FIG. 12) that includes one surface specified in the captured image G11. That is, the marker generation unit 1f generates the rectangular three-dimensional model K11 by comparing the three points P11-P13 specified in the captured image G11 with the design information about the reinforcement, which is the inspection target 9. That is, the marker generation unit 1f sets the shape of the three-dimensional model K11 to the shape of a rectangle that includes one surface specified by the three points P11-P13 in the captured image G11.

The marker generation unit 1f then generates the marker image M11 shown in FIG. 13 by perspectively projecting the three-dimensional model K11 onto the captured image G11. This marker image M11 is generated in the area where the three-dimensional model K11 is perspectively projected into the captured image G11, and is a three-dimensional graphic image in which the three-dimensional model K11 is displayed within the captured image G11. As a result, the display control unit 1c displays the marker image M11 on the screen D1, superimposed on the captured image G11, as shown in FIG. 13. The marker image M11 is an image that has a color, pattern, and design so that it can be distinguished from the captured image G11. The range of the marker image M11 in the captured image G11 is the same as the inspection range R11. In other words, the range of the marker image M11 relative to the captured image G11 includes the area Q93 in which the reinforcement bar that is the inspection target 9 is displayed.

Therefore, worker H1 can confirm the inspection range R11 within the captured image G11 by looking at the marker image M11 within the captured image G11 displayed on screen D1. As a result, the inspection system 1 can more accurately set the inspection range R11 within the captured image G11.

In addition, worker H1 may specify multiple faces of the rectangular outline of the reinforcement in area Q93 of the captured image G11. In this case, the inspection range setting unit 1e sets the inspection range R11 (see FIG. 11) by comparing the multiple faces specified in the captured image G11 with the design information of the reinforcement (reinforcement shown in area Q93) that is the inspection target 9. This inspection range R11 corresponds to the imaging area in which the rectangular body including the multiple faces specified in the captured image G11 is imaged. In addition, the marker generation unit 1f sets the shape of the three-dimensional model K11 to the shape of the rectangular body including the multiple faces specified in the captured image G11.

(5) Third Modification As shown in Fig. 14, the worker H1 may specify one side of the rectangular outline of the reinforcement in the area Q93 by specifying one point P21 in the area Q93 of the captured image G11 displayed on the screen D1. The point P21 in this modification defines the upper front side of the rectangular outline of the reinforcement. In other words, by specifying one point P21 in the captured image G11, the worker H1 can specify one of the eight sides that make up the outline of the reinforcement in the captured image G11.

In this case, the inspection range setting unit 1e sets the inspection range R11 (see FIG. 11) by comparing one point P21 specified in the captured image G11 with the design information of the reinforcement (reinforcement shown in area Q93) that is the inspection target 9. This inspection range R11 corresponds to the captured area in which an image of a rectangle with one side defined by one point P21 is captured.

The marker generation unit 1f then generates a marker image M11 (see Figure 13) that indicates the inspection range R11 in the captured image G11. Here, the marker generation unit 1f sets the shape of the three-dimensional model K11 to the shape of a rectangle with one side specified by one point P21 in the captured image G11.

In addition, worker H1 may specify multiple sides of the rectangular outline of the reinforcement for area Q93 in the captured image G11. In this case, the inspection range setting unit 1e sets the inspection range R11 (see FIG. 11) by comparing the multiple sides specified in the captured image G11 with the design information of the reinforcement (reinforcement shown in area Q93) that is the inspection target 9. This inspection range R11 corresponds to the imaging area in which the rectangular body including the multiple sides specified in the captured image G11 is imaged. In addition, the marker generation unit 1f sets the shape of the three-dimensional model K11 to the shape of the rectangular body including the multiple sides specified in the captured image G11.

(6) Fourth Modification The worker H1 may specify one point on the rectangular outline of the reinforcement in the area Q93 by specifying one point P21 in the area Q93 of the captured image G11 displayed on the screen D1, as shown in Fig. 14. Point P31 in this modification defines one point included in the upper front side of the rectangular outline of the reinforcement.

In this case, the inspection range setting unit 1e sets the inspection range R11 (see FIG. 11) by comparing one point P21 specified in the captured image G11 with the design information of the reinforcement (reinforcement shown in area Q93) that is the inspection target 9. This inspection range R11 corresponds to the imaging area in which the rectangular body including one point P21 is imaged.

The marker generation unit 1f then generates a marker image M11 (see Figure 13) that indicates the inspection range R11 in the captured image G11. Here, the marker generation unit 1f sets the shape of the three-dimensional model K11 to the shape of a rectangle that includes one point P21 in the captured image G11.

In addition, worker H1 may specify multiple points on the rectangular outline of the reinforcement in area Q93 of the captured image G11. In this case, the inspection range setting unit 1e sets the inspection range R11 (see FIG. 11) by comparing the multiple points specified in the captured image G11 with the design information of the reinforcement (reinforcement shown in area Q93) that is the inspection target 9. This inspection range R11 corresponds to the captured area that captures the rectangular body that includes the multiple points specified in the captured image G11. In addition, the marker generation unit 1f sets the shape of the three-dimensional model K11 to the shape of the rectangular body that includes the multiple points specified in the captured image G11.

(7) Fifth Modification FIG. 15 shows an inspection system 1A as a modification of the inspection system 1. As shown in FIG.

In the inspection system 1A, the inspection unit 1h is provided in the server device 2. The tablet terminal T1 and the server device 2 are configured to be able to communicate with each other via the network NT1, and the inspection unit 1h inspects the inspection object 9 by performing image recognition processing on the inspection range of the captured image.

Figure 16 shows inspection system 1B, another variant of inspection system 1.

In inspection system 1B, in addition to inspection unit 1h, server device 2 is also provided with inspection range setting unit 1e, marker generation unit 1f, and marker processing unit 1g. Tablet terminal T1 and server device 2 are configured to be able to communicate with each other via network NT1, and inspection range setting unit 1e, marker generation unit 1f, and marker processing unit 1g also have the same functions as those described above.

The components constituting inspection system 1 are not limited to being integrated into one device as in inspection system 1, or being distributed across two devices as in inspection systems 1A and 1B, but may also be distributed across three or more devices.

(8) Sixth Modification The inspection system 1 does not need to include the screen D1. For example, the screen D1 may be included in an external terminal such as a tablet terminal, a smartphone, or a personal computer that is configured to be able to communicate with the inspection system 1.

The inspection object 9 may be a component other than reinforcement, such as wallpaper, wiring fixtures, wiring, or piping. In other words, the inspection object 9 is not limited to a specific component, and may be any component having a three-dimensional shape.

Furthermore, by operating the tablet terminal T1, the worker H1 may change the live view image to a still image at the desired timing and set the inspection range for the still image. In this case, while monitoring the live view image, the worker H1 may change the live view image to a still image when the inspection object 9 appears in the live view image, thereby displaying a still image containing the inspection object 9 on the screen D1. Note that both the live view image and the still image correspond to captured image data.

The imaging unit 1a is not limited to a stereo camera with three lenses, but may be a stereo camera with two, four, or more lenses. The imaging unit 1a may also be a monocular camera, in which case captured image data including three-dimensional position information can be generated from a series of captured images. The imaging unit 1a may also be configured using lasers, such as a three-dimensional laser scanner or LiDAR (Light Detection and Ranging).

The inspection object 9 is not limited to reinforcement 91 and 93, but may also be reinforcement 92 and 94. Furthermore, the inspection object 9 may be something other than reinforcement, such as wallpaper, wiring fixtures, wiring, or piping. Furthermore, the inspection object 9 may also be a building.

In addition, in the above-described embodiment, the inspection range setting unit 1e and the marker generation unit 1f set the inspection range and generate the marker image based on the premise that the rectangular outline of the reinforcement is composed of eight sides extending vertically and horizontally in the three-dimensional space W1. However, the premise used by the inspection system 1 can be set appropriately depending on the shape, structure, and arrangement of the inspection object 9, and is not limited to any specific premise.

(9) Summary The inspection system (1, 1A, 1B) of the first aspect according to the above-described embodiment includes an image acquisition unit (1b), a display control unit (1c), an inspection range setting unit (1e), and an inspection unit (1h). The image acquisition unit (1b) acquires data of captured images (G1, G11) that are images of a three-dimensionally shaped inspection object (9) and include three-dimensional position information. The display control unit (1c) displays the captured images (G1, G11) on a screen (D1). The inspection range setting unit (1e) sets inspection ranges (R1, R11) that indicate the range in which the inspection object (9) is captured in the captured images (G1, G11) through operation by a person (H1). The inspection unit (1h) inspects the inspection object (9) by performing image recognition processing on the inspection ranges (R1, R11).

The above-described inspection system (1, 1A, 1B) allows for more accurate setting of the inspection range (R1, R11) within the captured image (G1, G11).

In the inspection system (1, 1A, 1B) of the second aspect of the above-described embodiment, in the first aspect, the inspection range setting unit (1e) preferably sets the inspection range (R1, R11) based on a position specified by a person (H1) in the captured image (G1, G11) displayed on the screen (D1).

The above-described inspection system (1, 1A, 1B) allows for more accurate setting of the inspection range (R1, R11) within the captured image (G1, G11).

In the inspection system (1, 1A, 1B) of the third aspect of the above-described embodiment, in the second aspect, the inspection range setting unit (1e) preferably sets an inspection range (R1, R11) that includes at least one point (P1-P4, P11-P13, P21) specified in the captured image (G1, G11).

The above-described inspection system (1, 1A, 1B) allows for more accurate setting of the inspection range (R1, R11) within the captured image (G1, G11).

In the inspection system (1, 1A, 1B) of the fourth aspect of the above-described embodiment, in the second aspect, it is preferable that the inspection range setting unit (1e) sets an inspection range (R1, R11) that includes at least one range specified in the captured image (G1, G11).

The above-described inspection system (1, 1A, 1B) allows for more accurate setting of the inspection range (R1, R11) within the captured image (G1, G11).

In the inspection system (1, 1A, 1B) of the fifth aspect of the above-described embodiment, in any one of the first to fourth aspects, the inspection range setting unit (1e) preferably sets the inspection range (R11) using dimensional information of the inspection object (9).

The above-mentioned inspection system (1, 1A, 1B) can simplify the operation of the person (H1) by using dimensional information of the object to be inspected (9).

The inspection system (1, 1A, 1B) of the sixth aspect of the above-described embodiment, in any one of the first to fifth aspects, preferably further includes a marker generation unit (1f) that generates marker images (M1, M11) indicating the inspection range (R1, R11) in the captured image (G1, G11). The display control unit (1c) displays the marker images (M1, M11) on the screen (D1) by superimposing them on the captured image (G1, G11).

In the above-described inspection system (1, 1A, 1B), a person (H1) can confirm the inspection range (R1, R11) within the captured image (G1, G11) by looking at the marker images (M1, M11) within the captured image (G1, G11) displayed on the screen (D1). As a result, the inspection system (1, 1A, 1B) can more accurately set the inspection range (R1, R11) within the captured image (G1, G11).

In the inspection system (1, 1A, 1B) of the seventh aspect of the above-described embodiment, in the sixth aspect, the marker generation unit (1f) preferably generates a three-dimensional model (K1, K11) of the inspection range (R1, R11) based on three-dimensional position information included in the inspection range (R1, R11). The marker generation unit (1f) generates a projected image of the three-dimensional model (K1, K11) as the marker image (M1, M11).

In the above-described inspection system (1, 1A, 1B), a person (H1) can easily confirm the inspection range (R1, R11) within the captured image (G1, G11).

In the inspection system (1, 1A, 1B) of the eighth aspect of the above-described embodiment, in the seventh aspect, it is preferable that the marker generation unit (1f) sets the shape of the three-dimensional model (K1, K11) to a rectangular shape including three mutually orthogonal faces specified in the captured image (G1, G11).

The above-mentioned inspection systems (1, 1A, 1B) can easily generate marker images (M1, M11).

In the inspection system (1, 1A, 1B) of the ninth aspect according to the above-described embodiment, in the seventh aspect, the marker generation unit (1f) preferably uses dimensional information of the inspection object (9) to generate a three-dimensional shape of the three-dimensional model (K11) that includes at least one surface specified in the captured image (G11).

The above-mentioned inspection systems (1, 1A, 1B) can easily generate marker images (M1, M11).

In the inspection system (1, 1A, 1B) of the tenth aspect of the above-described embodiment, in the seventh aspect, the marker generation unit (1f) preferably uses dimensional information of the inspection object (9) to generate a three-dimensional shape of the three-dimensional model (K11) that includes at least one side specified in the captured image (G11).

The above-mentioned inspection systems (1, 1A, 1B) can easily generate marker images (M1, M11).

In the inspection system (1, 1A, 1B) of the eleventh aspect of the above-described embodiment, in the seventh aspect, the marker generation unit (1f) preferably uses dimensional information of the inspection object (9) to generate a three-dimensional shape of the three-dimensional model (K11) that includes at least one point specified in the captured image (G11).

The above-mentioned inspection systems (1, 1A, 1B) can easily generate marker images (M1, M11).

In the inspection system (1, 1A, 1B) of the twelfth aspect of the above-described embodiment, in the sixth aspect, the marker generation unit (1f) preferably inputs the captured images (G1, G11) into a learning model constructed by machine learning using the captured images (G1, G11) and the ranges of the marker images (M1, M11) in the captured images (G1, G11) as training data, thereby obtaining the captured images (G1, G11) on which the marker images (M1, M11) are superimposed from the learning model.

The above-mentioned inspection systems (1, 1A, 1B) can easily generate marker images (M1, M11).

In the inspection system (1, 1A, 1B) of the thirteenth aspect according to the above-described embodiment, in any one of the sixth to twelfth aspects, the marker image (M1, M11) is a projection image of a three-dimensional model (K1, K11). It is preferable that the marker generation unit (1f) displays the marker image (M1, M11) on the screen (D1) by converting the three-dimensional coordinates of the three-dimensional model (K1, K11) into two-dimensional coordinates based on coordinate information indicating the position of the inspection object (9) in the three-dimensional space (W1) in which the inspection object (9) exists.

The above-mentioned inspection systems (1, 1A, 1B) can link marker images (M1, M11) to captured images (G1, G11).

The inspection system (1, 1A, 1B) of the 14th aspect of the above-described embodiment, in any one of the 6th to 13th aspects, preferably further includes a marker processing unit (1g) that performs at least one of the following processes on the marker image (M1, M11) displayed on the screen (D1) in response to an operation by a person (H1): movement, rotation, enlargement, and reduction. The inspection range setting unit (1e) modifies the inspection range (R1, R11) in response to at least one process performed on the marker image (M1, M11).

The above-mentioned inspection systems (1, 1A, 1B) can correct the inspection ranges (R1, R11) by correcting the marker images (M1, M11).

In the inspection system (1, 1A, 1B) of the 15th aspect of the above-described embodiment, in any one of the first to fourteenth aspects, the captured image (G1, G11) is preferably a live view image captured by a movable imaging unit (1a).

The above-mentioned inspection systems (1, 1A, 1B) can monitor captured images (G1, G11) in real time.

In the inspection system (1, 1A, 1B) of the 16th aspect of the above-described embodiment, in any one of the first to fifteenth aspects, the inspection object (9) is preferably reinforcement (91-94).

The above-mentioned inspection systems (1, 1A, 1B) can inspect reinforcement (91-94).

In the inspection system (1, 1A, 1B) of the 17th aspect of the above-described embodiment, in any one of the first to fifteenth aspects, it is preferable that the inspection object (9) is a building.

The above-mentioned inspection systems (1, 1A, 1B) can inspect buildings.

The inspection method of the 18th aspect of the above-described embodiment includes an image acquisition step (S1), a display step (S2), an inspection range setting step (S3), and an inspection step (S5). The image acquisition step (S1) acquires data of captured images (G1, G11) that are images of a three-dimensional inspection object (9) and include three-dimensional position information. The display step (S2) displays the captured images (G1, G11) on a screen (D1). The inspection range setting step (S3) sets inspection ranges (R1, R11) that indicate the range in which the inspection object (9) appears in the captured images (G1, G11) through operation by a person (H1). The inspection step (S5) inspects the inspection object (9) by performing image recognition processing on the inspection ranges (R1, R11).

The above-described inspection method allows for more accurate setting of the inspection range (R1, R11) within the captured image (G1, G11).

The program of the 19th aspect of the above-described embodiment causes a computer system to execute the inspection method of the 18th aspect.

The above program allows for more accurate setting of the inspection range (R1, R11) within the captured image (G1, G11).

1, 1A, 1B Inspection system 1a Imaging unit 1b Image acquisition unit 1c Display control unit 1e Inspection range setting unit 1f Marker generation unit 1g Marker processing unit 1h Inspection unit 9 Inspection object 91-94 Reinforcement D1 Screen W1 Three-dimensional space H1 Worker (person)
G1, G11: Captured image R1, R11: Inspection range M1, M11: Marker image K1, K11: 3D model P1-P4, P11-P13, P21: Points S1: Image acquisition step S2: Display step S3: Inspection range setting step S5: Inspection step

Claims (20)

an image acquisition unit that acquires data of an image of an inspection object having a three-dimensional shape, the image including three-dimensional position information;
a display control unit that displays the captured image on a screen;
an inspection range setting unit that sets, by human operation, an inspection range indicating a range in which the inspection object is captured in the captured image;
an inspection unit that inspects the inspection object by performing image recognition processing on the inspection range ,
the inspection range setting unit sets, as the inspection range, an imaging region in which an image of a rectangular body is captured, the four points specified by the human operation in the captured image displayed on the screen corresponding to the four vertices of the rectangular body;
one of the four points corresponds to a vertex common to three mutually orthogonal faces of the outline of the rectangular body, among the four vertices;
The remaining three points of the four points correspond to the ends of three sides extending from a vertex common to the three faces among the four vertices.
Inspection system. an image acquisition unit that acquires data of an image of an inspection object having a three-dimensional shape, the image including three-dimensional position information;
a display control unit that displays the captured image on a screen;
an inspection range setting unit that sets, by human operation, an inspection range indicating a range in which the inspection object is captured in the captured image;
an inspection unit that inspects the inspection object by performing image recognition processing on the inspection range,
The inspection range setting unit sets the inspection range using information on the dimensions of the inspection object.
Inspection system. a marker generation unit that generates a marker image that indicates the inspection range in the captured image,
The display control unit displays the marker image on the screen by superimposing the marker image on the captured image.
3. The inspection system of claim 1 or 2. an image acquisition unit that acquires data of an image of an inspection object having a three-dimensional shape, the image including three-dimensional position information;
a display control unit that displays the captured image on a screen;
an inspection range setting unit that sets, by human operation, an inspection range indicating a range in which the inspection object is captured in the captured image;
an inspection unit that inspects the inspection object by performing image recognition processing on the inspection range;
a marker generation unit that generates a marker image that indicates the inspection range in the captured image,
the display control unit displays the marker image on the screen by superimposing the marker image on the captured image;
The marker generation unit
generating a three-dimensional model of the inspection range based on the three-dimensional position information included in the inspection range;
The projected image of the three-dimensional model is used as the marker image.
Inspection system. The marker generation unit generates a shape of the three-dimensional model as a rectangular body including three mutually orthogonal faces specified in the captured image.
The inspection system of claim 4. The marker generation unit generates a shape of the three-dimensional model using information about dimensions of the inspection object, the shape including at least one surface specified in the captured image.
The inspection system of claim 4. The marker generation unit generates a shape of the three-dimensional model that includes at least one side specified in the captured image, using information about the dimensions of the object to be inspected.
The inspection system of claim 4. The marker generation unit generates a shape of the three-dimensional model using information about the dimensions of the inspection object, and generates a three-dimensional shape that includes at least one point specified in the captured image.
The inspection system of claim 4. The marker generation unit inputs the captured image into a learning model constructed by machine learning using the captured image and the range of the marker image in the captured image as training data, and thereby acquires the captured image on which the marker image is superimposed from the learning model.
The inspection system of claim 3. an image acquisition unit that acquires data of an image of an inspection object having a three-dimensional shape, the image including three-dimensional position information;
a display control unit that displays the captured image on a screen;
an inspection range setting unit that sets, by human operation, an inspection range indicating a range in which the inspection object is captured in the captured image;
an inspection unit that inspects the inspection object by performing image recognition processing on the inspection range;
a marker generation unit that generates a marker image that indicates the inspection range in the captured image,
the display control unit displays the marker image on the screen by superimposing the marker image on the captured image;
the marker image is a projection image of a three-dimensional model,
The marker generation unit converts three-dimensional coordinates of the three-dimensional model into two-dimensional coordinates based on coordinate information indicating a position of the inspection object in a three-dimensional space in which the inspection object exists, thereby displaying the marker image on the screen.
Inspection system. an image acquisition unit that acquires data of an image of an inspection object having a three-dimensional shape, the image including three-dimensional position information;
a display control unit that displays the captured image on a screen;
an inspection range setting unit that sets, by human operation, an inspection range indicating a range in which the inspection object is captured in the captured image;
an inspection unit that inspects the inspection object by performing image recognition processing on the inspection range;
a marker generation unit that generates a marker image that indicates the inspection range in the captured image,
the display control unit displays the marker image on the screen by superimposing the marker image on the captured image;
a marker processing unit that performs at least one of a process of moving, rotating, enlarging, and reducing the marker image displayed on the screen in response to the operation by the person;
The inspection range setting unit corrects the inspection range in accordance with the at least one process performed on the marker image.
Inspection system. The captured image is a live view image captured by a movable imaging unit.
12. An inspection system according to any one of claims 1 to 11. The inspection object is reinforcement
13. An inspection system according to any one of claims 1 to 12. The inspection object is a building.
13. An inspection system according to any one of claims 1 to 12. an image acquiring step of acquiring data of an image of an inspection object having a three-dimensional shape, the image including three-dimensional position information;
a display step of displaying the captured image on a screen;
an inspection range setting step of setting, by a human operation, an inspection range indicating a range in which the inspection object is captured in the captured image;
an inspection step of inspecting the inspection object by performing image recognition processing on the inspection range,
In the inspection range setting step, an imaging area in which an image of a rectangular body in which the four points specified by the human operation in the captured image displayed on the screen correspond to four vertices is set as the inspection range,
one of the four points corresponds to a vertex common to three mutually orthogonal faces of the outline of the rectangular body, among the four vertices;
The remaining three points of the four points correspond to the ends of three sides extending from a vertex common to the three faces among the four vertices.
Testing method. an image acquiring step of acquiring data of an image of an inspection object having a three-dimensional shape, the image including three-dimensional position information;
a display step of displaying the captured image on a screen;
an inspection range setting step of setting, by a human operation, an inspection range indicating a range in which the inspection object is captured in the captured image;
an inspection step of inspecting the inspection object by performing image recognition processing on the inspection range.
In the inspection range setting step, the inspection range is set using information on the dimensions of the inspection object.
Testing method. an image acquiring step of acquiring data of an image of an inspection object having a three-dimensional shape, the image including three-dimensional position information;
a display step of displaying the captured image on a screen;
an inspection range setting step of setting, by a human operation, an inspection range indicating a range in which the inspection object is captured in the captured image;
an inspection step of inspecting the inspection object by performing image recognition processing on the inspection range;
a marker image setting step of generating a marker image indicating the inspection range in the captured image,
In the marker image setting step,
The marker image is displayed on the screen by being superimposed on the captured image;
generating a three-dimensional model of the inspection range based on the three-dimensional position information included in the inspection range;
The projected image of the three-dimensional model is used as the marker image.
Testing method. an image acquiring step of acquiring data of an image of an inspection object having a three-dimensional shape, the image including three-dimensional position information;
a display step of displaying the captured image on a screen;
an inspection range setting step of setting, by a human operation, an inspection range indicating a range in which the inspection object is captured in the captured image;
an inspection step of inspecting the inspection object by performing image recognition processing on the inspection range;
a marker image setting step of generating a marker image indicating the inspection range in the captured image,
In the marker image setting step, the marker image is displayed on the screen in a state where it is superimposed on the captured image;
the marker image is a projection image of a three-dimensional model,
In the marker image setting step, the marker image is displayed on the screen by converting three-dimensional coordinates of the three-dimensional model into two-dimensional coordinates based on coordinate information indicating the position of the inspection object in the three-dimensional space in which the inspection object exists.
Testing method. an image acquiring step of acquiring data of an image of an inspection object having a three-dimensional shape, the image including three-dimensional position information;
a display step of displaying the captured image on a screen;
an inspection range setting step of setting, by a human operation, an inspection range indicating a range in which the inspection object is captured in the captured image;
an inspection step of inspecting the inspection object by performing image recognition processing on the inspection range;
a marker image setting step of generating a marker image indicating the inspection range in the captured image,
In the marker image setting step, the marker image is displayed on the screen in a state where it is superimposed on the captured image;
a marker processing step of performing at least one of a movement, a rotation, an enlargement, and a reduction on the marker image displayed on the screen in response to the operation by the person,
In the inspection range setting step, the inspection range is corrected in accordance with the at least one process performed on the marker image.
Testing method. A computer system is configured to execute the inspection method according to any one of claims 15 to 19.
Program.