JP7546507B2 - Reinforcement inspection device and method - Google Patents

Description

The present invention relates to a reinforcing bar inspection device and a reinforcing bar inspection method.

At construction sites for buildings made of reinforced concrete, reinforced steel concrete, steel frame, etc., reinforcement inspections are carried out on columns and beams that have been reinforced. As a reinforcement inspection system for use in such inspections, a system has been proposed that measures at least one of the diameter, number, and pitch of the rebar being inspected based on images of the rebar taken with a digital camera (Patent Document 1).

There is also a 3D measurement method and device that uses a stereo system to measure the shape of linear objects such as wires and cables (Patent Document 2). With the stereo system, corresponding points of the points to be measured on two images taken from different viewpoints are found, and the 3D position of the measurement point can be calculated from the corresponding points on each image and the relative positions of the two cameras using the principles of triangulation. All of the cameras used in the stereo system are color cameras.

JP 2016-3981 A International Publication WO2019/017360

However, conventional stereo cameras using color cameras often have false positives or fail to detect rebars, so there is a demand for further improvements in detection accuracy.

The present invention aims to provide a rebar inspection device and method that can improve rebar detection accuracy by using images from a monochrome camera, which can provide a higher resolution per pixel than images from a color camera with the same number or size of imaging elements, unless color information is required, and that can make the detection results easier to see by using images of the same object taken with a color camera.

The present invention has been made to solve at least some of the problems described above, and can be realized in the following aspects or application examples.

[ 1] One aspect of the reinforcing bar inspection device according to the present invention is as follows:
a first monochrome camera serving as a reference camera;
a second monochrome camera disposed at a predetermined distance from the first monochrome camera;
a color camera disposed at a predetermined distance from the first monochrome camera;
a main body on which the first monochrome camera, the second monochrome camera, and the color camera are mounted;
a synthesis processing unit that obtains three-dimensional reinforcement data including a plurality of reinforcement lines based on at least one pair of reinforcement images captured by the first monochrome camera and the second monochrome camera, which are reference cameras;
a display coordinate calculation unit that calculates display coordinates for arranging the plurality of reinforcement lines in the color reinforcement image captured by the color camera;
an estimation unit that estimates information on a posture relationship of the color camera with respect to the first monochrome camera based on a reinforcement image of the first monochrome camera and a color reinforcement image of the color camera;
a matching unit that performs matching between the reinforcement image captured by the first monochrome camera when capturing the color reinforcement image and the color reinforcement image;
Including ,
the first monochrome camera and the second monochrome camera have a higher resolution than the color camera;
The display coordinate calculation unit calculates the display coordinates in the color reinforcement image based on the matching result of the matching unit and the posture relationship information estimated by the estimation unit.

[ 2] Another aspect of the reinforcing bar inspection device according to the present invention is as follows:
a first monochrome camera serving as a reference camera;
a second monochrome camera disposed at a predetermined distance from the first monochrome camera;
a color camera disposed at a predetermined distance from the first monochrome camera;
a main body on which the first monochrome camera, the second monochrome camera, and the color camera are mounted;
a synthesis processing unit that obtains three-dimensional reinforcement data including a plurality of reinforcement lines based on at least one pair of reinforcement images captured by the first monochrome camera and the second monochrome camera, which are reference cameras;
a display coordinate calculation unit that calculates display coordinates for arranging the plurality of reinforcement lines in the color reinforcement image captured by the color camera;
an image calibration processing unit that acquires relative positional relationship information between the first monochrome camera, the second monochrome camera, and the color camera ;
Including ,
the first monochrome camera and the second monochrome camera have a higher resolution than the color camera;
The display coordinate calculation unit calculates the display coordinates in the color reinforcement image based on the relative positional relationship information obtained by the image calibration processing unit.

[ 3 ] In one aspect of the above-mentioned bar arrangement inspection device,
The image display apparatus may further include a display unit that displays a composite image in which the plurality of reinforcement lines are arranged on the color reinforcement image using the display coordinates calculated by the display coordinate calculation unit.

[ 4 ] One aspect of the reinforcing bar inspection method according to the present invention is to
The color camera and the display unit may be provided on a tablet terminal fixed to the main body.

[ 5 ] One aspect of the reinforcing bar inspection method according to the present invention is to
A method for inspecting reinforcement bars using the reinforcement bar inspection device according to the above [3],
synthesizing the three-dimensional reinforcement data including the plurality of reinforcement lines based on at least one pair of reinforcement images captured by the first monochrome camera and the second monochrome camera, which are reference cameras arranged at a predetermined interval;
calculating the display coordinates for arranging the plurality of reinforcement lines in the color reinforcement image captured by the color camera disposed at a predetermined distance from the first monochrome camera;
The color reinforcement image and the plurality of reinforcement lines are displayed superimposed on the display unit based on the display coordinates.

According to one aspect of the reinforcing bar inspection device and one aspect of the reinforcing bar inspection method of the present invention, the high resolution of a monochrome camera can be used to improve the detection accuracy of reinforcing bars, while a color camera can be used to make the detection results easier to see.

FIG. 1 is a diagram showing a schematic diagram of a bar arrangement inspection device and bar arrangement according to the present embodiment. FIG. 1 is a block diagram of a bar arrangement inspection device according to an embodiment of the present invention. 11 is a flowchart of a preparation process according to the present embodiment. 1 is a flowchart of a reinforcement bar inspection method according to the present embodiment. FIG. 13 is a diagram showing an example of a colored reinforcement bar image displayed on a display unit. FIG. 11 is a block diagram of a bar arrangement inspection device according to a first modified example. 13 is a flowchart of a preparation process according to the first modified example. 13 is a flowchart of a reinforcement bar inspection method according to Modification 1. FIG. 11 is a diagram illustrating a reinforcement bar inspection device according to a second modified example. FIG. 11 is a block diagram of a bar arrangement inspection device according to a second modified example.

Below, preferred embodiments of the present invention will be described in detail with reference to the drawings. Note that the embodiments described below do not unduly limit the content of the present invention described in the claims. Furthermore, not all of the configurations described below are necessarily essential components of the present invention.

One aspect of the reinforcement inspection device according to this embodiment is characterized in that it includes a first monochrome camera, a second monochrome camera arranged at a predetermined distance from the first monochrome camera, a color camera arranged at a predetermined distance from the first monochrome camera, a main body on which the first monochrome camera, the second monochrome camera, and the color camera are mounted, a synthesis processing unit that obtains three-dimensional reinforcement data including a plurality of reinforcement lines based on at least one set of reinforcement images taken by the first monochrome camera and the second monochrome camera, and a display coordinate calculation unit that calculates display coordinates for placing the plurality of reinforcement lines in the color reinforcement image taken by the color camera.

One aspect of the reinforcement inspection method according to this embodiment is characterized in that it synthesizes three-dimensional reinforcement data including multiple reinforcement lines based on at least one pair of reinforcement images taken by a first monochrome camera and a second monochrome camera arranged at a predetermined interval, calculates display coordinates for placing the multiple reinforcement lines on a color reinforcement image taken by a color camera arranged at a predetermined distance from the first monochrome camera, and displays the color reinforcement image and the multiple reinforcement lines superimposed on a display unit based on the display coordinates.

1. Overview of the Reinforcement Inspection Apparatus An embodiment of the present invention will be described with reference to Figures 1 and 2. Figure 1 is a schematic diagram showing a reinforcing bar inspection apparatus 10 and reinforcing bar 1 according to this embodiment, and Figure 2 is a block diagram of the reinforcing bar inspection apparatus 10 according to this embodiment.

As shown in Figures 1 and 2, the reinforcing bar inspection device 10 includes a first monochrome camera 12, a second monochrome camera 14, a color camera 16, a main body 11, a synthesis processing unit 30, and a display coordinate calculation unit 34. The reinforcing bar inspection device 10 may further include a display unit 38.

The reinforcing bar inspection device 10 can be used to inspect reinforcing bars 1 of columns, beams, etc. at the construction site of a reinforced concrete building. The reinforcing bar inspection device 10 can detect the main reinforcing bars 1a, etc. of the inspection target based on the image of the reinforcing bars 1 taken by the two monochrome cameras and create three-dimensional reinforcing bar data. Furthermore, the reinforcing bar inspection device 10 can display multiple reinforcing bar lines as a result of the detection on the display unit 38, superimposed on a color reinforcing bar image taken by the color camera 16. Furthermore, it is possible to set the color of the reinforcing bar line for each reinforcing bar diameter, for example, and display the distance between adjacent reinforcing bars arranged in parallel on the display unit 38. The operator can measure the diameter, number, pitch, etc. of the main reinforcing bars 1a, etc. based on the displayed three-dimensional reinforcing bar data and inspect whether they are as designed.

The main body 11 is equipped with a first monochrome camera 12, a second monochrome camera 14, and a color camera 16. All the cameras are arranged on the same plane (which may be a virtual plane) of the main body 11 facing the reinforcement 1 so that they can simultaneously photograph the reinforcement 1 to be inspected. The inside of the main body 11 may be equipped with a processor such as a central processing unit (CPU) or a graphics processing unit (GPU), a storage medium such as a read only memory (ROM), a random access memory (RAM), or a hard disk drive (HDD), a communication interface for high-speed data communication, and a user interface such as a touch panel or a keyboard. The main body 11 may be equipped with a display, or a separate tablet terminal placed on the mounting portion 18 of the main body 11 may be used as the display portion 38 as in this embodiment. In this case, the main body 11 and the tablet terminal may be directly connected to communicate data. The main body 11 and the display unit 38 may be configured as a notebook computer or a tablet terminal, and the three camera set may be attached externally. It is preferable that the bar arrangement inspection device 10 is a portable terminal at the construction site, but it does not necessarily have to be realized by a single device. For example, some processing units may be distributed to a client-server type computer or cloud computing.

As shown in FIG. 2, in addition to the synthesis processing unit 30 and the display coordinate calculation unit 34, the main body 11 may further include, for example, an image acquisition unit 22, an image calibration processing unit 23, an estimation unit 24, a first matching unit 26, a three-dimensional coordinate calculation unit 28, a second matching unit 32, and an output unit 36. The functions of these processing units are realized by a processor executing a program based on information stored in a storage medium of the main body 11 and cooperating with other hardware. The main body 11 may also be provided with, for example, a shutter button (not shown) as an operating means for the three cameras.

The first monochrome camera 12 and the second monochrome camera 14 are digital cameras with the same performance. The first monochrome camera 12 and the second monochrome camera 14 have a known optical system and imaging element, and can capture monochrome images. The imaging element is, for example, a monochrome CMOS sensor or a monochrome CCD sensor without a color filter. The first monochrome camera 12 and the second monochrome camera 14 have a higher resolution than the color camera 16 even with the same number of imaging elements because they do not use color filters. Therefore, the reinforcing bar inspection device 10 can perform highly accurate detection using high-resolution monochrome images. The first monochrome camera 12 and the second monochrome camera 14 capture the reinforcing bar 1 via the main body 11 in response to a command from the operator, and output the monochrome reinforcing bar image of the reinforcing bar 1 to the image acquisition unit 22.

The first monochrome camera 12 is a reference camera, and the second monochrome camera 14 is placed at a predetermined distance W1 from the first monochrome camera 12. The predetermined distance W1 is the center distance between the lenses of the first monochrome camera 12 and the second monochrome camera 14. The predetermined distance W1 can be set at any distance if distance calculation using parallax is possible, and is a fixed value at the time of shooting, but may be configured to be changeable depending on the object to be shot. In addition, the first monochrome camera 12 and the second monochrome camera 14 may be placed not only horizontally but also vertically or diagonally as long as they are separated by the predetermined distance W1. The first monochrome camera 12 and the second monochrome camera 14 are configured as a stereo camera 15, and using the reinforcement image captured by this stereo camera 15, the distance between the reinforcement 1 and the stereo camera 15 can be calculated from the parallax of the two images using the principle of triangulation. Then, using those distances, for example, by using a back projection method, the three-dimensional position of a point on the reinforcement 1 can be calculated. The stereo camera 15 needs to have at least two monochrome cameras, but may also be composed of three or more monochrome cameras arranged at a predetermined interval. Using three or more monochrome cameras can improve the detection accuracy of the bar arrangement inspection device 10.

The color camera 16 is disposed at a predetermined distance W2 from the first monochrome camera 12, which is a reference camera. The predetermined distance W2 is the distance between the centers of the lenses of the first monochrome camera 12 and the color camera 16. The predetermined distance W2 can be set to any distance, but it is desirable that the distance is within the adjustable deviation between the image captured by the first monochrome camera 12 and the image captured by the color camera 16, and is a fixed value at the time of shooting. In the example of FIG. 1, the color camera 16 is disposed directly below the first monochrome camera 12, but the position of the color camera 16 is not limited to this, and it is sufficient that the same object can be photographed simultaneously with the first monochrome camera 12. For example, the color camera 16 may be disposed in a line between the first monochrome camera 12 and the second monochrome camera 14 as long as it is separated by the predetermined distance W2. In addition, since the position of the color camera 16 is fixed relative to the other cameras, the main body 1 having the stereo camera 15 may be disposed so that the color camera 16 is located in a line between the first monochrome camera 12 and the second monochrome camera 14.
A monocular color camera may be fixed to the sensor 1.

The color camera 16 has a known optical system and imaging element, and is capable of capturing color images. The imaging element is, for example, a CMOS sensor or a CCD sensor equipped with RGB color filters. The color camera 16 captures images of the reinforcement 1 via the main body 11 in response to an operator's command, and outputs a color image of the reinforcement 1 to the image acquisition unit 22.

The image acquisition unit 22 can acquire at least one set of reinforcement images captured simultaneously by the first monochrome camera 12 and the second monochrome camera 14, and may acquire multiple sets within a given time, for example. The reinforcement image acquired by one capture by the stereo camera 15 is a set of a first image (right image) acquired by the first monochrome camera 12 and a second image (left image) acquired by the second monochrome camera 14.

The image calibration processing unit 23 estimates calibration information for the first monochrome camera 12 and the second monochrome camera 14, and stores the information in a storage medium. The calibration information includes, for example, estimated values of internal parameters that are inherent parameters of the stereo camera 15, and estimated values of external parameters related to the relative positional relationship of the second monochrome camera 14 with respect to the first monochrome camera 12. The internal parameters include focal length, principal point coordinates, distortion coefficients, etc., and the external parameters include the relationship between the coordinate systems, rotation matrix, and translation vectors of the two cameras.

The estimation unit 24 estimates the orientation of the first monochrome camera 12 and the second monochrome camera 14. Here, "orientation" refers to the difference between the rotation and translational movement of the stereo camera 15 at the current time, when the position of the stereo camera 15 in the initial frame at the start of measurement is used as a reference.

When estimating the posture of the stereo camera 15, a reinforcement image may be acquired with a three-dimensional marker placed on any surface of the reinforcement 1 (for example, the front surface 2). The three-dimensional marker is an AR (Augmented Reality) marker. By photographing the three-dimensional marker so that it is included in the reinforcement image together with the reinforcement 1, which is the subject of the photograph, the position and angle of the three-dimensional marker included in the reinforcement image can be recognized, and as a result, the posture of the stereo camera 15 relative to the reinforcement 1 can be estimated.

The estimation unit 24 also estimates multiple reinforcement lines based on at least one set of reinforcement images acquired by the image acquisition unit 22. The reinforcement lines are virtual lines extending along the reinforcing bars in the first or second image, and are estimated to overlap the reinforcing bars, preferably along the central axis of the reinforcing bars. The reinforcement lines can be estimated using the characteristics of the reinforcing bars (e.g., linearity, nodes, brightness, intersections, etc.). The estimation unit 24 detects reinforcement 1 and estimates the reinforcement lines for all reinforcement images acquired by the image acquisition unit 22. For example, a shape detection algorithm with a learning function can be used as a method for estimating the reinforcement lines. Furthermore, the estimation unit 24 may estimate the diameter of the reinforcing bars of reinforcement 1 from the reinforcement images.

The estimation unit 24 also estimates the attitude relationship information of the color camera 16 relative to the first monochrome camera 12 based on the monochrome reinforcement image of the first monochrome camera 12 and the color reinforcement image of the color camera. The "attitude relationship information" is information related to the rotation and translational movement of the color camera 16 relative to the first monochrome camera 12, and can be obtained, for example, as a relational expression between a rotation matrix and a translational movement vector. The relational expression related to the rotational movement can be expressed by a rotation matrix, as well as by known rotation vectors, quaternions, Euler angles, etc.

The first matching unit 26 matches the reinforcement lines estimated by the estimation unit 24 for each set of reinforcement images. Matching is to compare two images and "match the same things." Here, "things" refer to "points" or "areas" in the images. The first matching unit 26 may match feature points of the reinforcing bars detected when the estimation unit 24 estimates the reinforcement lines. In this case, the reinforcement lines may be matched by determining that they correspond to the matched reinforcing bars. As a matching method, feature point matching may be adopted. Before matching, pre-processing may be performed on each image.

The three-dimensional coordinate calculation unit 28 calculates the three-dimensional coordinates of each reinforcement line estimated by the estimation unit 24. The three-dimensional coordinates can be obtained by first using the rebar position captured in advance by the stereo camera 15 as the reference point for the three-dimensional coordinates, or by calculating using a three-dimensional marker installed on the reinforcement 1 as the reference point for the three-dimensional coordinates. Since the reinforcement line passes over multiple characteristic points used in the matching process, the distance between each characteristic point and the stereo camera 15 can be measured to calculate the three-dimensional coordinates of the reinforcement line.

The synthesis processing unit 30 obtains three-dimensional reinforcement data including multiple reinforcement lines. Specifically, the synthesis processing unit 30 synthesizes the three-dimensional reinforcement data by placing the reinforcement lines in a reference three-dimensional coordinate system based on the three-dimensional coordinates of each reinforcement line obtained by the three-dimensional coordinate calculation unit 28. In other words, all reinforcement lines whose three-dimensional coordinates have been identified are associated with a common reference three-dimensional coordinate system. This makes it possible to express the reinforcement lines as three-dimensional data.

The second matching unit 32 matches the reinforcing bar image captured by the first monochrome camera 12 with the color reinforcing bar image captured by the color camera 16. The second matching unit 32 may, for example, match feature points of the reinforcing bars in the two images, or may further match feature points of the background other than the reinforcing bars in the images. As a matching method, matching using known local features such as SIFT (Scale-Invariant Feature Transform), SURF (Speed-Upped Robust Feature), ORB (ORiented-BRIEF), and AKAZE (Accelerated KAZE) can be used. To explain the matching in detail, the local features of the feature points calculated in the reinforcing bar image captured by the first monochrome camera 12 are calculated, and a search is made to determine where a point having the same feature is located in the color reinforcing bar image captured by the color camera 16.

The display coordinate calculation unit 34 calculates display coordinates for placing multiple reinforcement lines in the color reinforcement image captured by the color camera 16. The display coordinate calculation unit 34 calculates display coordinates in the color reinforcement image based on the matching results of the second matching unit 32 and the attitude relationship information estimated by the estimation unit 24. The display coordinate calculation unit 34 uses the attitude relationship information estimated by the estimation unit 24. The display coordinates are coordinates for placing three-dimensional reinforcement lines in the two-dimensional color reinforcement image, and also include the rotation angle of the three-dimensional reinforcement lines when placed in two dimensions.

The output unit 36 outputs the display coordinates calculated by the display coordinate calculation unit 34, the color reinforcement image, and data signals of the multiple reinforcement lines to the display unit 38. The output unit 36 outputs data signals from the main body 11 to the display unit 38 by wireless communication, for example, Bluetooth (registered trademark). The connection between the main body 11 and the display unit 38 may be wired communication.

The display unit 38 displays a composite image in which multiple reinforcing bar lines are arranged on the color reinforcing bar image using the display coordinates calculated by the display coordinate calculation unit 34. In the composite image, the reinforcing bar lines are arranged to match each of the reinforcing bars in the color reinforcing bar image. The reinforcing bar inspection device 10 can improve the detection accuracy of reinforcing bars by utilizing the high resolution of the first monochrome camera 12 and the second monochrome camera 14, and can make the detection results easier to see by using the color camera 16. The display unit 38 may include a touch panel type input means and operation means for the operator.

2. Reinforcement Inspection Method The reinforcing bar inspection method according to this embodiment will be described with reference to Figures 1 to 5. Figure 3 is a flowchart of the advance preparation process according to this embodiment, Figure 4 is a flowchart of the reinforcing bar inspection method according to this embodiment, and Figure 5 is a diagram showing an example of a colored reinforcing bar image 50 displayed on the display unit 38. In the following description of the reinforcing bar inspection method, an example using the reinforcing bar inspection device 10 will be described, but the present invention is not limited to this.

As shown in FIG. 3, prior to the posture estimation process (S10) in FIG. 4, the image calibration processing unit 23 executes advance preparation processes (S1 to S3). The advance preparation processes (S1 to S3) include a process (S1) of photographing the calibration board with the stereo camera 15, a process (S2) of performing calibration and estimating the internal and external parameters of the stereo camera 15, and a process (S3) of saving the calibration information of the stereo camera 15.

Process of photographing the calibration board (S1): The reinforcing bar inspection device 10a photographs the same calibration board using the first monochrome camera 12 and the second monochrome camera 14.

Step (S2) of performing calibration and estimating the internal and external parameters of the stereo camera 15: The image calibration processing unit 23 of the bar arrangement inspection device 10 estimates the internal parameters (focal length and distortion matrix) from each image captured in S1. Furthermore, the calibration board is photographed by the stereo camera 15 with the distortion corrected, and the image calibration processing unit 23 estimates the external parameters (movement and rotation) of these stereo cameras 15.

Step (S3) of storing calibration information of the stereo camera 15: The reinforcing bar inspection device 10 stores the external parameters estimated in S2 in the storage medium of the main body 11 as calibration information of the stereo camera 15.

As shown in Figure 4, after the preliminary preparation processes (S1 to S3), the reinforcement inspection method includes at least a step of obtaining three-dimensional reinforcement data (S20), a step of calculating display coordinates (S46), and a step of outputting to the display unit 38 (S48). The reinforcement inspection method preferably further includes a step of estimating posture (S10), a step of identifying the rebar diameter (S30), a step of reflecting the current measurement results in the retained data (S40), a matching step (S42), and a step of estimating the posture relationship between the first monochrome camera 12 and the color camera 16 (S44). The steps will be explained in the order of the flowchart in Figure 4.

Attitude estimation process (S10): The bar arrangement inspection device 10 executes an attitude estimation process (S10) from the image of the stereo camera 15 to estimate the attitude of the stereo camera 15. Here, the "attitude of the stereo camera 15" is the difference between the rotation and translational movement of the stereo camera 15 at the current time point relative to the position of the stereo camera 15 in the initial frame at the start of measurement. By estimating the attitude of the stereo camera 15, the three-dimensional coordinates of the reinforcement line estimated at the current time point using the attitude estimated in this process (S10) in S20 described below can be converted to the initial attitude position (the position of the stereo camera 15 in the initial frame at the start of measurement), resulting in the effect of synthesizing three-dimensional reinforcement data.

The pose estimation process (S10) includes a process (S11) of capturing an image with the stereo camera 15, a process (S12) of calibrating the distortion, a process (S14) of parallelization transformation, and a process (S16) of estimating the pose of the stereo camera 15.

Step of photographing with stereo camera 15 (S11): The reinforcing bar inspection device 10 photographs the reinforcing bar 1 to be measured with the stereo camera 15. The image acquisition unit 22 acquires the monochrome reinforcing bar image photographed by the stereo camera 15.

Distortion calibration process (S12): The estimation unit 24 uses the pair of reinforcement images captured in S11 to calibrate the distortion from the internal parameters (focal length, principal point coordinates, distortion coefficient) of the first monochrome camera 12 and the second monochrome camera 14. Here, "distortion" refers to the distortion of the image captured through the camera lens.

Parallelization process (S14): The estimation unit 24 uses the reinforcement image captured in S11 to perform parallelization transformation based on the external parameters of the first monochrome camera 12 and the second monochrome camera 14 (the relationship between the coordinate systems, rotation matrix, and translation vectors of the two cameras).

Process of estimating the posture of the stereo camera 15 (S16): The estimation unit 24 estimates the posture of each of the first monochrome camera 12 and the second monochrome camera 14 in the current frame. The "current frame" is the monochrome reinforcement image captured at the time of S11.

Step of obtaining three-dimensional reinforcement data (S20): The synthesis processing unit 30 synthesizes three-dimensional reinforcement data including multiple reinforcement lines based on at least one pair of monochrome reinforcement images taken by the first monochrome camera 12 and the second monochrome camera 14, which are positioned at a predetermined distance W1 apart. Specifically, the estimation unit 24, the first matching unit 26, and the three-dimensional coordinate calculation unit 28 calculate the three-dimensional coordinates of the multiple reinforcement lines from the monochrome reinforcement image acquired by the image acquisition unit 22, and the synthesis processing unit 30 places the multiple reinforcement lines in a reference three-dimensional coordinate system to synthesize the three-dimensional reinforcement data. The reference three-dimensional coordinate system is a coordinate system that corresponds to the monochrome reinforcement image taken by the first monochrome camera 12, which is the reference camera.

Step of identifying rebar diameter (S30): The reinforcing bar inspection device 10 identifies the diameter of reinforcing bars 1, such as main bars 1a, shear reinforcement bars 1b, and abdominal bars 1c, based on at least one set of monochrome reinforcing bar images taken by the first monochrome camera 12 and the second monochrome camera 14. The monochrome reinforcing bar image used in S30 is the monochrome reinforcing bar image used in S20.

Process of reflecting the current measurement results in the stored data (S40): The reinforcing bar inspection device 10 reflects the current measurement results obtained from S10, S20, and S30 in the data stored in the storage medium (main memory) of the main body 11.

Matching process (S42): The second matching unit 32 performs matching between the monochrome reinforcement image captured by the first monochrome camera 12 and the color reinforcement image captured by the color camera 16 at the same time as this reinforcement image. By performing matching, in S44 and S46 described below, the pose relationship between the first monochrome camera 12 and the color camera 16 can be estimated, and the position of the estimated three-dimensional reinforcement data in the color reinforcement image captured by the color camera 16 can be calculated. Matching can be performed, for example, based on shape, size, identification marks, etc.

Process of estimating the attitude relationship between the first monochrome camera 12 and the color camera 16 (S44): Based on the results of S42, the estimation unit 24 of the reinforcement inspection device 10 estimates attitude relationship information between the first monochrome camera 12 and the color camera 16. If the attitude relationship information can be estimated in S44 using the matching results of S42, it becomes possible to arrange the three-dimensional reinforcement data reflected in S40 on the color reinforcement image captured by the color camera 16 in S48 described below.

Step of calculating display coordinates (S46): In the reinforcement inspection device 10, the display coordinate calculation unit 34 calculates display coordinates for arranging multiple reinforcement lines 40 in the color reinforcement image 50 captured by the color camera 16 placed at a predetermined interval W2 from the first monochrome camera 12. Here, the "display coordinates" refer to the two-dimensional coordinates of the projection destination calculated using the three-dimensional coordinates obtained in S20 and stored in the storage medium in S40, and the rotation matrix and translation vector estimated in S44.

Step of outputting to the display unit 38 (S48): As shown in FIG. 5, in the bar arrangement inspection device 10, the output unit 36 outputs the display coordinates calculated in S46, the color bar arrangement image, and each data signal of the multiple bar arrangement lines to the display unit 38. The display unit 38 displays the color bar arrangement image 50 and the multiple bar arrangement lines 40 on the display unit 38 based on the display coordinates output from the output unit 36. In the composite image 62 displayed on the display unit 38, the bar arrangement lines 40 are displayed superimposed on the bar arrangement 1 reflected in the color bar arrangement image 50, making it easy to understand the relationship between the bar arrangement 1 and the bar arrangement line 40. The color bar arrangement image 50 is displayed in a state where each bar arrangement 1 matches the bar arrangement line 40. By displaying the detection results on the display unit 38 in an easy-to-understand manner in this way, it becomes easier to inspect whether they are as designed. In addition, although not shown, the reinforcing bar diameter identified in S30 may be displayed superimposed on the color bar arrangement image 50 in the composite image 62 displayed on the display unit 38.

In this way, with this reinforcing bar inspection method, the images from the first monochrome camera 12 and the second monochrome camera 14, which have higher resolution than the images from the color camera 16 with the same number or size of imaging elements, can be used to improve the accuracy of reinforcing bar detection, and the detection results can be made easier to see by using the color camera 16.

Following S48, the reinforcement inspection device 10 may execute a brush-up process or a process for deleting the reinforcement lines 40. The brush-up process is a process for correcting parts of the reinforcement lines 40 displayed on the display unit 38 that do not match the reinforcement 1, such as deleting an erroneously estimated reinforcement line or extending the reinforcement line. The brush-up process may be performed automatically by the reinforcement inspection device 10, or may be performed manually while viewing the image on the display unit 38, or may be performed by combining automatic and manual processes. The process for deleting the reinforcement lines is a process for manually deleting the reinforcement lines while viewing the image on the display unit 38.

3. Reinforcement Inspection Device According to Modification 1 A reinforcing bar inspection device 10a according to modification 1 will be described with reference to Fig. 6. The basic configuration of the reinforcing bar inspection device 10a is the same as that of the reinforcing bar inspection device 10 described with reference to Figs. 1 and 2, so a description of the overlapping parts will be omitted. Fig. 6 is a block diagram of the reinforcing bar inspection device 10a according to modification 1.

As shown in FIG. 6, the bar arrangement inspection device 10a includes an image calibration processing unit 23a that acquires relative positional relationship information of the first monochrome camera 12, the second monochrome camera 14, and the color camera 16, instead of the image calibration processing unit 23 of the bar arrangement inspection device 10 described above. In addition to the image calibration processing unit 23a, the main body 11a of the bar arrangement inspection device 10a includes an image acquisition unit 22, an estimation unit 24a, a matching unit 26a, a three-dimensional coordinate calculation unit 28, a synthesis processing unit 30, a display coordinate calculation unit 34a, and an output unit 36. The matching unit 26a corresponds to the first matching unit 26 of the bar arrangement inspection device 10 described above, and executes similar processing.

The image calibration processing unit 23a stores relative positional relationship information of the first monochrome camera 12, the second monochrome camera 14, and the color camera 16, as well as the internal parameters of each camera, such as the camera matrix and distortion matrix, in the storage medium of the main body 11a.

Here, "relative position relationship information" refers to information regarding the rotational and translational movement of the second monochrome camera 14 relative to the first monochrome camera 12, and the rotational and translational movement of the color camera 16 relative to the first monochrome camera 12. The "relative position relationship information" is the so-called external parameters of the stereo camera 15 and the color camera 16.

By acquiring the relative position relationship information in advance, it is possible to omit the process of matching a monochrome reinforcing bar image with a color reinforcing bar image each time an image is captured, as in the second matching unit 32 in the reinforcing bar inspection device 10 described above. Therefore, the main body 11a does not have a processing unit corresponding to the second matching unit 32. In addition, since the estimation unit 24a of the reinforcing bar inspection device 10a in Variation 1 has already acquired the relative position relationship information using the image calibration processing unit 23a, the estimation unit 24 in the reinforcing bar inspection device 10 described above does not estimate the attitude relationship information between the first monochrome camera 12 and the color camera 16 by S44.

The display coordinate calculation unit 34a calculates the display coordinates in the color reinforcement image based on the relative position relationship information obtained by the image calibration processing unit 23a. By using the relative position relationship information, the display coordinates in the color reinforcement image can be set by a short calculation.

4. Reinforcement Inspection Method According to Modification 1 The reinforcement inspection method according to modification 1 will be described with reference to Fig. 7 and Fig. 8. Fig. 7 is a flowchart of the advance preparation process according to modification 1, and Fig. 8 is a flowchart of the reinforcement inspection method according to modification 1.

The preparatory process (S1a to S3a) shown in FIG. 7 is performed prior to the reinforcing bar inspection method in FIG. 8. The preparatory process (S1a to S3a) includes a step of photographing a calibration board (S1a), a step of performing calibration and estimating the internal and external parameters of each camera (S2a), and a step of saving the relative positional relationship information of the cameras (S3a).

Process of photographing the calibration board (S1a): The reinforcing bar inspection device 10a photographs the same calibration board using the first monochrome camera 12, the second monochrome camera 14, and the color camera 16.

Step (S2a) of performing calibration and estimating the internal and external parameters of each camera: In the bar arrangement inspection device 10a, the image calibration processing unit 23a estimates the internal parameters (focal length and distortion matrix) from each image taken in S1a. Furthermore, the calibration board is photographed with the first monochrome camera 12, the second monochrome camera 14, and the color camera 16, with the distortions corrected, and the image calibration processing unit 23a estimates the external parameters (movement and rotation) between the first monochrome camera 12 and the second monochrome camera 14, and the external parameters between the first monochrome camera 12 and the color camera 16.

Step of saving the relative positional relationship information of the cameras (S3a): The reinforcing bar inspection device 10a saves the external parameters estimated in S2a in the storage medium of the main body 11a as the relative positional relationship information of the first monochrome camera 12, the second monochrome camera 14, and the color camera 16.

The reinforcing bar inspection method shown in FIG. 8 includes a posture estimation step (S10), a step of obtaining three-dimensional reinforcing bar data (S20), a step of identifying the reinforcing bar diameter (S30), a step of reflecting the current measurement results in the stored data (S40a), a step of calculating display coordinates (S46a), and a step of outputting to the display unit 38 (S48a). S10, S20, and S30 are the same as the steps in FIG. 4, so the explanation will be omitted.

Process of reflecting the current measurement results in the retained data (S40a): The reinforcing bar inspection device 10a reflects the current measurement results obtained from S10, S20, and S30 in the retained data stored in the storage medium.

Step of calculating display coordinates (S46a): The reinforcement inspection device 10a calculates display coordinates for arranging multiple reinforcement lines 40 in the color reinforcement image 50 captured by the color camera 16. Since the relative positional relationship information between the first monochrome camera 12 and the color camera 16 was obtained in the advance preparation processes (S1a to S3a) executed when the system was first started up, and multiple reinforcement lines were arranged in the reference three-dimensional coordinate system of the monochrome reinforcement image captured by the first monochrome camera 12 in S20 to obtain three-dimensional reinforcement data, the display coordinates can be calculated without going through a matching process.

Step of outputting to the display unit 38 (S48a): As shown in FIG. 5, in the reinforcement inspection device 10a, the output unit 36 outputs the display coordinates calculated in S46a, the color reinforcement image, and each data signal of the multiple reinforcement lines to the display unit 38. The display unit 38 displays the color reinforcement image 50 and the multiple reinforcement lines 40 superimposed on the display unit 38 based on the display coordinates output from the output unit 36.

In this way, according to the reinforcing bar inspection method of the first modification, it is possible to improve the detection accuracy of reinforcing bars by using images from the first monochrome camera 12 and the second monochrome camera 14, which have higher resolution than a color camera with the same number or size of image pickup elements, and to make the detection results easier to see by using the color camera 16. Also, the reinforcing bar inspection method of the first modification may also perform a brush-up process or a process to delete the reinforcing bar lines 40.

5. Reinforcement Inspection Device According to Modification 2 The reinforcing bar inspection device 10b according to modification 2 will be described with reference to Figures 9 and 10. Figure 9 is a schematic diagram of the reinforcing bar inspection device 10b according to modification 2, and Figure 10 is a block diagram of the reinforcing bar inspection device 10b according to modification 2. Note that (a) in Figure 9 is a front view seen from the display unit 38a side, and (b) is a rear view seen from the stereo camera 15 side.

As shown in (a) and (b) of FIG. 9, the reinforcing bar inspection device 10b according to the second modification is composed of a tablet terminal 39 and a main body 11b to which the tablet terminal 39 is fixed.

The tablet terminal 39 is a portable information processing device equipped with a display unit 38a and a color camera 16a. The tablet terminal 39 is substantially flat and large enough for an operator to carry it with one hand. The display unit 38 is, for example, a 7-inch to 12-inch LCD screen, and may include a touch panel type input means and wireless communication means. The bar arrangement inspection device 10b uses a color camera 16a built into the tablet terminal 39 instead of the color camera 16 in the above-mentioned embodiment and variant example 1. The tablet terminal 39 can be one with a commercially available built-in camera, so the bar arrangement inspection device 10b can be constructed at low cost.

The main body 11b is equipped with a first monochrome camera 12 and a second monochrome camera 14. The main body 11b fixes the tablet terminal 39 with a fixing means (not shown) in a state where the tablet terminal 39 is placed on the mounting portion 18. By fixing the tablet terminal 39 to a predetermined position on the main body 11b, the position of the color camera 16a relative to the first monochrome camera 12 is determined.

As in the above-described embodiment and modified example 1, a processing unit such as the image acquisition unit 22 may be provided in the main body 11b, but for example, as shown in FIG. 10, a tablet terminal 39 may include the image acquisition unit 22, the image calibration processing unit 23a, the estimation unit 24a, the matching unit 26a, the three-dimensional coordinate calculation unit 28, the synthesis processing unit 30, the display coordinate calculation unit 34a, and the output unit 36. Since the processing units denoted by the same reference numerals as in the above-described embodiment or modified example 1 basically have the same functions, duplicated explanations will be omitted here. The main body 11b can communicate with the tablet terminal 39 via wired communication or wireless communication such as Bluetooth (registered trademark).

The present invention is not limited to the above-described embodiment, and various modifications are possible. For example, the present invention includes a configuration that is substantially the same as the configuration described in the embodiment (for example, a configuration with the same function, method, and result, or a configuration with the same purpose and effect). The present invention also includes a configuration in which non-essential parts of the configuration described in the embodiment are replaced. The present invention also includes a configuration that has the same action and effect as the configuration described in the embodiment, or can achieve the same purpose. The present invention also includes a configuration in which publicly known technology is added to the configuration described in the embodiment.

1...Reinforcement, 1a...Main reinforcement, 1b...Shear reinforcement, 1c...Abdominal reinforcement, 10, 10a, 10b...Reinforcement inspection device, 11, 11a, 11b...Main body, 12...First monochrome camera, 14...Second monochrome camera, 15...Stereo camera, 16...Color camera, 18...Placement unit, 22...Image acquisition unit, 23, 23a...Image calibration processing unit, 24...Estimation unit, 26...First matching unit, 26a...Matching unit, 28...Three-dimensional coordinate calculation unit, 30...Synthesis processing unit, 32...Second matching unit, 34, 34a...Display coordinate calculation unit, 36...Output unit, 38, 38a...Display unit, 39...Tablet terminal, 40...Reinforcement line, 50...Colored reinforcement image, 60...Three-dimensional reinforcement data, 62...Synthetic image, W1, W2...Predetermined interval

Claims (5)

a first monochrome camera serving as a reference camera;
a second monochrome camera disposed at a predetermined distance from the first monochrome camera;
a color camera disposed at a predetermined distance from the first monochrome camera;
a main body on which the first monochrome camera, the second monochrome camera, and the color camera are mounted;
a synthesis processing unit that obtains three-dimensional reinforcement data including a plurality of reinforcement lines based on at least one pair of reinforcement images captured by the first monochrome camera and the second monochrome camera, which are reference cameras;
a display coordinate calculation unit that calculates display coordinates for arranging the plurality of reinforcement lines in the color reinforcement image captured by the color camera;
an estimation unit that estimates information on a relationship between an attitude of the color camera and the first monochrome camera based on a reinforcement image of the first monochrome camera and a color reinforcement image of the color camera;
a matching unit that performs matching between the reinforcement image captured by the first monochrome camera when capturing the color reinforcement image and the color reinforcement image;
Including ,
the first monochrome camera and the second monochrome camera have a higher resolution than the color camera;
the display coordinate calculation unit calculates the display coordinates in the color bar arrangement image based on the matching result of the matching unit and the posture relationship information estimated by the estimation unit. a first monochrome camera serving as a reference camera;
a second monochrome camera disposed at a predetermined distance from the first monochrome camera;
a color camera disposed at a predetermined distance from the first monochrome camera;
a main body on which the first monochrome camera, the second monochrome camera, and the color camera are mounted;
a synthesis processing unit that obtains three-dimensional reinforcement data including a plurality of reinforcement lines based on at least one pair of reinforcement images captured by the first monochrome camera and the second monochrome camera, which are reference cameras;
a display coordinate calculation unit that calculates display coordinates for arranging the plurality of reinforcement lines in the color reinforcement image captured by the color camera;
an image calibration processing unit that acquires relative positional relationship information between the first monochrome camera, the second monochrome camera, and the color camera ;
Including ,
the first monochrome camera and the second monochrome camera have a higher resolution than the color camera;
the display coordinate calculation unit calculates the display coordinates in the color bar arrangement image based on the relative positional relationship information obtained by the image calibration processing unit. In claim 1 or 2 ,
a display unit that displays a composite image in which the plurality of reinforcement lines are arranged on the color reinforcement image using the display coordinates calculated by the display coordinate calculation unit. In claim 3 ,
The bar arrangement inspection device, characterized in that the color camera and the display unit are provided on a tablet terminal fixed to the main body. A method for inspecting bar arrangement using the bar arrangement inspection device according to claim 3,
synthesizing the three-dimensional reinforcement data including the plurality of reinforcement lines based on at least one pair of reinforcement images captured by the first monochrome camera and the second monochrome camera, which are reference cameras arranged at a predetermined interval;
calculating the display coordinates for arranging the plurality of reinforcement lines in the color reinforcement image captured by the color camera disposed at a predetermined distance from the first monochrome camera;
a display unit that displays the colored reinforcement image and the plurality of reinforcement lines in a superimposed manner based on the display coordinates,

Images