JP7637552B2 - Plate outline estimation device - Google Patents

Description

The present invention relates to an estimation device that estimates the contour line of multiple rectangular plate materials that are attached side by side by driving multiple fasteners into the plate.

Conventionally, as shown in Patent Document 1, board materials such as gypsum boards used for interior wall or ceiling materials are attached to a base via multiple fasteners such as screws. The multiple fasteners are driven into the board within a specified interval along at least the periphery of the board. After the board is installed, the installation status of the board is inspected to see if the intervals between fasteners in the same fastener row are within the specified interval.

JP 2018-031241 A

While such inspections can be performed visually, sometimes multiple plate materials attached using multiple fasteners are photographed and the mounting status of the plate materials is checked from the captured image. In this case, it is important to identify the contour line (i.e., the peripheral edge) of each plate material. However, since there are multiple fasteners in the vicinity along this contour line, it is expected that when attempting to identify the contour line of the plate material using image processing, it will be difficult to accurately identify the contour line from the images of these fasteners.

The present invention was made in consideration of these points, and its purpose is to provide an estimation device that can accurately estimate the outline of a rectangular plate material that is attached in multiple pieces using multiple fixing devices.

In consideration of the above problem, the estimation device for the contour of a plate material according to the present invention is an estimation device that estimates the contour of a rectangular plate material that is attached in a line by driving in a plurality of fasteners, and includes a fastener image identification unit that identifies images of the plurality of fasteners from an overall image including the plate material captured by an imaging device, an edge detection processing unit that performs edge detection processing on the overall image in which the images of the fasteners have been identified, a straight line detection processing unit that performs straight line detection processing on the overall image after edge detection processing, and a contour estimation unit that estimates the contour of the plate material from the overall image after straight line detection processing, and is characterized in that the estimation device further includes a fastener image removal unit that removes the images of the fasteners from the overall image before the straight line detection processing by the straight line detection processing unit after the images of the plurality of fasteners have been identified by the fastener image identification unit.

According to the present invention, the fastener image removal unit removes images of fasteners that are driven into the inside of the plate from the images of fasteners, so that the line detection processing unit can detect lines without being affected by the images of the fasteners when performing line detection processing on the entire image that has been subjected to edge detection processing. This allows the contour line estimation unit to accurately estimate the contour line of the plate for the entire image.

Here, the removal of the image of the fastener by the fastener image removal unit may be performed on the entire image before the edge detection process is performed, and the image from which the fastener is removed is not particularly limited as long as the image of the fastener can be removed. However, in a more preferred embodiment, the fastener image removal unit removes the image of the fastener from the entire image after the edge detection process based on the position of the image of the fastener identified by the fastener image identification unit.

According to this aspect, the image of the fastener is removed from the entire image that has been subjected to edge detection processing based on the position of the image of the fastener identified by the fastener image identification unit, so that the image of the fastener can be accurately removed. Here, when removing the image of the fastener from the entire image that includes the plate material captured by the imaging device, the color tone of the image of the fastener must be matched to the color tone of the surrounding plate material so that it is not detected by the edge detection and straight line detection processing. However, according to this aspect, the fastener image removal unit removes the image of the fastener that has been subjected to edge detection processing, so that it can be easily removed from the entire image that has been subjected to edge detection processing without adjusting the color tone of the image of the fastener.

In a more preferred embodiment, the estimation device further includes a fixture row identification unit that identifies a fixture row of fixtures that are driven into the inside of the sheet metal along the boundary between the sheet metals from among the contour lines of the sheet metal based on the images of the fixtures identified by the fixture image identification unit; an approximate line calculation unit that calculates an approximate line corresponding to the arrangement of multiple fixtures for each of the identified fixture rows; an estimated line calculation unit that calculates a line equidistant to the two approximate lines as an estimated line of the contour line based on the approximate lines of the two fixture rows that sandwich the boundary line between the adjacent sheet metals; and a boundary line determination unit that determines whether the estimated boundary line is good or bad based on the estimated line calculated by the estimated line calculation unit and the boundary line of the contour line estimated by the contour line estimation unit.

According to this aspect, a straight line that is equidistant from the approximation straight lines of the two fastener rows on either side of the boundary line of the plate material is taken as the estimated boundary line, and the quality of the estimated boundary line is judged based on this estimated line and the contour line of the boundary line estimated by the contour line estimation unit, so that a highly accurate boundary line can be obtained.

According to the present invention, the contour line of a rectangular plate material attached to multiple plates using multiple fixing devices can be estimated with high accuracy.

FIG. 2 is a schematic diagram of a bead pitch estimation device according to the present embodiment. FIG. 2 is a control block diagram of a calculation unit of the estimation device shown in FIG. 1 . 3 is a control block diagram of a boundary line determining unit shown in FIG. 2 . FIG. 13 is a diagram showing an entire image in which the image of the screw is specified. 5 is an enlarged image of part A in FIG. 4, where (a) is a diagram for explaining edge detection processing, (b) is a diagram for explaining processing for removing the image of a screw, and (c) is a diagram for explaining straight line detection processing. FIG. 13 is a diagram showing an entire image after line detection processing. 1A is a diagram showing the approximate straight lines of two screw rows that sandwich the boundary line between adjacent gypsum boards, and FIG. 1B is a diagram showing the estimated boundary line in addition to FIG. FIG. 2 is an estimation flow diagram using the estimation device shown in FIG. 1 .

The following describes the bead pitch estimation device 10 according to this embodiment, with reference to Figures 1 to 8.

1. Regarding boards and fasteners The estimation device 10 according to this embodiment is a device for estimating the contour of a rectangular board that is attached in a plurality of sheets and used for interior wall materials or ceiling materials, etc. In this embodiment, the estimation device 10 further estimates the pitch of fasteners driven into the boards. Here, fasteners are driven at a predetermined interval along at least the periphery of the boards. Examples of combinations of boards and fasteners include eaves soffit material (decorative material under the eaves) and nails for fixing it, structural plywood in wooden houses and nails for fixing it, and floor underlayment and nails/screws for fixing it.

In the following embodiment, gypsum board 5 is exemplified as the plate material, and screws 6 are exemplified as the fastener. Therefore, gypsum board 5 corresponds to the "plate material" in the present invention, and screws 6 correspond to the "fastener" in the present invention, and for example, the pitch of screws 6 (screw pitch) is the spacing between the fasteners in the present invention.

The fasteners for fixing the gypsum board 5 may be nails. The gypsum board 5 is used as an interior wall material or ceiling material. The gypsum board 5 is fixed with screws 6 along the length and width of the gypsum board 5 at a pitch equal to or less than that specified by laws and regulations. The screw pitch here refers to the distance between adjacent screws 6, 6 along the length and width of the gypsum board 5. When attaching the gypsum board 5 as an interior wall material, the long side of the gypsum board 5 is aligned vertically and the short side of the gypsum board 5 is aligned horizontally as shown in FIG. 1.

Here, the multiple screws 6, 6, ... are driven into the gypsum board 5 at a specified pitch along the periphery (contour line B) of the gypsum board 5, inside the periphery (contour line B). After the gypsum board 5 has been installed, the positions of the screws 6 driven along the contour line B of the gypsum board 5 are checked to check the installation condition of the gypsum board 5.

Furthermore, in this embodiment, the multiple screws 6, 6, ... are driven into the gypsum board 5 at the center of the width of the gypsum board 5 along the longitudinal direction of the gypsum board 5 at a specified pitch. After the gypsum board 5 is installed, the screw pitch is checked to see if it is within the specified pitch. The screw pitch is the center-to-center distance between adjacent screws 6 along the longitudinal and lateral directions when the gypsum board 5 is viewed from the front.

In view of this, in this embodiment, the estimation device 10 estimates the contour line B1 of the gypsum board 5 from the image G including the gypsum board 5 captured by the imaging device 20, and estimates the pitch of adjacent screws 6 as necessary.

2. Hardware Configuration of the Estimation Device 10 The estimation device 10 is configured with ROM, RAM, etc. as hardware, and includes a storage device 10A in which the conditions of the gypsum board 5, a screw pitch estimation program, etc. are recorded, and a calculation device 10B that executes the screw pitch estimation program.

An input device 31 and an output device 32 are connected to the estimation device 10. In this embodiment, the input device 31 and the output device 32 may be an integrated touch panel display. Data such as the specification conditions of the gypsum board 5 and a screw pitch estimation program are input via the input device 31. In this embodiment, image data captured by the imaging device 20 may be input via the input device 31. The data input by the input device 31 is stored in the memory device 10A. The output device 32 displays the image data captured by the imaging device 20, the calculation results calculated by the calculation device 10B, etc.

In this embodiment, the estimation device 10 is configured with a storage device 10A and a calculation device 10B, but may also include, for example, an input device 31 and an output device 32. In addition to the input device 31 and the output device 32, the estimation device 10 may further include an imaging device 20, and these may be integrated into a mobile terminal such as a smartphone or tablet.

3. Software Configuration of Estimation Device 10 In this embodiment, as shown in FIG. 2 , the estimation device 10 includes a screw image specifying unit (fixture image specifying unit) 11, an edge detection processing unit 12, a screw image removal unit (fixture image removal unit) 13, a straight line detection processing unit 14, a contour line estimation unit 15, a boundary line determination unit 16, a board image extraction unit 17, and a pitch calculation unit 18.

3-1. About the screw image identification unit 11 The screw image identification unit 11 identifies images of a plurality of screws 6 from the overall image G1 including the gypsum board 5 captured by the imaging device 20. Specifically, it identifies the position and range of each screw 6 in the overall image G1. It identifies the screws 6 from the overall image G1 using a classifier such as a cascade classifier that has machine-learned the shape of the screws.

The identified screws 6 are marked with a mark M. Each identified screw 6 is inside the square frame marked M in the overall image G1, and the center of the screw 6 coincides with the center of the mark M. In this embodiment, the screws 6 are surrounded by the mark M, but the mark M may be another shape such as a circular frame or a diamond frame, and the mark may be plotted at the position of the screw 6 so as to fill in the screw 6. Here, it is sufficient that at least the position (coordinates) of the screw 6 relative to the overall image G1 is identified from the shape of the screw 6. The screw image identification unit 11 may perform binarization or grayscale processing on the converted front image and identify the screw 6 from the number of pixels that have a predetermined brightness difference from the surrounding pixels.

3-2. About the edge detection processing unit 12 In this embodiment, edge detection processing is performed on the entire image G1 in which the image of the screw 6 is specified. In the edge detection processing, the edge of the entire image G1 may be detected using the Canny method or the like, or the edge of the entire image G1 may be detected by sequentially convolving various kernels with image matrices such as Frei-Chen, Prewitt, Roberts, and Sobel. Since these edge detections are generally known edge detection processes, detailed explanations are omitted. In this way, an entire image G2 subjected to edge detection processing as shown in FIG. 5(a) can be obtained. The boundary line B2 of the gypsum board 5 and the boundary line of the screw 6 are displayed in the entire image G2. FIGS. 5(a) to (c) are enlarged images of the A part in FIG. 4, and the screw 6 in FIG. 4 is drawn larger than FIG. 5(a) in order to clarify its position.

3-3. About the screw image removal unit 13 The screw image removal unit 13 removes the image of the screw 6 after the screw image specification unit 11 specifies the images of the multiple screws 6, 6, ..., and before the line detection processing unit 14, which will be described later, performs a line detection process. Specifically, in this embodiment, the screw image removal unit 13 removes the image of the screw 6 specified by the screw image specification unit 11 from the whole image G2 that has been subjected to the edge detection process.

To remove the image of the screw 6, the image of the screw 6 (or the image of the area surrounded by the mark M in FIG. 4) may first be removed, and then the pixels of the area from which the image of the screw 6 has been removed may be filled with pixels of the same brightness as the brightness of the surrounding image, for example, by commonly known expansion/contraction processing. In this way, there are no particular limitations on the processing, so long as the presence of the screw 6 is removed from the overall image G2 that has been subjected to edge detection processing.

In this way, the image of the screw 6 is removed from the overall image G2 subjected to edge detection processing based on the position of the image of the screw 6 identified by the screw image identification unit 11, so the image of the screw 6 can be accurately removed. The screw image removal unit 13 removes the image G1 of the screw 6 subjected to edge detection processing, so the image of the screw 6 can be easily removed from the overall image G2 subjected to edge detection processing without adjusting the color tone of the image of the screw 6. Note that the image of the screw 6 may be removed from the overall image G1 including the plaster board 5 captured by the imaging device 20, but in this case, the color tone of the image of the screw 6 must be matched to the color tone of the surrounding plaster board so that it is not detected by the edge detection and straight line detection processing.

3-4. Line detection processing section 14 The line detection processing section 14 performs line detection processing on the entire image G2 that has been subjected to edge detection processing. This allows for example discontinuous contour lines B1 caused by edge processing to be obtained as a single connected line LT. The line detection processing section 14 performs line detection on the entire image G2 that has been subjected to edge detection processing using a commonly known Hough transform, but line detection may also be performed using, for example, an LSD (Line Segment Detector).

3-5. Contour line estimation unit 15 The contour line estimation unit 15 estimates the contour line B3 of the plaster board 5 for the overall image G3 that has been subjected to the line detection process. Specifically, the contour line estimation unit 15 calculates the intersection points of the straight lines LT detected by the line detection processing unit 14, and estimates that an image formed of the four intersection points and the four straight lines LT is the contour line B3 surrounding the plaster board 5. Note that in FIG. 6, the straight lines LT and the contour line B3 are shown for the overall image G1 captured by the imaging device 20.

In this embodiment, the screw image removal unit 13 removes all images of the screws 6, including the images of the screws 6 that are driven into the inside of the plaster board 5 along the contour line B2 of the plaster board 5, from the images of the screws. Therefore, when the straight line detection processing unit 14 performs straight line detection processing on the edge-detected G2, it can detect the contour line B3 of the plaster board 5, which includes straight lines, without being affected by the images of the screws. This allows the contour line estimation unit 15 to accurately estimate the contour line B3 of the plaster board 5 for the overall image G1.

3-6. Regarding the Boundary Line Determination Unit 16 The boundary line determination unit 16 determines whether the boundary line BL between the two gypsum boards 5, among the estimated contour line B3, is a proper boundary line, and if the boundary line is a proper boundary line, determines that this is the boundary line between the two gypsum boards 5. As shown in Fig. 3, the boundary line determination unit 16 includes a screw row specification unit (fixture row specification unit) 16a, an approximate straight line calculation unit 16b, an estimated line calculation unit 16c, and a boundary line determination unit 16d.

As shown in Figures 4 and 7(a), the screw row identification unit 16a identifies screw rows PA, PB, ... of screws 6 driven into the inside of the plaster board 5 along the boundary line of the contour line B1 of the plaster board 5 in the overall image G1 based on the images of the screws identified by the screw image identification unit 11. The identification of the screw rows PA, PB, ... is carried out by, for example, determining that, for the screws 6 identified by the screw image identification unit 11, if the number of multiple screws 6, 6, ... lined up along the vertical direction of the overall image G1 is equal to or greater than a specified number (e.g., 10), these multiple screw rows PA, PB, ... are screws in the screw row.

As shown in FIG. 7(a), the approximate straight line calculation unit 16b calculates approximate straight lines LA, LB corresponding to the arrangement of the multiple screws 6, 6, ... for each screw row PA, PB, .... Specifically, a Cartesian coordinate system is set in the overall image G1 including the gypsum board 5, and a straight line equation is found by the least squares method for the centers of the screws 6, 6, ... of each screw row PA, PB, and the straight line of this equation is set as the approximate straight lines LA, LB.

As shown in FIG. 7(b), the estimated line calculation unit 16c calculates a straight line equidistant from the two approximate lines LA and LB as the estimated line L of the boundary line, based on the approximate lines of the two screw rows LA and LB that sandwich the boundary line B4 between adjacent gypsum boards 5 and 5. Note that the perpendicular lines drawn from any point on the estimated line L to the approximate lines LA and LB have the same length. This makes it possible to calculate the estimated line L of the boundary line based on the arrangement of the screw rows LA and LB.

The boundary line determination unit 16d determines whether the estimated boundary line B4 is acceptable or not based on the estimated line B5 calculated by the estimated line calculation unit 16c and the boundary line B4 of the contour line B3 estimated by the contour line estimation unit 15. Specifically, the boundary line B4 is determined to be an appropriate straight line under the condition that the slope of the boundary line B4 is within a specified range relative to the slope of the estimated line B5, and the intercepts of the estimated line B5 and the intercepts of the boundary line B4 are within specified ranges, and this is determined to be the boundary line between adjacent gypsum boards 5, 5. This makes it possible to obtain an accurate boundary line B4.

3-7. Regarding the board image extraction unit 17 The board image extraction unit 17 extracts an image of the gypsum board 5 surrounded by the contour line B3 from the overall image G1 of the gypsum board 5 from an image including the gypsum board 5. The board image extraction unit 17 performs projective transformation on the extracted image of the gypsum board 5 into a front image viewed from the front. As long as the extracted image of the gypsum board 5 can be projectively transformed into a front image viewed from the front, the method of projective transformation is not particularly limited. Since the contour line B3 of the gypsum board 5 is more accurately identified, a more accurate front image of the gypsum board 5 can be obtained by projective transformation.

3-8. About the pitch calculation unit 18 The pitch calculation unit 18 first identifies the image of the screw 6 in the projectively transformed image, and calculates the screw pitches Px and Py of adjacent screws 6, 6 for the screw rows determined in the longitudinal direction (vertical direction) and transverse direction (horizontal direction) of the gypsum board 5 (see FIG. 1). Specifically, the distance between the centers of the screws 6 or the centers of the identification marks is calculated as the number of pixels. Note that here, if the length of the long side or short side of the gypsum board 5 is input in advance, the actual length and width of each pixel can be calculated from the long side and short side, and the screw pitches Px and Py can be calculated from the actual length. In addition, if an image of the gypsum board 5 is acquired together with a scale of the actual length or a mark corresponding to it, the actual length and width of each pixel can be calculated.

The pitch calculation unit 18 further determines whether the calculated screw pitches Px, Py are within a specified range. If the measured screw pitch is within the specified range, for example, the two corresponding screws may be filled with circles of the same color, and if it is not within the specified range, the two corresponding screws may be filled with circles of different colors. As long as the difference between these can be clearly identified, the means is not particularly limited.

An estimation flow diagram using the estimation device will be described below with reference to FIG.
First, in step S1, the imaging device 20 captures an image of an area to be inspected, including the plaster board 5, to obtain an overall image G1. Next, in step S2, the screw image identification unit 11 identifies an image of the screw 6. In step S3, the edge detection processing unit 12 performs edge detection processing on the overall image G1 in which the image of the screw 6 has been identified.

In step S4, the screw image removal unit 13 can remove all images of the screws 6 from the overall image G2 that has been subjected to edge detection processing, including images of the screws 6 that are driven into the inside of the plaster board 5 along the contour line B1 of the plaster board 5. In step S5, the straight line detection processing unit 14 performs straight line detection processing on the overall image G2 that has been subjected to edge detection processing. In step S6, the contour line estimation unit 15 estimates the contour line B3 of the plaster board 5 for the overall image G3 that has been subjected to straight line detection processing.

In step S7, the screw row identification unit 16a identifies the screw rows PA, PB, ... of the screws 6 driven into the inside of the gypsum board 5. In step S8, the approximate line calculation unit 16b calculates approximate lines LA, LB corresponding to the arrangement of the multiple screws 6, 6, ... for each screw row PA, PB, .... In step S9, the estimated line calculation unit 16c calculates an estimated line B3 of the boundary line based on the approximate lines of the two screw rows LA, LB. In step S10, the boundary line determination unit 16d determines whether the estimated boundary line is appropriate based on the estimated line L calculated by the estimated line calculation unit 16c and the boundary line B4 estimated by the contour line estimation unit 15, and if it is appropriate, determines this boundary line B4 to be the boundary line.

In step S11, the board image extraction unit 17 extracts an image of the gypsum board 5 from the overall image G1 including the gypsum board 5, and in step S12, the board image extraction unit 17 performs projective transformation of the image of the gypsum board 5 into a front image, if necessary.

Next, in step S13, the pitch calculation unit 18 identifies the position coordinates of the screw 6 relative to the front image after projective transformation from the image (position) of the screw 6 before projective transformation identified in step S2. In step S14, the pitch calculation unit 18 calculates the screw pitches Px and Py of adjacent screws from the identified image of the screw 6. Finally, in step S15, the pitch calculation unit 18 determines whether the screw pitch is within a specified range.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above-mentioned embodiments, and various design changes can be made without departing from the spirit of the present invention as described in the claims.

5: plasterboard (board material), 6: screws (fixtures), 10: estimation device, 11: screw image identification section (fixture image identification section), 12: edge detection processing section, 13: screw image removal section (fixture image removal section), 14: straight line detection processing section, 15: contour line estimation section, 16a: screw row identification section (fixture row identification section), 16b: approximate straight line calculation section, 16c: estimated line calculation section, 16d: boundary line determination section, 20: imaging device, B1-B3: contour line, B4: boundary line, L: estimated line, PA, PB: screw row (fixture row), L: estimated line, LA, LB: approximate straight line, G1-G3: whole image

Claims (3)

An estimation device that estimates a contour line of a rectangular plate material that is attached in a line by driving in a plurality of fasteners, comprising:
a fixture image specifying unit that specifies images of a plurality of fixtures from an overall image including the plate material captured by an imaging device;
an edge detection processing unit that performs edge detection processing on the entire image from which the image of the fixing tool has been identified;
a line detection processing unit that performs a line detection process on the entire image that has been subjected to the edge detection process;
a contour line estimation unit that estimates a contour line of the plate material from the entire image that has been subjected to the straight line detection process,
the estimation device further comprising a fixture image removal unit that removes images of the fasteners from the overall image before straight line detection processing by the straight line detection processing unit after images of the plurality of fasteners have been identified by the fastener image identification unit. The plate outline estimation device according to claim 1, characterized in that the fixture image removal unit removes the image of the fixture from the overall image that has been subjected to edge detection processing based on the position of the image of the fixture identified by the fixture image identification unit. The estimation device includes a fastener row identifying unit that identifies a fastener row of fasteners driven into an inner side of the plate material along a boundary line between the plate materials among contour lines of the plate material based on an image of the fastener identified by the fastener image identifying unit;
an approximation line calculation unit that calculates an approximation line according to an arrangement of a plurality of fasteners for each of the identified fastener rows;
an estimated line calculation unit that calculates a straight line equidistant from two approximate straight lines of the two fastener rows that sandwich a boundary line between the adjacent plate materials as an estimated line of the contour line;
a boundary line determination unit that determines whether the estimated boundary line is acceptable or not based on the estimated line calculated by the estimated line calculation unit and the boundary line of the contour line estimated by the contour line estimation unit;
The plate material contour line estimation device according to claim 1 or 2, further comprising:

Images