JP7718295B2 - Inspection method - Google Patents

Description

The present invention relates to a method for inspecting clips that connect hoses and brackets.

In recent years, vehicles have been using hoses and brackets assembled to carry engine coolant and fuel. Typically, clips are used to attach the hose to the bracket. After the hose and bracket are assembled, an inspection is carried out to ensure that the clip is properly attached.

Patent Document 1 describes an inspection method in which an image of the inspection area is acquired using an imaging means and whether the inspection area is normal or not is determined.

Japanese Patent Application Laid-Open No. 2001-082928

The inspection method disclosed in Patent Document 1 can obtain the number of pixels (number of pixels) of an object when it is imaged by an imaging device, but there is a problem in that this number of pixels cannot be compared with the actual dimensions. As a result, it may not be possible to guarantee accurate judgment depending on the inspection item.

On the other hand, if the inspection of whether the assembly state using the hose, bracket, and clip is normal is performed by a human visual inspection, the inspection time becomes long, which causes a problem of reduced productivity.
The present invention provides an inspection method that improves the accuracy of determination and performs inspection efficiently.

The inspection method of the present invention is an inspection method for inspecting the assembly state of a hose, a bracket, and a clip joining the hose and the bracket, wherein an imaging means images the assembly state, an identification means identifies the respective positions and sizes of the clip and the bracket from the acquired image acquired by the imaging means, a calculation means associates the respective positions of the clip and the bracket and their respective sizes in the acquired image identified by the identification means with the sizes of the clip and the bracket in real space that were acquired in advance, and a determination means determines whether the assembly state is good or bad using the length in real space of the part to be inspected calculated from the size in the acquired image based on the association.
This allows automatic pass/fail judgment to be made while associating the pixels in the acquired image with the size of the object to be inspected.

This allows for more efficient testing while improving accuracy.

FIG. 1 is a diagram illustrating a configuration of an inspection device and an object to be inspected. FIG. 10 is a diagram showing an example of an assembled state of a hose, a bracket, and a clip. 1 is a cross-sectional view showing an object to be inspected along a vertical direction. FIG. 2 is a front view of the appearance of the inspection object. FIG. 10 is a horizontal cross-sectional view of the location where the clip is provided. FIG. 10 is a diagram showing first to fifth inspection items, an example of a standard, and an example of a defect that occurs when the standard is not met. 10 is a flowchart showing a procedure for inspecting a second inspection item. 10A and 10B are diagrams showing an example of the number of pixels in the acquired image of a bracket and a clip and their sizes in real space. FIG. 10 is a diagram showing an example of the ratio of the distance of the bracket to the distance of the clip to the inspection target location. 10 is a flowchart showing a procedure for inspecting a fourth inspection item. FIG. 10 is a perspective view showing the positions of the claws and oval portions of the clip. FIG. 10 is a diagram illustrating calculation of an elliptical shape in an elliptical portion. FIG. 10 is a diagram showing a state in which the angle of the claw portion is calculated assuming that the claw of the clip is on an ellipse. 10A and 10B are diagrams illustrating calculation of the angle of the claw portion. 10 is a flowchart showing a procedure for inspecting a first inspection item. 10A and 10B are diagrams showing the arc shapes of the hose and the bracket and the locations where the positional relationship between them is acquired. FIG. 10 is a diagram showing a state in which an inspection object is placed obliquely in an acquired image. FIG. 10 is a diagram showing a state in which the first center coordinates and the second center coordinates are connected by a straight line. FIG. 10 is a diagram showing a state in which a processed image is created from an acquired image by projective transformation.

First Embodiment
An inspection device and an inspection object according to an embodiment of the present invention will be described below with reference to the drawings. Fig. 1 shows an inspection device 1 and a vehicle 2 that is the inspection object. Here, a hose 21, a bracket 22, and a clip 23 (see Fig. 2) that joins the hose 21 and the bracket 22, which are the inspection objects, are assembled to the vehicle 2. The bracket 22 has a circular bottom 22a in top view and an upward protrusion 22b (see Fig. 3).

The inspection device 1 includes an imaging means 11 that captures images of the state in which each inspection object is assembled to the vehicle 2, an identification means 12 that identifies the position and size of each bracket 22 and clip 23 from the image captured by the imaging means 11, a calculation means 13, and a determination means 14 that determines whether the bracket 22 and clip 23 are good or bad based on the position and size of each bracket 22 and clip 23.

Note that below, with regard to real space, the horizontal direction may be referred to as the width direction or left-right direction, and the height direction perpendicular to the horizontal direction may be referred to as the up-down direction. Furthermore, with regard to images acquired by the imaging means 11, the left-right direction on the image may be referred to as the width, and the up-down direction on the image may be referred to as the height.

As shown in Figure 1, a camera can be used as the imaging means 11. Furthermore, a computer 3, which includes a main memory device, an auxiliary memory device, an arithmetic unit, and other components working together, can be used as the identification means 12 and the determination means 14. By executing a specific program, this computer can realize the functions of the identification means 12, the arithmetic unit 13, and the determination means 14.

The cameras used as the imaging means 11 can be industrial cameras, security cameras, or web cameras that are installed in a fixed position. This allows these imaging means 11 to capture images of the hose 21, bracket 22, and clip 23 that connects the hose 21 and bracket 22, which are installed on the vehicle 2.

The imaging means 11 can be a mobile device with a camera, such as a smartphone, held by the worker. Alternatively, a wearable camera worn by the worker can be used.

Alternatively, the imaging means 11 can capture images using a camera held by the robot hand. In this case, the robot hand's operation can be controlled by a computer 3 having an identification means 12, calculation means 13, and judgment means 14, or by another computer, and its position relative to the object to be inspected can be adjusted. Note that the imaging methods used by the imaging means 11 are not limited to these, and other methods may also be used.

The imaging means 11 can capture any location on the vehicle 2. For example, the imaging means 11 can capture images of the inside of the hood from the upper front of the vehicle 2, the interior of the vehicle from the upper rear of the vehicle 2, or from underneath the vehicle 2. The imaging means 11 can capture at least one of still images or moving images, but the following description will be given assuming that it captures still images.

The identification means 12 identifies the position and size of the bracket 22 and clip 23 from the image acquired by the imaging means 11.

For example, the identification means 12 can identify the position coordinates of the bracket 22 on the acquired image acquired by the imaging means 11 as (X1, Y1), and the width and height as (x1, y1). In other words, the identification means 12 can identify that the bracket 22 is located within a range of x1 pixels to the right and y1 pixels below the coordinate position (X1, Y1) on the acquired image. Typically, x1 pixels and y1 pixels are several tens to several hundreds of pixels each.

Similarly, the identification means 12 can identify the position coordinates of the clip 23 on the image acquired by the imaging means 11 as (X2, Y2), and the horizontal and vertical lengths as (x2, y2). In other words, the identification means 12 can identify that the clip 23 is located within a range of x2 pixels to the right and y2 pixels below the coordinate position (X2, Y2) on the acquired image. Typically, x2 pixels and y2 pixels are several tens to several hundreds of pixels each.

The calculation means 13 performs calculations based on the coordinates identified on the image by the identification means 12, allowing the determination means 14 to make a determination.

As an example, the identification means 12 identifies that the clip 23 is displayed with a height of 60 pixels on the image. Then, the calculation means 13, having in advance size information that the clip 23 is formed with a height of 36 mm, can calculate that 1 pixel on the image is 0.6 mm. Typically, this size information is the standard value for the size of the clip 23.

The determination means 14 determines whether the image is good or bad by matching the number of pixels in the width and height of the object in the image acquired by the identification means 12, and the number of pixels between the objects.

As an example, let us assume that the pixel spacing in the vertical direction between the bottom 22a of the bracket 22 and the clip 23 on the image, as determined by the determination means 12, is 6 pixels. Here, if the calculation means 13 determines that 1 pixel is 0.6 mm, then 6 pixels on the image will be 3.6 mm in real space.

Here, the determination means 14 determines that the spacing between the bracket 22 and the clip 23 is good if the spacing in the height direction between the bottom 22a of the bracket 22 and the clip 23 is within 2 to 7 mm. As a result, the determination means 14 can determine that the product is good because the spacing identified by the identification means 12 is 6 pixels on the image, which is equivalent to 3.6 mm in real space.

Next, the hose 21, bracket 22, and clip 23, which are the objects of inspection, will be described using Figures 2 to 5. Furthermore, an overview of the five inspection items performed on these objects using the inspection device 1 will be provided.

Note that Figure 2 is an example of a view from diagonally above showing the external appearance of the inspection object, hose 21, and protrusion 22b of bracket 22 connected and fastened with clip 23. Also, Figure 3 is an example of a cross-sectional view showing a cross section along the vertical direction of these inspection objects, Figure 4 is a front view showing the external appearance of these inspection objects from the front, and Figure 5 is a diagram showing the open state of claws 23a and 23b, which are the two ends of clip 23, and is a cross-sectional view showing a horizontal cross section of the location where clip 23 is installed.

As shown in Figures 2 to 4, the hose 21 and bracket 22 are arranged so that the hose 21 is on the upper side and the bracket 22 is on the lower side. Furthermore, the protruding portions 22b of the hose 21 and bracket 22 are each cylindrical with their axial direction extending in the vertical direction.

More specifically, as shown in Figure 3, bracket 22 is attached to the top of another component that has a wider diameter, and protruding portion 22b is cylindrical with its axis in the vertical direction. Meanwhile, the portion near the bottom of hose 21 is hollow and cylindrical with its axis in the vertical direction, with cylindrical bracket 22 inserted inside hose 21 at the bottom of hose 21. In other words, the inner circumferential surface of the bottom of hose 21 and the outer circumferential surface of protruding portion 22b of bracket 22 abut against each other in an opposing relationship.

As shown in Figure 3, the protruding portion 22b of the bracket 22 will be inserted into the hose 21. However, if this length is insufficient, i.e., if the insertion is shallow, the hose 21 and the protruding portion 22b of the bracket 22 will easily come loose. Therefore, the first inspection item is whether the length of the protruding portion 22b of the bracket 22 inserted into the hose 21 is sufficient. The inspection point for the first inspection item is indicated as "A" in Figure 3.

As shown in Figure 2, a clip 23 is arranged around the hose 21 in the circumferential direction at the location where the hose 21 and the protruding portion 22b of the bracket 22 are arranged inward and outward.

In other words, the clip 23 is short in the vertical direction and long in the horizontal direction, and is wrapped around the hose 21 near the end of the hose 21 while tightening it around the outer circumference of the hose 21. This allows the clip 23 to press the hose 21 inward, securing the hose 21 to the protruding portion 22b of the bracket 22.

As shown in Figure 3, if the clip 23 is not positioned appropriately, the hose 21 and the protruding portion 22b of the bracket 22 will not be sufficiently fixed, and the hose 21 will be more likely to come loose from the protruding portion 22b of the bracket 22. Therefore, the second inspection item is whether the position where the clip 23 is positioned is a specified distance from the end of the hose 21. The inspection location for the second inspection item is indicated as "B" in Figure 3.

Furthermore, as shown in Figures 2 and 5, the clip 23 has claws 23a and 23b at both ends in the extension direction. The clip 23 fastens the hose 21 to the protruding portion 22b of the bracket 22 when the claws 23a and 23b mesh with each other at their respective vicinity and are open within a predetermined range.

More specifically, if the gap between claws 23a and 23b is less than a predetermined angle, the force with which clip 23 fastens hose 21 to protruding portion 22b of bracket 22 will be insufficient, making it easier for hose 21 to come loose from protruding portion 22b of bracket 22. Therefore, the third inspection item is whether the gap between claws 23a and 23b is greater than a predetermined angle, such as 30 degrees or more. The inspection location for the third inspection item is indicated as "C" in Figure 5.

On the other hand, if the gap between claw portion 23a and claw portion 23b is too wide, interference with other components may occur, potentially causing damage to those components. Therefore, the fourth inspection item is whether the gap between claw portion 23a and claw portion 23b is equal to or less than a predetermined angle, such as 120° or less. The area to be inspected for the fourth inspection item is indicated as "D" in Figure 5.

As shown in Figure 4, the outer peripheral surface of the hose 21 and the bottom 22a of the bracket 22 are provided with paint marks 21a and 22c, respectively, to align the two. The paint marks 21a and 22c on the hose 21 and bracket 22 are provided so that twisting of the hose 21 is sufficiently minimized when the hose 21 and bracket 22 are connected in the correct positions.

In other words, if the hose 21 and bracket 22 are connected with these paint marks misaligned, the hose 21 will become twisted significantly, which may cause interference with other components and result in damage. Therefore, the fifth inspection item is whether the relative positional deviation between the paint mark 21a on the hose 21 and the paint mark 22c on the bottom 22a of the bracket 22 is within a specified range. The inspection location for the fifth inspection item is indicated as "E" in Figure 4.

Here, the paint mark 21a on the hose 21 and the paint mark 22c on the bracket 22 are placed almost on the front side. This front side is the direction in which the claws 23a and 23b are located when the clip 23 is wrapped around the hose 21. In other words, when the imaging range of the imaging means 11 is set so that the claws 23a and 23b are sufficiently visible, the paint mark 21a on the hose 21 and the paint mark 22c on the bracket 22 are also included in the imaging range.

The paint mark 22c on the bottom 22a of the bracket 22 may be placed, for example, near the outer periphery of the bottom 22a, or at another location such as the protrusion 22b, but should be located in a position that is included in the image captured by the imaging means 11 when the hose 21 and bracket 22 are properly assembled.

Figure 6 summarizes the first to fifth inspection items, an example of the standard, and an example of a problem that may occur if the hose 21, bracket 22, and clip 23 are arranged in a manner that does not comply with the standard.

Next, we will explain detailed examples of how to perform inspections using the inspection device 1 for each of the first through fifth inspection items.

When performing the first to fifth inspection items, the imaging unit 11 photographs the inspection object so that the hose 21, bracket 22, and clip 23 are sufficiently within the imaging range. Typically, the imaging unit 11 photographs the inspection object from the front and from an angle above. For example, the imaging unit 11 can photograph the inspection object from the front and from an angle above at an angle of approximately 30° from the horizontal, but this is not limited to this.

Below, to make it easier to understand the explanation of the testing methods for each test item, we will explain the second, fourth, third, first, and fifth test items in that order.

<Second inspection item>
First, a description will be given of the second inspection item using the inspection device 1. In the second inspection item, it is inspected whether the distance from the end of the hose 21 to the position where the clip 23 is arranged is a predetermined distance.

Note that a common issue when performing multiple of the first through fifth inspection items using the inspection device 1 rather than the human eye is matching the pixels in the image acquired by the imaging means 11 with lengths in real space. This section explains the inspection method steps for the second inspection item, as well as how to match the pixels in the acquired image with lengths in real space.

Figure 7 is a flowchart showing the procedure for inspecting the second inspection item using the inspection device 1. As mentioned above, an example of a non-defective product condition in this case can be set as a distance from the end of the hose 21 to the clip 23 being within 2 to 7 mm.

First, the worker checks the standard size of the object in advance using drawings, etc., and obtains size information (step S71).

In a later step, the pixels in the image captured by the imaging means 11 are converted to lengths in real space. However, if there is only one object to be inspected, the accuracy of the conversion between pixels in the image and lengths in real space will be low, so the ratio is calculated for two objects.

In this example, the first object is the bracket 22, and the second object is the clip 23. Specifically, this explanation assumes that information on the vertical lengths of the bracket 22 and the clip 23 in real space is acquired in advance and used.

The inspection device 1 detects the inspection object to be measured (step S72). Specifically, the imaging means 11 captures an image of the inspection object, and the identification means 12 identifies the size and position of the target area on the captured image. The imaging means 11 captures the image while keeping the distance from the imaging means 11 to the inspection objects (hose 21, bracket 22, and clip 23) as close as possible. For example, the image captured by the imaging means 11 should capture a sufficiently long image from the bottom end of the bracket 22 to the top end of the clip 23.

The size and position of the target area on the acquired image identified by the identification means 12 here refers to the vertical length and position of the bracket 22 on the image, the vertical length and position of the clip 23, and the length and position from the end of the hose 21 to the clip 23. Figure 8 is a simplified diagram of the acquired image acquired by the imaging means 11. As shown in Figure 8, the identification means 12 identifies the vertical length of the bracket 22 on the acquired image as 690 pixels, the vertical length of the clip 23 as 70 pixels, and the length from the end of the hose 21 to the clip 23 as 100 pixels, and the respective positions have been identified.

At this time, the calculation means 13, by using the identification means 12 to identify the position coordinates of the bracket 22 and clip 23, can calculate that the location from the end of the hose 21 to the clip 23 ("B" in Figure 4), which is the inspection target area for the second inspection item (inspection content in Figure 6), is located closer to the clip 23 than the bracket 22, and that the distance ratio is 1:4.

As a more specific example, as shown in FIG. 9, the calculation means 13 acquires the vertical center position coordinate (first position coordinate) of the clip 23 identified by the identification means 12, the vertical center position coordinate (second position coordinate) of the bracket 22, and the vertical center position coordinate (third position coordinate) of the distance from the end of the hose 21 to the clip 23. The calculation means 13 then calculates the difference between the vertical distance between the first position coordinate and the third position coordinate and the vertical distance between the second position coordinate and the third position coordinate. This results in a distance ratio of 1:4.

Next, the calculation means 13 uses the information on the position and size of the object to be inspected identified by the identification means 12 in step S72 to calculate the ratio of the pixel for each position in the acquired image to the length in real space (step S73).

For example, the calculation means 13 can calculate that 1 mm corresponds to 6.9 pixels because the standard vertical length of the bracket 22 in real space, which is known in advance, is 100 mm and its length in the acquired image is 690 pixels. On the other hand, the calculation means 13 can calculate that 1 mm corresponds to 7 pixels because the standard vertical length of the clip 23 in real space, which is known in advance, is 10 mm and its length in the acquired image is 70 pixels.

Then, since the distance ratio calculated in step S72 is 1:4, the calculation means 13 uses this ratio and calculates that 1 mm is 6.98 pixels at the distance from the end of the hose 21 to the clip 23.

Furthermore, the calculation means 13 can calculate that the distance from the end of the hose 21 identified by the identification means 12 to the clip 23 is 14.32 mm in real space, since it is 100 pixels in the acquired image.

Finally, the determination means 14 determines whether the distance between the end of the hose 21 and the clip 23 in real space, calculated in step S73, meets the conditions for a good product (step S74). Here, the condition for a good product is within 2 to 7 mm, and since the distance between the end of the hose 21 and the clip 23 in real space is calculated to be 14.32 mm in step S73, the determination means 14 can determine that the product is not good.

<Fourth inspection item>
Next, a fourth inspection item, i.e., a method for inspecting whether the gap between the claws 23a and 23b of the clip 23 is equal to or smaller than a predetermined angle, will be described. Fig. 10 is a diagram showing the inspection procedure for the fourth inspection item using the inspection device 1.

As shown in FIG. 11, the identification means 12 identifies the position and size of the claw portions 23a, 23b and the elliptical portion 21b of the clip 23 (step S101). AI learning such as deep learning can be used to identify each part by the identification means 12.

Here, the elliptical portion 21b refers to the portion of the hose 21 that is approximately circular in cross section when viewed from above, but is captured as an ellipse in the acquired image. This elliptical portion 21b is located near the upper portion of the hose 21 where the clip 23 is wrapped around it.

12, the calculation means 13 calculates the elliptical shape of the elliptical portion 21b (step S102). Specifically, the calculation means 13 calculates the degree of inclination of the hose 21 in the oblique direction from the length of the minor axis _rx and the length of the major axis _ry of the elliptical portion 21b.

Next, the calculation means 13 uses the position information of the claws 23a and 23b acquired by the identification means 12 and assumes that the claws 23a and 23b are located on this ellipse. That is, the calculation means 13 sets position coordinates on the ellipse that allow calculation of the angles of the claws 23a and 23b from the center of the ellipse (step S103).

Then, as shown in FIG. 13, the calculation means 13 calculates the angles θ1 and θ2 from the center of the ellipse using the position coordinates on the ellipse of the claws 23a and 23b set in step S103 and trigonometric functions (step S104).

Here, the details of the calculation procedure of the angles θ1 and θ2 in steps S103 and S104 will be described. Considering a point P(x, y) on an ellipse as shown in FIG. 14(a),
Here, r _x and _ry can be obtained in accordance with the detection of the elliptical portion 21 b of the hose 21 .

On the other hand, for example, the coordinates P'(x', y') of the claw portion 23a are calculated by detecting the positions of the claw portions 23a and 23b by the specifying means 12. At this time, as shown in FIG. 14(b),
It can be expressed as:

Here, if θ1 + θ2 is θ', then
It can be expressed as:

The determination means 14 determines whether θ' calculated in steps S103 and S104 meets the quality criteria (step S105). As an example, the determination means 14 determines that the product is quality if θ' is within 120°.

<Third inspection item>
Here, the third inspection item, i.e., the method for inspecting whether the gap between claw portion 23a and claw portion 23b is equal to or greater than a predetermined angle, can be performed in the same manner as the fourth inspection item.

In this case, the determination means 14 can make a determination in step S105 shown in FIG. 10 based on whether θ' calculated in steps S103 and S104 is equal to or greater than 30°, which is the acceptable product condition.

<First inspection item>
Next, a description will be given of the first inspection item, i.e., a method for inspecting whether or not the protruding portion 22b of the bracket 22 is sufficiently inserted into the hose 21. Fig. 15 is a diagram showing the inspection procedure for the first inspection item using the inspection device 1.

As a preliminary step, the inspection device 1 creates correct answer data for the position information of the object being inspected and performs learning using the created correct answer data (step S151).

Specifically, the inspection device 1 creates multiple sets of correct answer data, i.e., positional information, for learning by labeling images previously acquired by the imaging means 11 and the hose 21 and bracket 22 that appear in the images.

In this case, if the objects to be labeled are located close to each other, the identification means 12 can first label the objects so that the labeling range is narrower, and then add the narrower labeling range later. By labeling from such a narrow range, the identification means 12 can improve labeling accuracy. The identification means 12 can include position information and size information for the hose 21 and bracket 22 in the correct answer data created using this labeling.

The inspection device 1 then uses this correct answer data to learn the positions of the hose 21 and bracket 22. In the following, the inspection device 1 will be described as having fully learned the position information of the hose 21 and bracket 22 from multiple correct answer data.

Next, processing is performed using the acquired images showing the actual inspection targets, the hose 21 and bracket 22.

The imaging means 11 captures an image of the hose 21 and bracket 22. The identification means 12 then detects the positions of the hose 21 and bracket 22. Specifically, the identification means 12 identifies the position and type of each object, including the hose 21 and the bottom 22a of the bracket 22 (step S152).

In particular, the identification means 12 uses object detection to identify the arc shape of the lower end of the hose 21 and the arc shape of the outer periphery of the bottom 22a of the bracket 22 in the acquired image. Figure 16 shows the two arc shapes identified by the identification means 12 and their positional relationship for determination by the determination means 14.

Here, the identification means 12 can identify the positions of the hose 21 and bracket 22 using the position information of the hose 21 and bracket 22 learned in step S151.

As an example, the identification means 12 can perform processing to identify the position of the object by assuming that the positional relationship between the hose 21 and the bracket 22 learned in step S151 is substantially the same as the positional relationship between the hose 21 and the bracket 22 in the image acquired in step S152. In other words, the identification means 12 can identify the positions of the hose 21 and the bracket 22 in the image acquired in S152 by comparing the image acquired in step S152 with the content learned using the correct answer data in step S151 and processing the positions of the hose 21 and the bracket 22.

In the same manner as in step S151, the identification means 12 can perform the process of step S152 by first identifying the position of the target object within a narrow range, and then expanding the identified range, but this is not limited to this, and the identification procedure can be freely changed.

Here, the number of pixels in the vertical direction of the arc shape of the near half circumference of the lower end of the hose 21 in the acquired image identified by the identification means 12 is set to α pixels, and the number of pixels in the vertical direction of the arc shape of the near half circumference of the outer periphery of the bottom 22a of the bracket 22 is set to β pixels. This near side is the side closest to the imaging means 11 when capturing an image using the imaging means 11.

Furthermore, the difference in vertical position between the coordinates of the upper end of the portion identified as the arc shape of the lower end of the hose 21 and the coordinates of the lower end of the portion identified as the arc shape of the outer periphery of the bottom 22a of the bracket 22 is defined as γ pixels.

The calculation means 13 calculates the ratio between the pixel in the image acquired by the imaging means 11 and the length in real space (step S153). This ratio can be calculated using a method similar to that shown in steps S71 to S73 of the second test item described above, and so a detailed explanation will be omitted.

The calculation means 13 calculates the lengths α'mm, β'mm, and γ'mm in real space for the α pixel, β pixel, and γ pixel on the acquired image calculated in step S152, using the ratios calculated in step S153 (step S154).

The determination means 14 determines whether the result of subtracting α'mm and β'mm from γ'mm calculated in step S153 is within 1 mm, which is the acceptable product condition (step S155).

<Fifth inspection item>
Next, a fifth inspection item, that is, a method for inspecting whether or not the relative positional deviation between the paint mark 21a of the hose 21 and the paint mark 22c provided on the bottom 22a of the bracket 22 is within a predetermined range, will be described.

The identification means 12 acquires the position and size of the bracket 22 and clip 23, and the coordinates of the paint mark 21a and paint mark 22c in the image acquired by the imaging means 11.

Here, the calculation means 13 calculates the ratio between the pixel in the acquired image acquired by the imaging means 11 and the length in real space. This ratio can be calculated using the same method as shown in steps S71 to S73 of the second inspection item described above, so a detailed explanation will be omitted. However, for example, based on the position and size of the bracket 22 and clip 23, it is possible to calculate how many millimeters in real space one pixel in the acquired image at the position of the paint marks 21a and 22c corresponds to.

Typically, as shown in Figure 4, the identification means 12 identifies the horizontal pixel offset between paint marks 21a and 22c in the acquired image. Then, the calculation means 13 calculates the real-space offset distance between paint marks 21a and 22c from the horizontal pixel offset identified by the identification means 12.

The determination means 14 determines whether the deviation distance in real space between the paint marks 21a and 22c calculated by the calculation means 13 is within a predetermined range. As an example, the determination means 14 can determine that the product is non-defective if the deviation distance in real space between the paint marks 21a and 22c is within 3 mm.

In this way, the inspection device 1 can perform inspections for each of the first through fifth inspection items. Note that the inspection device 1 may perform any one of the first through fifth inspection items, or may perform a combination of multiple items.

<Example of preparation for acquiring images>
In the above description, when each of the first to fifth inspection items is performed by the inspection device 1, the bracket 22 is photographed in the lower center of the image acquired by the imaging means 11, and the hose 21 is photographed in the upper center of the image, as shown in Fig. 2. Furthermore, when identifying each inspection object by the identification means 12, this is done based on an uninclined rectangle consisting of two horizontal sides and two vertical sides on the acquired image.

However, as shown in Figure 17, the imaging means 11 may capture an image of the object to be inspected in a diagonal direction of the acquired image, i.e., the bracket 22 may be positioned at the bottom left of the acquired image and the hose at the top right of the acquired image. In such a case, if the identification means 12 performs identification processing based on a rectangular shape that is not tilted relative to the acquired image as shown in Figure 17, there is a high possibility that the area identified as the object to be inspected will include extraneous areas in addition to the area that is actually desired to be identified.

For this reason, the inspection device 1 performs image processing on the acquired image captured by the imaging means 11, and can create a processed image in which the bracket 22 is positioned at the bottom center of the acquired image and the hose 21 is positioned at the top center of the acquired image. An example of the procedure for creating this processed image is shown below.

First, when the inspection object in the image acquired by the imaging means 11 is tilted as shown in Figure 17, the identification means 12 acquires the positions and sizes of the two inspection objects. Here, the identification means 12 identifies the positions and sizes of the arc-shaped region at the bottom end of the bracket 22 and the arc-shaped region at the bottom end of the hose 21.

At this time, the identification means 12 identifies the position and size of these areas based on an untilted rectangle consisting of two horizontal sides and two vertical sides on the acquired image. Therefore, when the identification means 12 identifies the arc-shaped area at the bottom end of the bracket 22, it will include the surrounding area of this area, and similarly, when the identification means 12 identifies the arc-shaped area at the bottom end of the hose 21, it will include the surrounding area of this area.

Next, the calculation means 13 calculates the center coordinates (first center coordinates) of the arc-shaped area at the lower end of the bracket 22 identified by the identification means 12. Similarly, the calculation means 13 calculates the center coordinates (second center coordinates) of the arc-shaped area at the lower end of the hose 21 identified by the identification means 12.

Next, as shown in Figure 18, the calculation means 13 connects the first center coordinate and the second center coordinate with a straight line, and can calculate the tilt angle of the object to be inspected in the acquired image from the angle of this line.

Therefore, the calculation means 13 performs projective transformation on the acquired image based on the calculated tilt angle, and can create a processed image in which the bracket 22 is positioned at the bottom center and the hose 21 is positioned at the top center.

This allows the position and angle of the object being inspected in the image captured by the imaging means 11 to be converted so that the first to fifth inspection methods can be easily processed.

As a result of the above, the inspection device 1 can automatically determine whether the first to fifth inspection items are pass or fail by associating the pixels in the image acquired by the imaging means 11 with the length of the object being inspected in real space.

Specifically, in the inspection device 1, the imaging means 11 captures an image of the assembled state, and the identification means 12 can identify the respective positions and sizes of the clip 23 and bracket 22 from the captured images acquired by the imaging means 11. The calculation means 13 then uses the respective positions and sizes of the clip 23 and bracket 22 identified by the identification means 12 in the captured images and the standard sizes of the clip 23 and bracket 22 in real space that were previously acquired to associate pixels in the captured images with lengths in real space. Furthermore, the determination means 14 can use this association to determine the pass/fail of the assembly state for each inspection item, using the real-space length of the inspection target area calculated from the size in the captured image.

In this way, the inspection device 1 can automatically determine whether an object being inspected is good or bad, saving inspectors the time and effort required for visual inspection. Furthermore, since there is no need to deploy inspectors, the length of the production line can be shortened and the occurrence of human error can be reduced.

Furthermore, the inspection device 1 makes a pass/fail judgment after determining the correspondence between the pixel size in the acquired image and the length in real space, thereby improving judgment accuracy.

Furthermore, as shown in the third and fourth inspection items, even in situations where the claw portions 23a, 23b of the clip 23 must be photographed from an oblique direction using the imaging means 11, the degree of opening of the claw portions 23a, 23b can be determined.

The present invention is not limited to the above-described embodiment, and modifications can be made as appropriate without departing from the spirit of the invention. In other words, the above description has been omitted or simplified as appropriate for clarity, and those skilled in the art can easily modify, add, or convert each element of the embodiment within the scope of the present invention.

For example, while the above description assumes that the correspondence between pixels in the acquired image and lengths in real space is based particularly on the positions and sizes of the bracket 22 and clip 23, it is also possible to correspond pixels in the acquired image of the hose 21 or other inspection objects not listed above to lengths in real space. Furthermore, it is also possible to use three or more of these components to correspond pixels in the acquired image to lengths in real space.

REFERENCE SIGNS LIST 1 Inspection device 2 Vehicle 3 Computer 11 Imaging means 12 Identification means 13 Calculation means 14 Determination means 21 Hose 22 Bracket 23 Clip

Claims (1)

An inspection method for inspecting an assembly state of a hose, a bracket, and a clip that joins the hose and the bracket, comprising:
The imaging means images the assembled state,
The identifying means identifies the respective positions and sizes of the clip and the bracket from the captured image acquired by the imaging means,
a calculation means for associating pixels in the acquired image with lengths in real space by using the positions and sizes in the acquired image of the clip and the bracket identified by the identification means and the sizes in real space of the clip and the bracket that have been acquired in advance;
The determination means is
When determining whether the assembly state is good or bad using the length in real space of the inspection target portion calculated from the size in the acquired image by the association,
Assuming that the cross section of the hose is circular, and assuming that the hose in the acquired image has a shape that combines a part of an ellipse arc and a straight line , the inclination of the hose is calculated, and the calculated inclination of the hose is used to calculate the angle between the two claws provided on the clip, and the quality of the claws is determined,
a determination of the length of the protrusion provided on the bracket that has penetrated into the hose is made based on the arc shape of the front half of the circumference of the lower end of the hose, which is elliptical in the acquired image, and the arc shape of the front half of the circumference of the bottom of the bracket;
Testing method.