JP7585255B2 - Inspection device and inspection method - Google Patents

Description

The present invention relates to an inspection device and an inspection method for inspecting the surface of an object to be inspected, such as an aircraft, and more specifically to an inspection device and an inspection method that can reduce the inspection time while suppressing a decrease in inspection accuracy.

Various inspection devices have been proposed for inspecting the exterior of aircraft (see, for example, Patent Document 1). In Patent Document 1, an arm extending from the ground is used to obtain the three-dimensional coordinates of the camera, and a distance sensor is used to detect the tilt of the camera, thereby determining the position of the image captured by the camera.

The inspection equipment was large and took a long time to move, which meant that the inspection took a long time. Even if they tried to shorten the inspection time by introducing multiple inspection devices, this was difficult to achieve because the devices would interfere with each other. In the first place, inspection equipment is relatively expensive, so it was difficult to introduce multiple units of inspection equipment.

Japanese Patent Application Publication No. 2019-219378

The present invention was made in consideration of the above problems, and its purpose is to provide an inspection device and an inspection method that can reduce the inspection time while suppressing a decrease in inspection accuracy.

An inspection device for achieving the above-mentioned object is an inspection device for inspecting the condition of the surface of an object to be inspected, comprising a camera that photographs the surrounding scenery along with the surface of the object to be inspected, a processing mechanism that acquires image data from the camera and estimates the position of the camera based on the surrounding scenery, and a leg structure having one end fixed to the camera and the other end configured to be able to contact the surface of the object to be inspected at at least three points, wherein the leg structure has a leg portion having one end fixed to the camera and a contact portion connected to the other end of the leg and configured to be able to contact the surface of the object to be inspected at at least three points, and the contact portion is configured as a frame body, and the frame body is configured to be able to contact the surface of the object to be inspected .

An inspection method for achieving the above-mentioned object is an inspection method for inspecting the surface condition of an object to be inspected, the inspection method comprising: a camera for photographing the surface of the object to be inspected as well as the surrounding scenery; a processing mechanism for acquiring image data from the camera; and a leg structure fixed to the camera , the leg structure having a leg portion having one end fixed to the camera and a contact portion connected to the other end of the leg portion and configured to be able to contact the surface of the object to be inspected at at least three points, the contact portion being configured as a frame body, the frame body being configured to be able to contact the surface of the object to be inspected, the inspection method comprising: an installation step for determining the position and attitude of the camera by contacting at least three points of the leg structure with the surface of the object to be inspected; a photographing step by the camera; and an estimation step by the processing mechanism for estimating the position of the camera based on the surrounding scenery of the image data acquired from the camera.

According to the present invention, the leg structure can be brought into contact with the surface of the object to be inspected and photographed with a camera, so image data can be acquired with the distance and orientation of the camera relative to the surface constant. This is advantageous in improving the accuracy of estimating the camera position from the surrounding scenery photographed along with the surface.

FIG. 1 is an explanatory diagram illustrating an outline of an inspection device. 1. FIG. 4 is an explanatory diagram illustrating a modified example of the inspection device of FIG. 1. FIG. 4 is an explanatory diagram illustrating a modified example of the inspection device of FIG. 1. FIG. 4 is an explanatory diagram illustrating a modified example of the inspection device of FIG. FIG. 1 is an explanatory diagram illustrating an aircraft in a plan view. 2 is an explanatory diagram illustrating a state in which an inspection is being performed by the inspection device of FIG. 1; FIG. 2 is an explanatory diagram illustrating an example of image data acquired by an inspection device. FIG. 8 is an explanatory diagram illustrating a modified example of FIG. 7 .

The inspection device and inspection method will be described below based on the embodiment shown in the figure. In the figure, the optical axis direction of the camera is indicated by arrow z, a first direction perpendicular to the optical axis direction z is indicated by arrow x, and a second direction perpendicular to the optical axis direction z and the first direction x is indicated by arrow y.

As shown in FIG. 1, the inspection device 1 includes a camera 2, a processing mechanism 3 that acquires image data from the camera 2, and a leg structure 4 that is fixed to the camera 2. In this embodiment, the camera 2 has a lens 5. For the sake of explanation, part of the surface of the lens 5 is shown by a dashed line in the figure.

Camera 2 is, for example, a semi-spherical camera capable of capturing 360° in the horizontal direction. Camera 2 may be a spherical camera capable of capturing 360° in the horizontal and vertical directions. The configuration of camera 2 is not limited to this. It is sufficient that the camera 2 is configured to capture the surface scratches of the aircraft body or wings, which is the object of inspection, and the surrounding scenery. Camera 2 may be, for example, a camera with a wide-angle lens or a fisheye lens, or may be configured by combining multiple cameras. In this case, for example, one camera can capture the surface of the object of inspection, and another camera can capture the surrounding scenery. In this specification, the surrounding scenery refers to the wings, windows, engines, etc. of the aircraft. In other words, the surrounding scenery refers to other structures that exist around the surface of the object of inspection. The surrounding scenery may include the color and pattern of the aircraft.

The processing mechanism 3 is connected to the camera 2 by wire or wirelessly, and is configured to acquire image data from the camera 2. The processing mechanism 3 is configured to estimate the position of the camera 2 from the surrounding scenery of the acquired image data. In this embodiment, the processing mechanism 3 is incorporated in the camera 2. For the sake of explanation, the processing mechanism 3 is shown by a dashed line in the figure. The processing mechanism 3 may also be configured to be located outside the camera 2.

In the embodiment illustrated in FIG. 1, the leg structure 4 has four legs 4a, one end of which is fixed to the camera 2, and a contact portion 4b connected to the other end of the leg portion 4a. The contact portion 4b comes into contact with the surface of the object to be inspected. The contact portion 4b is composed of a pair of rod-shaped bodies, and the pair of rod-shaped bodies are positioned in a parallel relationship to each other. The rod-shaped bodies extend along the second direction y, and are configured to connect the upper ends of a pair of legs 4a that face each other in the second direction y.

The leg structure 4 is not limited to the above configuration. It may have a configuration in which one end is fixed to the camera 2 and the other end can contact the surface of the object to be inspected at at least three points. As illustrated in FIG. 2, the leg structure 4 may be composed of two legs 4a, one end of which is fixed to the camera 2, and a contact portion 4b connected to the other end of the leg portion 4a. The contact portion 4b is composed of a pair of rod-shaped bodies that are parallel to each other. The leg portion 4a is connected to approximately the center of the contact portion 4b in the second direction y.

In this embodiment, the processing mechanism 3 is configured separately from the camera 2 and is disposed outside the camera 2. The camera 2 and the processing mechanism 3 are connected by wire or wirelessly.

As shown in FIG. 3, the contact portion 4b may be configured as a frame. In this embodiment, the leg structure 4 has one leg 4a and a contact portion 4b configured as a triangular frame. This triangular frame is curved so as to be convex toward the camera 2. This contact portion 4b is shaped to easily contact the surface along the curvature of the surface of the object to be inspected. If the surface of the object to be inspected is flat, the frame may not be curved. The shape of the frame is not limited to a triangle. The frame may be configured as a polygon such as a square or a pentagon, or may be formed in a circle.

As shown in FIG. 4, the leg structure 4 may be made of a cylindrical transparent material. In this embodiment, the leg structure 4 is made of a colorless transparent material that transmits light, such as acrylic resin. The leg structure 4 may also be made of a colored transparent material. The lens 5 of the camera 2 is disposed inside the leg structure 4. The upper end of the leg structure 4 is formed into a flat or curved shape to match the shape of the surface of the object to be inspected. The entire leg structure 4 is made of a transparent material that transmits light. This prevents the leg structure 4 from interfering with the imaging by the camera 2. A portion of the cylindrical leg structure 4 may be made of a transparent material.

In the leg structure 4 of the embodiment illustrated in Figures 1 to 3, at least a portion may be made of a transparent material. This is advantageous in preventing the leg structure 4 from blocking the surrounding scenery. In addition, the entire leg structure 4 may be made of a transparent material.

Next, an inspection method using the inspection device 1 will be described. An example will be described in which the object to be inspected 6 is an aircraft wing, as illustrated in FIG. 5. The object to be inspected 6 is not limited to aircraft. The object to be inspected 6 can also be a wind turbine blade, a rocket, or a ship. The object to be inspected 6 is not limited to the exterior of an aircraft, and can also be the interior. The object to be inspected 6 can also be, for example, piping in a factory. When a scratch 7 is formed on the surface of the object to be inspected 6, as illustrated in FIG. 5, the inspection device 1 is installed near the scratch 7.

As shown in FIG. 6, first, the inspection device 1 is installed on the surface 6a of an aircraft wing, which is the object to be inspected 6 (hereinafter, this may be referred to as the installation process). This inspection device 1 has the landing gear structure 4 of the embodiment shown in FIG. 1. In the installation process, for example, the inspection device 1 is installed in a position facing the scratch 7 formed on the surface 6a. At that time, the inspection device 1 is installed with at least three points of the landing gear structure 4 in contact with the surface 6a. The landing gear structure 4 has a strength sufficient to prevent deformation even when the inspection device 1 is pressed against the surface 6a.

When attempting to bring at least three points of the contact portion 4b, which is composed of a pair of rod-shaped bodies, into contact with the surface 6a, the pair of rod-shaped bodies will extend in a second direction y perpendicular to a first direction x, which is the curvature direction of the surface 6a. At this time, the contact portion 4b will be in contact with the surface 6a along a line extending in the second direction y. In other words, the contact portion 4b will be in contact with the surface 6a at three or more points.

The plane of the leg structure 4 is determined by contacting the surface 6a at three or more points with the leg structure 4. Therefore, the attitude (direction and inclination) of the camera 2 with respect to the curved surface 6a and the distance from the surface 6a to the lens 5 of the camera 2 are uniquely determined. In this specification, the attitude of the camera 2 refers to the direction and inclination of the camera 2 with respect to the surface 6a. For example, when photographing the scratch 7 shown in Figure 6, the attitude and distance of the camera 2 with respect to the scratch 7 will be almost constant no matter how many times the photograph is taken.

The leg structure 4 may be configured to contact the surface 6a via a predetermined imaginary surface. The leg structure 4 may be configured, for example, of three or more rod-shaped bodies protruding from the camera 2. If the other ends of all the rod-shaped bodies are in contact with the surface 6a, the attitude of the camera 2 is uniquely determined.

While the inspection device 1 is pressed against the surface 6a, the camera 2 takes an image (hereinafter, sometimes referred to as the photographing process). The camera 2 photographs the surface 6a having the scratch 7 as well as the surrounding scenery. An example of image data obtained by photographing with the camera 2 is shown in FIG. 7. The scratch 7 is captured in the center of the image data. For the sake of explanation, the area of the surface 6a to be inspected is shown by a dashed line in FIG. 7. The area outside the dashed line is the surrounding scenery. The surrounding scenery includes the boundary between the aircraft body and the wing, and the engine installed on the wing.

The processing mechanism 3, which has acquired the image data shown in FIG. 7, estimates the position of the camera 2 based on the surrounding scenery (hereinafter, this may be referred to as the estimation process). The position of the camera 2 when the image data was acquired can be estimated from its relative position to the wings and windows of the aircraft. Drawing data such as 3D CAD data of the aircraft may be stored in advance in the processing mechanism 3, and the position of the camera 2 may be estimated from the surrounding scenery using this drawing data. Alternatively, the surface 6a of the inspection object 6 may be photographed in advance at multiple locations, and the positional relationship between the camera 2 and the surrounding scenery may be stored in the processing mechanism 3 as basic information. Based on this basic information, the processing mechanism 3 can be configured to estimate the position of the camera 2 in newly acquired image data. For the sake of explanation, parts other than the inspection object 6 are colored gray in FIG. 7.

As shown in FIG. 8, even if the range of the surrounding scenery captured by the camera 2 is narrow, the aircraft engine, the seams and colors of the wing surfaces, etc. can be obtained as part of the surrounding scenery. The processing mechanism 3 can accurately estimate the position of the camera 2 based on this surrounding scenery.

The camera 2 captures the surface 6a containing the flaw 7, etc., to be inspected, as well as the surrounding scenery, so the processing mechanism 3 can estimate the position of the surface 6a containing the flaw 7, etc., from the surrounding scenery. The inspection device 1 can obtain position information from the image data. By accumulating image data including position information, maintenance of the inspection object 6, such as an aircraft, can be performed efficiently.

The image is acquired by bringing the leg structure 4 into contact with the surface 6a, so the distance between the camera 2 and the surface 6a is constant when the image is acquired. When a specified range of the surface 6a is photographed, the image data contains almost the same surrounding scenery. This is advantageous for improving the accuracy of the position information of the image data.

Since the position information of the image data can be obtained with high accuracy, there is no need for arms extending from the ground. It is possible for workers to carry the inspection device 1 and perform inspections. This is advantageous for shortening inspection times. Because the inspection device 1 can be carried, it is also possible to inspect wind turbine blades for wind power generation and rockets at high altitudes. Because the inspection device 1 can be made smaller, multiple inspection devices 1 can be used to perform inspections simultaneously. It is possible to significantly shorten inspection times.

The inspection device 1 may be configured so that multiple leg structures 4 with different configurations are prepared in advance and the leg structure 4 fixed to the camera 2 can be switched. For example, the inspection device 1 may be configured so that multiple leg structures 4 with different configurations, as exemplified in Figures 1 to 4, are prepared in advance. Also, the inspection device 1 may be configured so that multiple leg structures 4 with different curvatures of the curved portion of the contact portion 4b, as exemplified in Figure 3, are prepared in advance.

The object 6 to be inspected may have parts with different curvatures on the surface 6a, such as the body and wings of an aircraft. The leg structure 4 may be switched to an appropriate one depending on the shape of the surface 6a of the object 6 to be inspected (hereinafter, this may be referred to as the switching process). The switching process allows the inspection device 1 to be pressed against the surface 6a of the object 6 to be inspected in an appropriate state and photographed. This is advantageous for improving the accuracy of estimating the position of the image data from the surrounding scenery in the estimation process.

The processing mechanism 3 may have in advance map information that divides the surface 6a of the inspection object 6 into multiple ranges. The map information can be created using drawing data or the like that shows the shape of an aircraft or the like. In this case, the processing mechanism 3 is configured to estimate the range among the multiple ranges in which the camera 2 is located based on the surrounding scenery. If the cameras 2 are located in the same range, the processing mechanism 3 can process the image data as image data obtained in the same range even if the installation positions of the cameras 2 are slightly different. It becomes easier to determine whether scratches 7, etc. included in the image data are newly developed or are the same as scratches 7 already found in a previous inspection.

Reference Signs List 1 Inspection device 2 Camera 3 Processing mechanism 4 Leg structure 4a Leg 4b Contact portion 5 Lens 6 Inspection object 6a Surface 7 Scratch x First direction y Second direction z Optical axis direction

Claims (6)

In an inspection device for inspecting the surface condition of an object to be inspected,
The inspection method includes a camera that captures an image of the surface of the inspection object and a surrounding scene, a processing mechanism that acquires image data from the camera and estimates the position of the camera based on the surrounding scene, and a leg structure that is fixed to the camera on one side and is configured so that the other side can contact the surface of the inspection object at at least three points,
the leg structure has a leg portion having one end fixed to the camera, and a contact portion connected to the other end of the leg portion and configured to be able to come into contact with the surface of the inspection object at at least three points;
An inspection device, characterized in that the contact portion is constituted by a frame body, and the frame body is configured to be able to come into contact with the surface of the object to be inspected. 2. The inspection device according to claim 1 , wherein at least a portion of the leg structure is made of a transparent material that transmits light. The inspection device according to claim 1, in which a plurality of leg structures of different shapes are prepared in advance, and the leg structure fixed to the camera can be switched. The inspection device according to claim 1, wherein the processing mechanism has map information that divides the surface into a plurality of ranges in advance, and is configured to estimate the range in which the camera is located among the plurality of ranges based on the surrounding scenery. 1. An inspection method for inspecting a surface condition of an inspection object, comprising:
The inspection system includes a camera that captures an image of the surface of the inspection object and a surrounding landscape, a processing mechanism that acquires image data from the camera, and a leg structure that is fixed to the camera,
the leg structure has a leg portion having one end fixed to the camera, and a contact portion connected to the other end of the leg portion and configured to be able to come into contact with the surface of the object to be inspected at at least three points, the contact portion being configured as a frame body, and the frame body being configured to be able to come into contact with the surface of the object to be inspected,
An inspection method comprising: an installation process for determining a position and attitude of the camera by contacting at least three points of the leg structure with the surface of the object to be inspected; a photographing process for the camera to photograph; and an estimation process for the processing mechanism to estimate the position of the camera based on the surrounding scenery of the image data obtained from the camera. The inspection method according to claim 5 , further comprising a switching step in which a plurality of leg structures having different shapes are prepared in advance and the leg structure fixed to the camera is switched depending on the shape of the surface.

Images