JP7620992B2 - Self-location estimation system and method for unmanned aerial vehicle, unmanned aerial vehicle, program, and storage medium - Google Patents

Description

The present invention relates to a system and method for estimating self-position for an unmanned aerial vehicle, an unmanned aerial vehicle, a program, and a storage medium.

Recently, attempts have been made to use drones to inspect the exterior walls of horizontally extending structures such as tunnels. Drones can estimate their own position in the horizontal direction using GPS satellite positioning information, as well as their own position in the vertical direction using an altimeter or the like, and fly autonomously along a flight plan route (Non-Patent Document 1 below). Then, an image of the tunnel exterior wall surface is captured by an imaging device such as a camera mounted on the unmanned aerial vehicle, and inspection is carried out based on the captured image.

Kenzo Nonami, "All about drone industrial applications", Ohmsha, February 2018, p.73

In such inspections using an imaging device mounted on an unmanned aerial vehicle, it is necessary to capture images of the outer wall surface of a structure such as a tunnel from a predetermined position. However, even if a flight plan route is set to fly a certain distance from the outer wall surface of the tunnel and the unmanned aerial vehicle flies according to the flight plan route set based on GPS measurement data, there may be an error between the set position and the actual position of the unmanned aerial vehicle relative to the structure.

Therefore, one of the objectives of the present invention is to provide a system and method that enables an unmanned aerial vehicle to fly autonomously at a predetermined position relative to a structure such as a tunnel when flying outside the structure.

One aspect of the present invention provides a self-position estimation system for an unmanned aerial vehicle flying along an outer wall surface of a structure extending laterally along a predetermined path, the unmanned aerial vehicle including a longitudinal section measurement device that measures a boundary shape within a longitudinal section in a direction intersecting the flight path of the unmanned aerial vehicle and generates longitudinal section measurement data, the self-position estimation system including a longitudinal section measurement data acquisition unit that acquires the longitudinal section measurement data measured by the longitudinal section measurement device when the unmanned aerial vehicle flies along the outer wall surface of the structure extending laterally along the predetermined path, and a longitudinal section position estimation unit that matches the acquired longitudinal section measurement data with a first predetermined shape corresponding to the outer wall surface of the structure and estimates the position of the unmanned aerial vehicle within the longitudinal section based on the self-position relative to the matched predetermined shape.

In one aspect of the present invention, the first predetermined shape corresponds at least in part to a geometric shape, and the longitudinal section position estimation unit matches the geometric shape to the longitudinal section measurement data and estimates the position within the longitudinal section based on the self-position with respect to the matched geometric shape.

In one aspect of the present invention, the first predetermined shape is all or part of a circle or an ellipse, and the longitudinal section position estimation unit estimates the longitudinal section position of the unmanned aerial vehicle within the longitudinal section as a distance based on the center point of the first predetermined shape and an angle with respect to a reference axis.

In one aspect of the present invention, the longitudinal section measurement data is discrete data, and the longitudinal section position estimation unit removes outliers in the discrete data and performs matching.

In one aspect of the present invention, the system further includes a flight control unit that controls the flight of the unmanned aircraft so that it flies along the flight plan route based on the position of the unmanned aircraft within the longitudinal section estimated by the longitudinal section position estimation unit.

In one aspect of the present invention, a flight plan route is set as a plurality of waypoints, and the waypoints are set including an angle relative to a reference axis passing through a reference point of the structure corresponding to the center point of a first predetermined shape in the longitudinal section, and a distance from the outer wall surface of the structure at the angle.

In one aspect of the present invention, the unmanned aerial vehicle includes an imaging device, an attitude adjustment device that adjusts the attitude of the imaging device, and a longitudinal section measurement device that measures the boundary shape within a longitudinal section in a direction transverse to the flight path of the unmanned aerial vehicle and generates longitudinal section measurement data, and the flight control system further includes an attitude control unit that controls the attitude adjustment device so that the imaging direction of the imaging device is directed toward the outer wall surface of the structure, depending on the positional relationship between the unmanned aerial vehicle's own position estimated by the longitudinal section position estimation unit and the structure.

In one aspect of the present invention, the first predetermined shape is all or part of a circle or an ellipse, and the attitude control unit controls the attitude adjustment device so that the imaging direction of the imaging device faces the center or focus of the matching circle or ellipse.

One aspect of the present invention provides an unmanned aerial vehicle having the above-mentioned self-location estimation system.

One aspect of the present invention provides a self-position estimation method for an unmanned aerial vehicle executed by a computer, in which the unmanned aerial vehicle includes a longitudinal section measuring device that measures a boundary shape within a longitudinal section in a direction transverse to the flight path of the unmanned aerial vehicle and generates longitudinal section measurement data, the self-position estimation method including: a longitudinal section acquisition step of acquiring the longitudinal section measurement data measured by the longitudinal section measuring device when the unmanned aerial vehicle flies along an outer wall surface of a structure extending laterally along a predetermined path; and a longitudinal section position estimation step of matching the acquired longitudinal section measurement data with a first predetermined shape corresponding to the outer wall surface of the structure, and estimating the position of the unmanned aerial vehicle within the longitudinal section based on the self-position relative to the matched predetermined shape.

One aspect of the present invention provides a program for causing a computer to execute the above-mentioned self-location estimation method.

One aspect of the present invention provides a computer-readable recording medium having the above-mentioned program recorded thereon.

The present invention provides a system and method for enabling an unmanned aerial vehicle to fly autonomously to a predetermined position relative to a structure, such as a tunnel, when flying outside the structure.

1 is an external view of a multicopter, which is an example of an unmanned aerial vehicle (multicopter) according to one embodiment of the present invention. FIG. FIG. 2 is a diagram showing a flight control system for the unmanned aerial vehicle shown in FIG. 1 . 2 is a block diagram showing the configuration of an information processing unit of the unmanned aerial vehicle shown in FIG. 1 . 2 is a diagram illustrating a hardware configuration of an information processing unit of the unmanned aerial vehicle shown in FIG. 1 . FIG. 2 is a vertical cross-section of a tunnel illustrating the tunnel and flight plan path in one embodiment of the present invention. FIG. 2 is a top view illustrating a tunnel and a flight plan path in one embodiment of the present invention. 13 is a diagram for explaining a method for estimating the position of an unmanned aerial vehicle within a longitudinal section. FIG. 4 is a flowchart showing a process for autonomously flying the unmanned aerial vehicle shown in FIG. 1 inside a tunnel and photographing the inner wall surface of the tunnel. 2 is a flowchart showing in detail a flow of autonomous flight by the unmanned aerial vehicle shown in FIG. 1 .

The following describes an embodiment of the present invention with reference to the drawings. However, the present invention is not limited to the specific aspects described below, and various aspects can be taken within the scope of the technical concept of the present invention. For example, the unmanned aerial vehicle of the present invention is not limited to the multicopter shown in FIG. 1, but may be any unmanned aerial vehicle such as a rotorcraft or fixed-wing aircraft. The system configuration of the unmanned aerial vehicle 1 is also not limited to that shown in the figure, and may be any configuration as long as it is capable of similar operation. For example, the operation performed by multiple components may be performed by a single component, such as integrating the function of the communication circuit into the flight control unit, or the operation performed by a single component may be performed by multiple components, such as distributing the function of the main calculation unit to multiple calculation units. In addition, the various data stored in the memory of the unmanned aerial vehicle 1 may be stored in a different location, and the information recorded in the various memories may be stored by distributing one type of information to multiple types, or multiple types of information may be stored together in one type.

In the following explanation, an unmanned aerial vehicle autonomously flies outside a horizontally extending tunnel and captures images of the tunnel's outer wall surface. The shape of the tunnel's outer wall surface is known from blueprints, etc., and in this embodiment, the shape of the tunnel's outer wall surface in a vertical cross section (longitudinal cross section) is semicircular, with the central semicircular portion extending horizontally (laterally) in the same cross section. Note that protrusions are provided on the outer wall surface at intervals in the circumferential and depth directions.

1 is an external view of a multicopter, which is an example of an unmanned aerial vehicle (multicopter) 1 according to an embodiment of the present invention. With regard to the external appearance, the unmanned aerial vehicle 1 includes a control unit 101, six motors 102 driven by control signals from the control unit 101, six rotors (rotating wings) 103 that rotate and generate lift when driven by each of the motors 102, six arms 104 connecting the control unit 101 and each of the motors 102, and landing legs 105 that support the unmanned aerial vehicle when landing. The number of motors 102, rotors 103, and arms 104 can each be three or more, such as three or four. The six motors 102 are rotated by control signals from the control unit 101, and the rotation speed of each of the six rotors 103 is controlled to control the flight of the unmanned aerial vehicle 1, such as ascent, descent, forward/backward/left/right flight, and turning.

A pedestal 106 is attached above the control unit 101. A camera 108 for photographing an object at high resolution is attached to the pedestal 106 via an attitude adjustment device 107 that supports the camera 108 so that it can rotate horizontally and vertically.

A stand 110 is attached below the control unit 101. A longitudinal section measuring device (vertical section LIDAR) 109 is provided in front of the stand 110 to measure the shape of the surrounding space in a vertical section perpendicular to the central axis.
The unmanned aerial vehicle 1 also has a flight position sensor 111 and an antenna 117.

Figure 2 is a diagram showing the flight control system of the unmanned aerial vehicle shown in Figure 1. The flight control system 150 of the unmanned aerial vehicle 1 has a control unit 101, a motor 102 electrically connected to the control unit 101, a rotor 103 mechanically connected to the motor 102, a longitudinal section measuring device 109, a flight position sensor 111, a compass 112, an IMU 113, and an altimeter 114.

The control unit 101 is configured to process information for flight control of the unmanned aerial vehicle 1 and control the electrical signals for that purpose, and is typically a unit configured with the circuits required to realize such functions by arranging and wiring various electronic components on a board. The control unit 101 is further composed of an information processing unit 120, a communication circuit 121, a control signal generation unit 122, a speed controller 123, and an interface 125.

The camera 108 is a camera for capturing images of an object (in this embodiment, the outer wall surface of a tunnel) at high resolution. The camera 108 is attached to the base 106 via an attitude adjustment device 107 that supports the camera 108 so that it can rotate horizontally and vertically. The camera 108 acquires image data of its shooting range while the unmanned aerial vehicle 1 is flying, and the acquired images are stored in a storage device such as a memory described below. The images are typically video images consisting of a series of still images.

The attitude adjustment device 107 is configured to change the attitude of the base 106 on which the camera 108 is attached in response to a control signal from the control unit 101, thereby changing the imaging direction of the camera 108.

In this embodiment, the longitudinal section measuring device 109 is a vertical section LIDAR, and when the unmanned aerial vehicle 1 is on the ground or hovering, a pulsed laser is irradiated at a predetermined angular interval around the entire circumference of a horizontal axis extending in the longitudinal direction of the unmanned aerial vehicle 1 in a vertical cross section in the longitudinal direction of the unmanned aerial vehicle 1, and the laser reflected by an object in the surrounding space is received, thereby measuring the distance to the object in the surrounding space based on the time from irradiation to reception. By driving the longitudinal section measuring device 109 while the unmanned aerial vehicle 1 is flying in the tunnel, the longitudinal section measuring device 109 generates longitudinal section measurement data regarding the vertical cross-sectional shape (longitudinal cross-sectional shape) of the inner wall surface of the tunnel. As the longitudinal section measurement data, for example, discrete data in polar coordinates is obtained. In this embodiment, a vertical section LIDAR is used as the longitudinal section measuring device 109, but the present invention is not limited to this. For example, by capturing images of the entire circumference of a vertical section centered on the unmanned aerial vehicle using a visible light camera, integrating the captured images, and creating a 3D model, it is possible to measure the shape of the inner wall surface of the tunnel in the vertical section and estimate the vehicle's own position in the vertical section. SfM is a known algorithm for creating such a 3D model, and Dense Visual SLAM is a known algorithm for estimating the vehicle's own position using SfM.

The flight position sensor 111 is a navigation sensor, such as a GPS (Global Positioning System) sensor, that detects the coordinates of the flight position of the unmanned aerial vehicle 1. The flight position sensor 111 preferably detects three-dimensional coordinates. The coordinates acquired by the flight position sensor 111 include latitude, longitude, and altitude.
The compass 112 is a so-called compass needle, and detects the angle ahead of the unmanned aerial vehicle 1 relative to north.

IMU113 is an inertial measurement unit that detects translational motion using an acceleration sensor and rotational motion using an angular velocity sensor (gyro). Furthermore, IMU113 can calculate velocity by integrating the translational motion (acceleration) detected by the acceleration sensor, and can calculate travel distance (position) by integrating the velocity. Similarly, IMU113 can calculate angle (attitude) by integrating rotational motion (angular velocity) detected by the angular velocity sensor.

The altimeter 114 is, for example, a barometer, and estimates the altitude of the unmanned aerial vehicle based on air pressure.
The antenna 117 is an aerial for receiving radio signals including information and various data for piloting and controlling the unmanned aerial vehicle 1, and for transmitting radio signals including telemetry signals from the unmanned aerial vehicle 1.

The communication circuit 121 is an electronic circuit, typically a wireless signal processing IC, for demodulating the wireless signals received through the antenna 117 to input the control signals, control signals, and various data for the unmanned aerial vehicle 1 to the information processing unit 120, and for generating wireless signals for carrying telemetry signals output from the unmanned aerial vehicle 1. For example, the communication of the control signals and the communication of the control signals and various data may be performed by different communication circuits in different frequency bands. For example, it is possible to adopt a configuration in which communication with the transmitter of the controller (radio control) for manual operation is performed at a frequency of 950 MHz, and data communication is performed at frequencies of 2 GHz, 1.7 GHz, 1.5 GHz, and 800 MHz.

The control signal generating unit 122 is configured to convert the control command value data obtained by calculation by the information processing unit 120 into a pulse signal (such as a PWM signal) representing a voltage, and is typically an IC including an oscillator circuit and a switching circuit. The speed controller 123 is configured to convert the pulse signal from the control signal generating unit 122 into a drive voltage that drives the motor 102, and is typically a smoothing circuit and an analog amplifier. Although not shown, the unmanned aerial vehicle 1 is equipped with a power supply system including a battery device such as a lithium polymer battery or a lithium ion battery, and a power distribution system to each element.

The interface 125 is a configuration that electrically connects the information processing unit 120 to functional elements such as the camera 108, attitude adjustment device 107, longitudinal section measurement device 109, flight position sensor 111, compass 112, IMU 113, and altimeter 114 by converting the form of signals so that signals can be transmitted and received between them. For convenience of explanation, the interface is shown as one configuration in the drawings, but it is normal to use a different interface depending on the type of functional element to be connected. In addition, depending on the type of signal input/output of the functional element to be connected, the interface 125 may not be necessary. In addition, even if the information processing unit 120 in FIG. 2 is connected without the interface 125, an interface may be required depending on the type of signal input/output of the functional element to be connected.

3 is a block diagram showing the configuration of the information processing unit of the unmanned aerial vehicle shown in FIG. 1. As shown in FIG. 3, the information processing unit 120 includes a cross-sectional shape data storage unit 131, an optical vertical section position estimation unit 132, a self-position data generation unit 133, a flight path data storage unit 134, a flight control unit 135, a camera attitude calculation unit 136, an attitude control unit 137, a vertical section measurement data acquisition unit 138, and a sensor measurement data acquisition unit 139. In this embodiment, the cross-sectional shape data storage unit 131, the vertical section position estimation unit 132, the self-position data generation unit 133, the flight path data storage unit 134, the flight control unit 135, the camera attitude calculation unit 136, the attitude control unit 137, the vertical section measurement data acquisition unit 138, and the sensor measurement data acquisition unit 139 constitute a self-position estimation system 140 that generates self-position data regarding the self-position of the unmanned aerial vehicle.

Figure 4 is a diagram showing the hardware configuration of the information processing unit of the unmanned aerial vehicle shown in Figure 1. The information processing unit 120 includes a CPU 120a, a RAM 120b, a ROM 120c, an external memory 120d, an input unit 120e, an output unit 120f, and a communication unit 120g. The RAM 120b, the ROM 120c, the external memory 120d, the input unit 120e, the output unit 120f, and the communication unit 120g are connected to the CPU 120a via a bus 120h.

The CPU 120a comprehensively controls each device connected to a system bus 120h.
The ROM 120c and the external memory store the BIOS and OS, which are control programs for the CPU 120a, as well as various programs and data necessary to realize the functions executed by the computer.

The RAM 120b functions as a main memory, a working area, etc. for the CPU 120a. The CPU 120a loads programs and the like required for executing processes from the ROM 120c or the external memory 120d into the RAM 120b, and executes the loaded programs to realize various operations.
The external memory 120d is composed of, for example, a flash memory, a hard disk, a DVD-RAM, a USB memory, or the like.

The input unit 120e receives operation instructions, etc. from a user, etc. The input unit 120e is composed of input devices such as, for example, an input button, a keyboard, a pointing device, a wireless remote control, a microphone, and a camera.

The output unit 120f outputs data processed by the CPU 120a and data stored in the RAM 120b, the ROM 120c, and the external memory 120d. The output unit 120f is composed of output devices such as a CRT display, an LCD, an organic EL panel, a printer, and a speaker.

The communication unit 120g is an interface for connecting and communicating with an external device via a network or directly. The communication unit 120g is composed of interfaces such as a serial interface and a LAN interface.

Each of the parts 121, 122, 125, 131 to 139 of the self-location estimation system 140 shown in Figures 2 and 3 is realized by various programs stored in the ROM 120c and the external memory 120d using the CPU 120a, RAM 120b, ROM 120c, the external memory 120d, the input part 120e, the output part 120f, the communication part 120g, etc. as resources.

The cross-sectional shape data storage unit 131 stores cross-sectional shape data relating to the cross-sectional shape of the inner wall surface of the tunnel. Additionally, the flight path data storage unit 134 stores flight plan route data. The cross-sectional shape data storage unit 131 and the flight path data storage unit 134 are realized by reading data recorded on a recording medium such as ROM 120c or external memory 120d, and storing the data in the memory. Additionally, the cross-sectional shape data storage unit 131 and the flight path data storage unit 134 can also be omitted by incorporating the cross-sectional shape data and flight plan route data into the program.

In this embodiment, the information processing unit 120 functions as a self-location estimation system, but these systems may be installed separately from the information processing unit 120 and configured to be provided in the unmanned aerial vehicle. In addition, the self-location estimation system or its components do not need to be configured as one physical device, and may be configured as multiple physical devices. In addition, the self-location estimation system may be configured as any appropriate device such as a ground station computer, PC, smartphone, tablet terminal, etc. separate from the unmanned aerial vehicle, a cloud computing system, or a combination thereof. In addition, the functions of each part of the self-location estimation system may be configured to be executed in a distributed manner in one or more of one or more devices provided in the unmanned aerial vehicle and one or more devices separate from the unmanned aerial vehicle.

5A and 5B show a tunnel and a flight plan route in one embodiment of the present invention, with 5A being a vertical cross-sectional view of the tunnel and 5B being a top view. As shown in 5A and 5B, in this embodiment, the longitudinal cross-section of the outer wall surface of the tunnel is approximately semicircular, and a protrusion that protrudes outward is formed on the outer wall surface. Furthermore, the tunnel extends linearly with the same cross-sectional shape.

Therefore, the cross-sectional shape data includes, for example, a semicircle (center coordinates (X=0, Z=0) and radius R) represented by ^X2 + ^Z2 = ^R2 , which corresponds to the outer wall surface 202 of the tunnel. Note that since the shape of the inner wall surface of the tunnel is constant regardless of the depth direction (Y direction), Y does not exist in the above formula.

In this embodiment, the cross-sectional shape data includes data indicating a circle (semicircle) as the geometric shape corresponding to the outer wall surface of the tunnel, but is not limited to this and may be a shape composed of a convex curve such as an ellipse depending on the cross-sectional shape of the tunnel. In addition, the data representing the semicircle includes the radius R and the center coordinate, but may only include information that it is a circle. Note that the geometric shape referred to here refers to a shape that can be specified by a mathematical formula related to the X coordinate and Z coordinate, for example.

The flight plan route data is data that represents the flight plan route of the unmanned aerial vehicle 1, and is typically data of a set of a series of multiple waypoints P ₁ to P _4n that exist on the flight plan route. In this embodiment, each waypoint P ₁ to P _4n is set by a set (Y, α, d) of the Y coordinate in the depth direction of the tunnel, the counterclockwise angle α based on the upper side of the center O of the outer wall surface of the tunnel in the vertical section, and the distance d from the outer wall surface of the tunnel in the vertical section. The flight plan route is typically a set of straight lines that connect the multiple waypoints in order, but may also be a curve with a predetermined curvature within a predetermined range of the waypoints. The flight plan route data may include data that determines the flight speed at the multiple waypoints. The flight plan route data is typically used to determine the flight plan route in an autonomous flight, but can also be used as a guide during flight in a non-autonomous flight. The flight plan route data is typically input to the unmanned aerial vehicle 1 before flight and stored.

In this embodiment, the Y coordinate is not necessarily limited to a linear coordinate. In other words, if the tunnel is curved, the Y coordinate may be the distance along the path of the tunnel from the starting point toward the depth of the tunnel.

In this embodiment, as shown in Figures 5A and 5B, the flight plan route is set as follows.
- Move a predetermined distance toward the back in the tunnel extension direction (Y direction) at a predetermined position (α and d are constant) in the longitudinal section (XZ section) (waypoints P ₁ to P _n ),
- moving a certain angle counterclockwise in FIG. 5A on the concentric circle of the tunnel in the longitudinal section (XZ section) (Z and d are constant, waypoints P _n to P _n+1 );
- moving a predetermined distance in the direction of the tunnel (Y direction) from a predetermined position (α and d are constant) in the longitudinal section (XZ section) toward the hand (Z and d are constant, waypoints P _n+1 to P _2n );
- Move a certain angle counterclockwise on the concentric circle of the tunnel in the longitudinal section (XZ section) in FIG. 5A (waypoints P _2n to P _2n+1 ),
- Move a predetermined distance toward the back in the tunnel extension direction (Y direction) at a predetermined position (α and d are constant) in the longitudinal section (XZ section) (waypoints P _2n+1 to P _3n ),
- Move a certain angle counterclockwise in FIG. 5A on the concentric circle of the tunnel in the longitudinal section (XZ section) (Z and d are constant, waypoints P _3n to P _3n+1 ),
- Move a predetermined distance toward the front in the direction in which the tunnel extends (Y direction) at a predetermined position (α and d are constant) in the longitudinal section (XZ section) (waypoints P _3n+1 to P _4n )
Each of the waypoints P ₁ to P _4n is set as a position at which the outer wall surface of the tunnel is photographed by the camera 108. Also, each of the waypoints P ₁ to P _4n may be set in combination with a heading for each of the waypoints P ₁ to P _4n .

The longitudinal section measurement data acquisition unit 138 can control the longitudinal section measuring device 109 and acquires the longitudinal section measurement data acquired by the longitudinal section measuring device 109 .
The sensor measurement data acquisition unit 139 is capable of communicating with the flight position sensor 111, the compass 112, the IMU 113, and the altimeter 114, and acquires measurement values from each sensor.

The longitudinal section position estimation unit 132 estimates the longitudinal section position (α, d) of the unmanned aerial vehicle in the longitudinal section based on the longitudinal section measurement data acquired from the longitudinal section measurement device 109 and the cross-sectional shape data recorded in the cross-sectional shape data storage unit 131. FIG. 6 is a diagram for explaining a method for estimating the longitudinal section position of the unmanned aerial vehicle in the longitudinal section. As shown in FIG. 6, the longitudinal section measurement data consists of points Q1 to Qn (n=12 in the example shown in FIG. 6) represented in polar coordinates with respect to the position of the longitudinal section measurement device 109 of the unmanned aerial vehicle. In this embodiment, the longitudinal section position estimation unit 132 matches a first predetermined shape corresponding to the outer wall surface of the tunnel, which is a circle in this embodiment. Note that matching a circle does not only include matching a part of a circle, but also includes matching a part of a circle such as a semicircle or an arc.

Specifically, a circle is matched to points Q1 to Qn. As an algorithm for circle matching, the least squares method, RANSAC, etc. can be used, but RANSAC, which performs matching while excluding outliers, is suitable. When matching circles using RANSAC, the following procedure is performed.
Step 1. Randomly select three points from the discrete points Q1 to Q12.
Step 2. Derive a tentative circle model that passes through the three points determined in step 1.
Step 3. The error between the obtained provisional circular model and the discrete points Q1 to Q12 is calculated. If the error is smaller than a predetermined value, it is added to the "correct model candidates."
Step 4. Repeat steps 1-3.
Step 5. The candidate correct model that best matches the data is taken as the result of circle matching.

In step 3, if the radius of the obtained provisional circular model is farther from the radius R of the tunnel outer wall surface in the cross-sectional shape data than a predetermined value (i.e., if the difference between the radius of the provisional circular model and the radius R of the tunnel outer wall surface in the cross-sectional shape data is equal to or greater than a threshold value), it may be excluded from the "correct model candidates".

For example, in the example shown in FIG. 6, Q2 and Q7 are measurement points on the protrusion 204 of the outer wall surface 202 of the tunnel 200, and are significantly different from the trends of other discrete points among the discrete points corresponding to the outer wall surface 202 of the tunnel 200. In the example shown in FIG. 6, if Q2, Q7, and Q11 are selected in step 1, the circle model becomes C1 shown by the dashed line. In contrast, if Q1, Q5, and Q10 are selected in step 1, the circle model becomes C2 shown by the dashed line. When the errors between such circle models C1 and C2 and Q1 to Q12 are calculated, the error of the circle model C1 is larger than the error of the circle model C2. For this reason, in step 5, values that deviate from the trends of other data such as Q2 and Q7 used in setting the circle model C1 are not used as data for determining the circle, and a circle with the influence of outliers removed, such as the circle model C2, is matched.

In this embodiment, RANSAC is used as an algorithm to remove outliers and perform matching, but it is also possible to use Markov Localization, Monte-Carlo Localization, Particle Filter, Hough Transform, etc.

The longitudinal section position estimation unit 132 estimates the position (angle α and distance d) of the unmanned aerial vehicle in the longitudinal section based on the circle corresponding to the matched outer wall surface of the tunnel in this manner. Note that angle α is the angle with respect to the vertical direction based on center O of circle C2. Furthermore, d is the distance between position A of the drone and point B on circle C2 that is closest to point A, and the straight line connecting points A and B passes through the center of circle C2. For this reason, for example, if the X coordinate of position A of the unmanned aerial vehicle based on origin O of the circle matched by the longitudinal section position estimation unit 132 is XA and the Z coordinate is ZA,
Angle α=arctan(XA/-ZA)
Distance d=(XA ² +ZA ² ) ^1/2 -R
can be converted as:

The self-position data generating unit 133 generates self-position data including a three-dimensional position and attitude (heading θ) relative to a reference position. Specifically, the self-position data generating unit 133 acquires the angle α and distance d estimated by the longitudinal section position estimating unit 132. The self-position data generating unit 133 also estimates the Y coordinate position Y in the direction in which the tunnel 200 extends based on the coordinates of the flight position of the unmanned aerial vehicle 1 measured by the flight position sensor 111. In this embodiment, the self-position data generating unit 133 estimates the Y coordinate position Y based on the flight position sensor 111, but if the tunnel 200 is curved, this Y coordinate position may be the distance in the direction in which the tunnel extends at the center of the width of the tunnel 200.

The self-position data generator 133 also generates data on heading θ based on the angle measured by the compass 112. Based on these data, the self-position data generator 133 generates self-position data including three-dimensional position (Y, α, d) information, which is a set of the Y coordinate in the depth direction of the tunnel, the counterclockwise angle α with respect to a reference axis that passes through the center point O of the tunnel outer wall surface in the longitudinal section and extends upward, and the distance d from the tunnel outer wall surface in the longitudinal section, as well as information on heading θ. The reference axis only needs to pass through the center point O, and may be set to extend horizontally, for example.

In this embodiment, the self-position data generation unit 133 can estimate a reference X coordinate position _Xr , a reference Y coordinate position _Yr , and a reference heading _θr based on measurement data from the IMU 113. The self-position data generation unit 133 can also estimate a reference Z coordinate position _Zr based on measurement data from the altimeter 114. The reference coordinates _Xr , _Yr , and _Zr measured by the IMU 113 and the altimeter 114 are converted by the self-position data generation unit 133 into a reference three-dimensional position ( _Yr , _αr , _dr ), and are used together with the reference heading _θr to detect the accuracy of self-position estimation, which will be described later.

The flight control unit 135 controls the flight of the unmanned aerial vehicle to follow the flight plan route of the flight plan route data stored in the flight route data storage unit 134 based on the self-position data generated by the self-position data generation unit 133. Specifically, the flight control unit 135 determines the attitude, speed, etc. of the unmanned aerial vehicle 1 using various sensors, determines the current flight position and heading of the unmanned aerial vehicle 1 based on the self-position data generated by the self-position data generation unit 133, calculates control command values for each rotor 103 by comparing with target values such as the control signal, flight plan route (target), speed limit, and altitude limit, and outputs data indicating the control command values to the control signal generation unit 122. The control signal generation unit 122 converts the control command value into a pulse signal representing a voltage and transmits it to each speed controller 123. Each speed controller 123 converts the pulse signal into a drive voltage and applies it to each motor 102, thereby controlling the drive of each motor 102 to control the rotation speed of each rotor 103, thereby controlling the flight of the unmanned aerial vehicle 1.

Based on the self-position data of the unmanned aerial vehicle 1 estimated by the self-position data generation unit 133, the camera attitude calculation unit 136 estimates the shooting direction of the camera 108 so that the camera 108 can shoot a predetermined area of the outer wall surface 202 of the tunnel 200. Note that the shooting direction of the camera 108 here refers to the direction of the center of the angle of view of the camera 108. Specifically, in this embodiment, since the outer wall surface of the tunnel 200 is circular, the camera attitude calculation unit 136 calculates the shooting angle so that the shooting direction of the camera 108 is perpendicular to the outer wall surface 202 of the tunnel 200, and the shooting direction of the camera 108 is along the measurement surface of the longitudinal section measurement device 109 and faces the center point of the outer wall surface 202 of the tunnel 200. In this embodiment, since the cross-sectional shape of the outer wall surface 202 of the tunnel 200 is approximately circular, the camera posture calculation unit 136 calculates an angle such that the shooting direction faces the center of the circle. However, this is not limited to this, and if the outer wall surface 202 of the tunnel 200 is elliptical, the shooting angle may be calculated to face the center (the intersection of the major axis and minor axis) or the focal point.

The attitude control unit 137 calculates the rotation angle of the attitude adjustment device 107 based on the shooting angle of the camera 108 calculated by the camera attitude calculation unit 136, and transmits a control signal corresponding to the calculated rotation angle to the attitude adjustment device 107.

The flow of how the unmanned aerial vehicle described above autonomously flies inside a tunnel and photographs the inner wall surface of the tunnel will be described below.
Fig. 7 is a flow chart showing a flow of autonomously flying inside a tunnel and photographing the inner wall surface of the tunnel by the unmanned aerial vehicle shown in Fig. 1. As shown in Fig. 7, first, the information processing unit 120 refers to the flight plan route data stored in the flight route data storage unit 134, and the flight control unit 135 sets the first waypoint as a target (S100).

Next, the information processing unit 120 estimates the self-position and attitude of the unmanned aerial vehicle 1 using the self-position data generation unit 133, while the flight control unit 135 controls the number of rotations of the motor 102 based on the estimated self-position and attitude, and the unmanned aerial vehicle 1 performs autonomous flight toward the target (S110). Figure 8 is a flowchart showing in detail the flow of autonomous flight by the unmanned aerial vehicle shown in Figure 1.

As shown in Figure 8, first, the sensor measurement data acquisition unit 139 acquires the coordinates of the flight position of the unmanned aerial vehicle 1 from the flight position sensor 111, and estimates the Y-coordinate position Y based on the distance from the initial position (S111).

Next, the self-position data generation unit 133 acquires data on the heading θ of the unmanned aerial vehicle 1 from the compass 112 (S112). At this time, if the acquired heading θ deviates from the set direction, for example, does not follow the direction in which the tunnel extends, the heading may be adjusted so that the unmanned aerial vehicle 1 faces the set direction.

Next, the longitudinal section measurement data acquisition unit 138 drives the longitudinal section measurement device 109 to generate longitudinal section measurement data regarding the longitudinal section shape of the outer wall surface of the tunnel, and acquires this generated longitudinal section measurement data (S113).

Next, the longitudinal section position estimation unit 132 refers to the cross-sectional shape data recorded in the cross-sectional shape data storage unit 131, matches a circle to the longitudinal section measurement data, and estimates the position of the unmanned aerial vehicle in the longitudinal section (angle α and distance d) based on the matched circle (S114).

Next, the self-position data generation unit 133 generates self-position data including a three-dimensional position (Y, α, d) and information on the heading θ based on the Y coordinate position Y estimated in S111, the position in the longitudinal section (angle α and distance d) estimated in S112, and the heading θ acquired in S114 (S115).

Next, the sensor measurement data acquisition unit 139 acquires the measurement data of the IMU 113, and the self-position data generation unit 133 estimates a reference X coordinate position X _r , a reference Y coordinate position Y _r , and a reference heading θ _r based on the acquired measurement data (S116).

Next, the sensor measurement data acquisition unit 139 acquires measurement data relating to altitude from the altimeter 114, and the self-position data generation unit 133 estimates the reference Z coordinate position _Zr based on the measurement data (S117).

Next, the self-position data generation unit 133 converts the reference X coordinate position _Xr , the reference Y coordinate position _Yr obtained in S116, and the reference Z coordinate position _Zr obtained in S117 into a reference three-dimensional position ( _Yr , _αr , _dr ) (S118).

Then, the self-location data generating unit 133 considers the stability of the self-location data. Specifically, the three-dimensional position (Y, α, d) and heading θ included in the self-location data generated in S115 are compared with the reference heading _θr estimated in S116 and the reference three-dimensional position ( _Yr , _αr , _dr ) estimated in S118 (S119). In this embodiment, the respective values of the heading θ and the three-dimensional position (Y, α, d) are individually compared with the respective values of the reference heading _θr and the reference three-dimensional position ( _Yr , _αr , _dr ). Then, when the difference between any value of the self-location data and the reference value becomes a predetermined value or more, the self-location data generating unit 133 determines that the self-location data is unstable. Then, in S119, if it is determined that the self-position data is unstable (NO in S119), the flight control unit 135 hovers or, for example, flies in a direction away from the center of the tunnel based on the GPS position information acquired from the flight position sensor 111, and repeats S111 to S119 again.

Furthermore, if it is determined in S119 that the self-location data is stable (YES in S119), the flight control unit 135 determines whether the unmanned aircraft 1 has reached the target based on the self-location data generated by the self-location data generating unit 133 (S120).

If the flight control unit 135 determines that the unmanned aerial vehicle 1 has not reached the target (NO in S120), the flight control unit 135 calculates control command values for each rotor 103 based on the self-position data generated by the self-position data generation unit 133 so as to fly toward the target, and outputs data indicating the control command values to the control signal generation unit 122 (S121). The control signal generation unit 122 converts the control command value into a pulse signal representing a voltage and transmits it to each speed controller 123. Each speed controller 123 converts the pulse signal into a drive voltage and applies it to each motor 102, which controls the drive of each motor 102 to control the rotation speed of each rotor 103, thereby controlling the flight of the unmanned aerial vehicle 1.

In this embodiment, the waypoint is set by a set (Y, α, d) of the Y coordinate, the counterclockwise angle α based on the above the center O of the tunnel outer wall surface in the longitudinal section, and the distance d from the tunnel outer wall surface in the longitudinal section. Therefore, even if the tunnel has a shape that curves laterally or expands in the Y coordinate direction, it is possible to fly at a specified position (e.g., at a fixed distance) relative to the tunnel wall surface.

By repeating steps S111 to S121 at a predetermined short time interval, the unmanned aerial vehicle 1 flies autonomously toward the target. Note that in this embodiment, the estimation of the Y coordinate position in S111, the estimation of the position in the longitudinal section in S112 and S113, and the acquisition of information on the heading θ from the compass in S114 are performed in this order, but the order of these estimation steps may be changed, or they may be performed simultaneously.

In this way, when the unmanned aerial vehicle 1 reaches the target, and in S120 the flight control unit 135 determines that the unmanned aerial vehicle 1 has reached the target (YES in S120), the camera attitude calculation unit 136 calculates the rotation angle of the attitude adjustment device 107 so that the direction of shooting by the camera 108 is perpendicular to the outer wall surface 202 of the tunnel 200. Specifically, based on the self-position data generated by the self-position data generation unit 133, the camera attitude calculation unit 136 calculates the shooting angle along the measurement surface of the longitudinal section measurement device 109 and to the center of the tunnel 200 relative to the unmanned aerial vehicle 1 in the longitudinal section. Then, based on the shooting angle calculated by the attitude control unit 137, the rotation angle of the attitude adjustment device 107 is calculated, and a control signal according to the calculated rotation angle is transmitted to the attitude adjustment device 107 to control the attitude of the attitude adjustment device 107 (S140).

Next, the information processing unit 120 drives the camera 108 to capture an image of the tunnel inner wall surface (S150). Image data of the captured image is stored in the memory of the information processing unit 120 or recorded in an appropriate terminal via the antenna 117.

If the target reached by the unmanned aerial vehicle 1 is not the final waypoint P _4n (NO in S160), the next waypoint P is set as the target (S170), and S110 to S160 are repeated.

Furthermore, if the target reached by the unmanned aircraft 1 is the final waypoint _P4n (NO in S160), the unmanned aircraft 1 determines that all necessary processes have been completed, and the flight control unit 135 ends the operation, for example, by flying back to the initial position.

According to this embodiment, the following effects are achieved.
In this embodiment, the longitudinal section position estimation unit 132 matches a circle to the longitudinal section measurement data, and estimates the position of the unmanned aerial vehicle in the longitudinal section based on its own position relative to the matched circle. This allows the accurate position of the unmanned aerial vehicle relative to the outer wall surface 202 of the tunnel 200 to be estimated, and based on this, the flight control unit 135 can control the unmanned aerial vehicle 1 to fly at a specified position relative to the tunnel without being affected by protrusions.

In addition, in this embodiment, the longitudinal section position estimation unit 132 performs matching using a geometric shape, that is, a circle. This reduces the amount of calculation required for matching, and allows easy matching with limited memory.

In addition, in this embodiment, the longitudinal section position estimation unit 132 performs matching using circles and estimates the distance and angle based on the center of the circle. If the outer wall surface 202 of the tunnel 200 is approximately circular, the unmanned aerial vehicle 1 can be easily flown at a predetermined distance from the outer wall surface 202 of the tunnel 200 by estimating the distance and angle based on the center of the circle.

In addition, in this embodiment, the waypoints indicating the flight plan route are set to include an angle based on the center point in the longitudinal section of the tunnel and a distance from the outer wall surface. This allows the polar coordinates estimated by the longitudinal section position estimation unit 132 to be used directly for flight control, and also allows the unmanned aircraft 1 to be reliably flown a desired distance from the outer wall surface. Furthermore, even if there is no information regarding the radius of the tunnel outer wall surface, the unmanned aircraft can be flown a predetermined distance from the outer wall surface of the tunnel.

In addition, in this embodiment, the longitudinal section data is discrete data, and the longitudinal section position estimation unit 132 removes outliers in the discrete data using the RANSAC algorithm and performs matching. As a result, even if a protrusion 204 is formed on the outer wall surface 202 of the tunnel 200, the position of the tunnel outer wall surface can be accurately identified without being affected by the protrusion 204.

In this embodiment, the attitude control unit 137 controls the attitude adjustment device 107 so that the imaging direction of the camera 108 faces the center of the tunnel based on the self-position data generated by the self-position data generation unit 133. This allows the camera 108 to reliably capture images in a direction facing the center of the tunnel.

In this embodiment, the cross-sectional shape of the tunnel is described as semicircular, but the present invention is not limited to this, and the cross-sectional shape may be circular or elliptical, or a part of a circle or ellipse (arc-shaped), or even a geometric shape such as rectangular.

In addition, in this embodiment, the tunnel is described as extending horizontally in a straight line, but it may extend in a curved shape. Furthermore, in this embodiment, the tunnel is described as extending with the same cross-sectional shape, but the present invention can be applied even if the tunnel expands or contracts in diameter.

REFERENCE SIGNS LIST 1 Unmanned Aerial Vehicle 101 Control Unit 102 Motor 103 Rotor 104 Arm 105 Landing Leg 106 Base 107 Attitude Adjustment Device 108 Camera 109 Longitudinal Section Measurement Device 110 Base 111 Flight Position Sensor 112 Compass 114 Altimeter 117 Antenna 120 Information Processing Unit 120a CPU
120b RAM
120c ROM
120d External memory 120e Input unit 120f Output unit 120g Communication unit 120h System bus 121 Communication circuit 122 Control signal generation unit 123 Speed controller 125 Interface 131 Cross-sectional shape data storage unit 132 Vertical cross-sectional position estimation unit 133 Self-position data generation unit 134 Flight path data storage unit 135 Flight control unit 136 Camera attitude calculation unit 137 Attitude control unit 140 Self-position estimation system 200 Tunnel 202 Outer wall surface 204 Protrusion

Claims (12)

A self-position estimation system for an unmanned aerial vehicle that flies along an outer wall surface of a structure extending laterally along a predetermined path, comprising:
The unmanned aerial vehicle includes a longitudinal section measurement device that measures a boundary shape within a longitudinal section in a direction transverse to a flight path of the unmanned aerial vehicle and generates longitudinal section measurement data;
a longitudinal section measurement data acquisition unit that acquires the longitudinal section measurement data measured by the longitudinal section measurement device when the unmanned aerial vehicle flies along an outer wall surface of a structure extending along a predetermined path in a lateral direction;
a longitudinal section position estimation unit that matches a first predetermined shape corresponding to an outer wall surface of the structure with the acquired longitudinal section measurement data, and estimates a position of the unmanned aerial vehicle within the longitudinal section based on its own position with respect to the matched predetermined shape;
A self-position data generation unit that generates self-position data based on the position in the longitudinal section estimated by the longitudinal section position estimation unit and the coordinates of the flight position acquired by the flight position sensor;
Including,
The self-location data generating unit compares the three-dimensional position of the generated self-location data with a reference three-dimensional position obtained from an IMU and an altimeter, and determines that the self-location data is unstable when a difference between the three-dimensional position included in the self-location data and the reference three-dimensional position is equal to or greater than a predetermined value.
Self-location estimation system. the first predetermined shape corresponds at least in part to a geometric shape;
the longitudinal section position estimation unit matches the geometric shape to the longitudinal section measurement data, and estimates the position in the longitudinal section based on a self-position with respect to the matched geometric shape;
The self-location estimation system according to claim 1 . the first predetermined shape being all or part of a circle or an ellipse;
The longitudinal section position estimation unit estimates a distance based on a center point of the first predetermined shape and an angle with respect to a reference axis as a longitudinal section position of the unmanned aerial vehicle in the longitudinal section.
The self-location estimation system according to claim 2 . The longitudinal section measurement data is discrete data;
The longitudinal section position estimation unit removes outliers in the discrete data and performs matching.
The self-location estimation system according to any one of claims 1 to 3. The self-location estimation system according to any one of claims 1 to 4, further comprising a flight control unit that controls the flight of the unmanned aircraft so as to fly along a flight plan route based on the position of the unmanned aircraft in the longitudinal section estimated by the longitudinal section position estimation unit. The flight plan route is set as a plurality of waypoints,
The waypoint is set by including an angle with respect to a reference axis passing through a reference point of the structure corresponding to a center point of the first predetermined shape in the longitudinal section, and a distance from an outer wall surface of the structure at the angle.
The self-location estimation system according to claim 5 . The unmanned aerial vehicle includes:
An imaging device;
an attitude adjustment device for adjusting the attitude of the imaging device;
a longitudinal section measuring device that measures a boundary shape within a longitudinal section in a direction transverse to a flight path of the unmanned aerial vehicle and generates longitudinal section measurement data;
a flight control system;
The flight control system includes:
The self-position estimation system according to claim 1, further comprising an attitude control unit that controls the attitude adjustment device so that the imaging direction of the imaging device is directed toward an outer wall surface of the structure, in accordance with a positional relationship between the self-position estimated by the longitudinal section position estimation unit and the structure. the first predetermined shape being all or part of a circle or an ellipse;
The attitude control unit controls the attitude adjustment device so that the imaging direction of the imaging device faces the center or focus of the matching circle or ellipse.
The self-location estimation system according to claim 7 . A self-location estimation system according to any one of claims 1 to 8,
Unmanned aerial vehicle. 1. A computer-implemented method for estimating a self-location of an unmanned aerial vehicle, comprising:
The unmanned aerial vehicle includes a longitudinal section measurement device that measures a boundary shape within a longitudinal section in a direction transverse to a flight path of the unmanned aerial vehicle and generates longitudinal section measurement data;
a longitudinal section acquisition step of acquiring the longitudinal section measurement data measured by the longitudinal section measurement device when the unmanned aerial vehicle flies along an outer wall surface of a structure extending laterally along a predetermined path;
a longitudinal section position estimation step of matching a first predetermined shape corresponding to an outer wall surface of the structure to the acquired longitudinal section measurement data, and estimating a position of the unmanned aerial vehicle within the longitudinal section based on its own position relative to the matched predetermined shape;
a self-position data generating step of generating self-position data based on the position in the longitudinal section estimated in the position in the longitudinal section estimating step and the coordinates of the flight position acquired by the flight position sensor;
a comparison step of comparing a three-dimensional position of the self-location data generated in the self-location data generation step with a reference three-dimensional position obtained from an IMU and an altimeter, and determining that the self-location data is unstable when a difference between the three-dimensional position included in the self-location data and the reference three-dimensional position is equal to or greater than a predetermined value;
A self-location estimation method comprising: A program for causing a computer to execute the self-location estimation method according to claim 10. A computer-readable recording medium having the program according to claim 11 recorded thereon.

Images