JP7603272B2 - 3D map data creation program - Google Patents

Description

The present invention relates to a program for creating 3D map data.

Patent Document 1 discloses three-dimensional map data that represents the flight space in which an aircraft flies using multiple three-dimensional spaces. This three-dimensional map data includes position data that identifies the position that each three-dimensional space occupies in the flight space, and property data that is assigned to each three-dimensional space and represents the property of that three-dimensional space for the aircraft.

JP 2018-180359 A

According to the Ministry of Economy, Trade and Industry's "Roadmap for the Air Industrial Revolution 2019," flights of unmanned aerial vehicles (UAVs) beyond visual line of sight (Level 4) in populated areas are scheduled to begin in fiscal year 2022. There is a demand for three-dimensional maps to support the flights of such unmanned aerial vehicles.

The present invention aims to create a three-dimensional map database for unmanned aerial vehicle flight.

In order to achieve the above object, a 3D map data creation program is provided for creating 3D map data for unmanned aerial vehicle flight. This program causes a computer to execute the following steps: an acquisition step of acquiring a surrounding image of the moving body captured by a camera mounted on the moving body, an imaging direction of the camera, position coordinate information of the moving body at the time of capturing the image, and distance information between the moving body and an object captured in the surrounding image of the moving body; a determination step of determining a type of an object captured in the surrounding image by using a mathematical model that has been trained using an input image and the type of the object captured in the input image as teacher data; a calculation step of calculating position coordinates of the object whose type has been determined in the determination step based on the position coordinate information of the moving body, the imaging direction of the camera, and the distance information, and calculating a size of the object whose type has been determined in the determination step based on the size of the object in the surrounding image and the distance information; a creation step of creating, for each object whose type has been determined in the determination step, three-dimensional map data in which the type of the object, the position coordinates of the object, and the size of the object are combined; and a step of storing and reusing a group of the created three-dimensional map data as a three-dimensional map database.

The present invention makes it possible to create a three-dimensional map database for unmanned aerial vehicle flight.

FIG. 1 is an explanatory diagram showing a system for creating a 3D map database. FIG. 2 is an explanatory diagram showing the configuration of a moving body and a center. FIG. 2 is an explanatory diagram showing an example of a still image. FIG. 2 is an explanatory diagram illustrating an example of a functional configuration of an edge computer. 13 is a flowchart showing a startup process performed by an edge computer. 13 is a flowchart showing a 3D map database creation process performed by an edge computer. FIG. 13 is an explanatory diagram showing an example of a viewer display based on a 3D map database; FIG. 2 is an explanatory diagram illustrating an example of a functional configuration of a server. 13 is a flowchart showing a 3D map database creation process performed by the server. 13 is a flowchart showing a learning process performed by the server. 13 is a flowchart showing another learning process performed by the server. FIG. 13 is an explanatory diagram showing an example of a viewer display of a horizontal cross section based on a 3D map database; FIG. 2 is an explanatory diagram illustrating an example of a hardware configuration of a server.

The present invention will be described below based on the illustrated embodiment. However, the present invention is not limited to the embodiment described below.

Figures 1 and 2 show a system S for creating a 3D map database used for flying an unmanned aerial vehicle. This system S includes a mobile object 1 and a center (or ground station) 2. In the system S, the mobile object 1 for creating the 3D map database is not limited to an unmanned aerial vehicle 1a, but may be any mobile object such as a vehicle 1b.

The moving object 1 is equipped with a camera 11, a GNSS receiver 12, an inertial measurement unit (IMU) 13, a LiDAR device 14, and an edge computer 15. A recording medium 16 such as an SD card is connected to the edge computer 15.

The camera 11 captures video of the surroundings of the mobile object 1 while the mobile object 1 is moving or stopped. Figure 3 shows an example of a still image IM1 obtained from the video of the camera 11. The GNSS receiver 12 receives electromagnetic waves emitted by GNSS satellites such as GPS satellites. These electromagnetic waves contain data for calculating the position coordinates of the mobile object 1.

The inertial measurement unit 13 measures the angular velocity of the moving body 1 about three mutually orthogonal axes, and the acceleration of the moving body 1. The LiDAR device 14 uses a technology called LiDAR (Light Detection and Ranging, or Laser Imaging Detection and Ranging) to obtain images capturing the surroundings of the moving body 1, and measures the distance (depth) between the moving body 1 and objects captured in the images.

The video captured by the camera 11, the electromagnetic wave data received by the GNSS receiver 12, the measurement data from the inertial measurement unit 13, and the measurement data from the LiDAR device 14 are sent to the edge computer 15.

The center 2 includes a server 21 that is connected to the edge computer 15 in the mobile object 1 via a communication network, and an updated map database 22 that is connected to the server. The updated map data at that time is stored in the updated map database 22.

As shown in FIG. 4, the edge computer 15 includes a data acquisition unit 151, an object determination unit 152, a position and size calculation unit 153, and a 3D modeler (modeler within the edge computer 15) 154.

The data acquisition unit 151 acquires data sent from the camera 11, the GNSS receiver 12, the inertial measurement unit 13, and the LiDAR device 14. In addition, the data acquisition unit 151 is configured to acquire various types of data from the server 21.

The object determination unit 152 determines an object that appears in a still image obtained from a moving image captured by the camera 11, such as the still image IM1 acquired by the data acquisition unit 151, and the position of the object within the still image. To perform this determination, the object determination unit 152 uses a mathematical model such as a CNN (convolutional neural network). This mathematical model is also referred to as a first mathematical model.

The input of this mathematical model is a still image, such as still image IM1, obtained from a video captured by camera 11. The output of this mathematical model is the type of object that appears in the input still image and the position of that object within the still image.

This mathematical model is trained in advance using the input learning image, the target object shown in the learning image, and the label (type of object) and position in the image of the object as training data. In other words, the object indicated in the learning image, the label of the object, and the position in the image are annotations for the learning image. Examples of objects that can be annotated include buildings, pedestrian bridges, utility poles, pillars, roadside trees, electric wires, manholes, and mailboxes.
Objects that must be avoided in order for the drone to fly safely can be the subject of annotation. The learning flow will be described later.

Using the learned mathematical model, the object determination unit 152 determines the type of object appearing in a still image obtained from the video captured by the camera 11 and the position of the object within the still image. If the learning image has eight objects as annotations - buildings, footbridges, utility poles, pillars, roadside trees, power lines, manholes, and mailboxes - the object determination unit 152 determines the type of object appearing in the still image as one of these eight objects.

The object determination unit 152 can specify the shape of each object whose type has been determined. For each object type, the corresponding shape is defined as a table. This table is shown as a geometric shape model table 163 in the recording medium 16 in FIG. 4. The size of an object obtained by depth determination can be determined by applying a numerical value based on the shape derived from this table. In this table, the object type and the corresponding shape are defined, for example, as follows:
Buildings: Combination of one or more cubes Utility poles: Cylinders Supports: Combination of cylinders and thin rectangular parallelepipeds (plates) Roadside trees: Cones

The position and size calculation unit 153 calculates the position coordinates (latitude, longitude) of the moving body 1 based on the GNSS electromagnetic wave data acquired by the data acquisition unit 151. The position and size calculation unit 153 also calculates the imaging direction of the camera 11 based on the measurement data of the inertial measurement unit 13 acquired by the data acquisition unit 151. Furthermore, the position and size calculation unit 153 uses data from the LiDAR device 14 to calculate the distance between each object determined by the object determination unit 152 and the moving body 1. The position and size calculation unit 153 calculates the position coordinates (latitude, longitude) of each object using the position coordinates (latitude, longitude) of the moving body 1 obtained in this way, the imaging direction of the camera 11, and the distance between each object determined by the object determination unit 152 and the moving body 1.

The position and size calculation unit 153 also calculates the actual size of each object determined by the object determination unit 152. This calculation is performed based on the depth data acquired by the data acquisition unit 151 from the LiDAR device 14, the size of the object in the still image, and the shape derived from the geometric shape model table 163.

In this manner, the position and calculation unit 153 calculates the position coordinates (latitude, longitude) and actual size of each object determined by the object determination unit 152.

The 3D modeler 154 creates a relief map database (also called a simplified relief map database) using the determination results of the object determination unit 152 and the calculation results of the position and size calculation unit 153. This relief map database has a relief map data record created for each object determined by the object determination unit 152. Each relief map data record has the identification number of the corresponding object, the type of the object, the position coordinates (latitude, longitude) of the object, and the actual size of the object. Furthermore, the 3D modeler 154 records the created simplified relief map data record on the recording medium 16 as a simplified relief map database 161.

Figure 5 shows the flow of the startup process performed by the edge computer 15 in the mobile object 1. In step S101, the server 21 receives a designation of the target area of the mobile object 1 (the area in which the mobile object 1 is about to fly or drive) from the user of the system S. An example of the target area data is "a 3 km square area around Nihonbashi 3-chome, Chuo-ku, Tokyo." Furthermore, in the same step, the data acquisition unit 151 acquires the target area data of the mobile object 1 from the server 21.

In step S102, the data acquisition unit 151 acquires the latest map data in the latest map database 22 through the server 21. The initial latest map data contains only topographical information and highly accurately measured coordinate control point information obtained from outside the system S, such as a map company, or only topographical information, building information, and coordinate information obtained from outside. Topographical information is information about topography such as hills and valleys, acquired from a digital elevation model or the like. Coordinate control point information is information about the latitude and longitude of utility poles, manholes, etc., with guaranteed accuracy. Building information is information about the latitude and longitude of the building and the size of the building.

In the automatic flight of the unmanned aerial vehicle 1a, the route is usually set at the center 2 (ground station) before flight. The software of this ground station (or the software of the control system) usually has a two-dimensional map, and the flight route is defined on that map. The unmanned aerial vehicle 1a flies according to this route while obtaining its own position coordinates using GNSS. Alternatively, the flight route may be defined based on the latest map data read in step S102. The latest map data acquired in step S102 can be used when moving the mobile body 1.

By the process described below, detailed 3D map data is created from the simplified 3D map data obtained by the mobile unit 1 in the system S. This detailed 3D map data is then added to the latest map database 22.

In step S103, the data acquisition unit 151 acquires, from the server 21, parameters of a mathematical model that the object determination unit 152 uses when determining an object. Note that, regarding steps S102 and S103, one of the steps may be performed before the other step, or both steps may be performed simultaneously.

In step S104, the data acquisition unit 151 writes the latest map data acquired in step S102 and the parameters acquired in step S103 to the recording medium 16.

Figure 6 shows the flow of the process for creating a simplified 3D map database performed by the edge computer 15 in the mobile object 1. This creation process is performed following the startup process shown in Figure 5. In step S201, the data acquisition unit 151 acquires data. Specifically, the data acquisition unit 151 acquires still images from the video captured by the camera 11, and acquires data of received electromagnetic waves from the GNSS receiver 12. The data acquisition unit 151 further acquires measurement data from the inertial measurement unit 13, and acquires measurement data using LiDAR technology from the LiDAR device 14.

In step S202, the object determination unit 152 uses the mathematical model to determine the type of object appearing in the still image acquired in step S201 and the position of the object within the image. The details of this determination are as described above.

In step S203, the position and size calculation unit 153 calculates the position coordinates (latitude, longitude) and actual size of each object determined by the object determination unit 152. Details of this calculation are as described above.

In step S204, the 3D modeler 154 creates a simplified version of the three-dimensional map data.

The server 21 generates a distance estimation model (distance estimation AI) that estimates the distance between the object and the moving body 1 based on the type of object and the size of the object in the still image. This distance estimation model is also a mathematical model such as a convolutional neural network. The input of the distance estimation model is an image of the surroundings of the moving body, and the output is the type of object shown in the image of the surroundings, the position of the object in the image, and the distance to the object. In a moving body that is not equipped with a LiDAR device 14, the distance is estimated using this distance estimation model. Details will be described later. The distance estimation model is also called a second mathematical model.

In step S205, the 3D modeler 154 records the simplified 3D map data as a simplified 3D map database 161 on the recording medium 16. This post-arrangement data is used for drone collision avoidance and for calculating parameters of a distance estimation model (distance estimation AI) in the server 21. In the same step, the data acquisition unit 151 also records video data from the camera 11 on the recording medium 16 as a video database 162.

In step S206, the 3D modeler 154 sends the simplified 3D map database to the server 21, and the data acquisition unit 151 sends the video database to the server 21. Note that step S205 is not essential. For example, if the capacity of the communication network between the mobile object 1 and the center 2 is limited, step S206 may not be performed.

Regarding steps S205 and S206, one of the steps may be performed before the other, or both steps may be performed simultaneously.

Image IM2 is shown in Figure 7. For ease of understanding, this image shows the map after objects (i.e., simple models) approximated by cubes, cones, etc. have been placed on it. This image IM2 can be rendered on a display device by the viewer through normal perspective transformation according to the evaluator's position, line of sight, and projection table distance.

The area enclosed by the dashed line in image IM2 indicates the space between 5 meters and 150 meters above ground, which is the main flight space of the unmanned aerial vehicle.

Figure 8 shows an example of the functional configuration of the server 21. The server 21 includes a data acquisition unit 211, a first correction processing unit 212, a second correction processing unit 213, and a parameter calculation unit 214. As shown in Figure 1, the latest map database 22 is connected to the server 21, and the main structure database 23, the simplified 3D map database 161, and the video database 162 are also connected to the server 21. The simplified 3D map database 161 and the video database 162 are either sent from the mobile unit 1 to the server 21 in step S206 in Figure 6, or are acquired from the recording medium 16 after recording is completed in step S205 in the same figure.

The major structure database 23 stores the position coordinates and size of major structures whose latitude and longitude have already been measured and determined as coordinates. Examples of major structures include existing buildings, triangulation points, manholes, and utility poles.

Figure 9 shows the flow of the process of creating a detailed 3D map database performed by the server 21. In step S301, the data acquisition unit 211 reads out records in the main structure database 23, records in the simplified 3D map database 161, and video data in the video database 162.

In step S302, the first correction processing unit 212 determines whether the simplified 3D map database 161 read in step S301 contains a record relating to a structure that is the same as the structure (as a reference point) stored in the main structure database 23. If the result of this determination is positive, step S311 is performed next, and if not, step S321 is performed.

In step S311, the first correction processing unit 212 rewrites the position coordinate information (latitude, longitude) of the record in the simplified 3D map database 161 found in step S302 to the corresponding position coordinate information (latitude, longitude) in the main structure database 23. This rewriting process is the first correction process. In the same step, the first correction processing unit 212 also writes the record after the first correction process to the latest map database 22.

In step S312, the first correction processing unit 212 writes a history of the correction processing performed in step S311 to a specified log file. This log file can be stored, for example, in a storage device in the server 21. After step S312, the processing ends.

Normally, the position coordinate information in the main structure database 23 is based on a reference point measured and specified by the Ministry of Land, Infrastructure, Transport and Tourism or on highly accurate coordinates actually measured from that reference point; these highly accurate coordinates are referred to here as confirmed coordinates. In contrast, the position coordinate information in the simplified 3D map database 161 is based on electromagnetic wave data emitted by GNSS satellites, and therefore the accuracy is not guaranteed. Therefore, in step S311, a correction process is performed to replace the position coordinate information in the simplified 3D map database 161 with coordinates in the main structure database 23.

However, there may be cases where the position coordinate information in the main structure database 23 has a larger error than the position coordinate information in the simplified 3D map database 161. For example, this may occur when data in the main structure database 23 is acquired before a disaster occurs, and a record in the simplified 3D map database is created after the disaster occurs. In preparation for such cases, the log file that records the coordinate corrections created in step S312 can be used to retrospectively verify whether the correction process in step S311 was appropriate.

In step S321, the second correction processing unit 213 determines whether triangulation is possible for objects in the simplified 3D map database 161 that are not major structures. If at least two major structures required for triangulation of an object that is not a major structure are shown in the image in which the object appears, the second correction processing unit 213 determines that surveying is possible, otherwise it determines that surveying is impossible. If it is determined that surveying is possible, step S322 is performed next, and if it is determined that surveying is impossible, step S331 is performed.

In step S322, the second correction processing unit 213 performs triangulation of an object that is not a main structure from an image that includes the object and at least two of the main structures, and obtains the position coordinates of the object. The position coordinates calculated by triangulation from the two confirmed coordinates are recognized as confirmed coordinates.

In step S323, the second correction processing unit 213 rewrites the coordinate information in the simplified 3D map database 161 of the object that is not a major structure to the coordinates acquired in step S322, and at the same time stores this target object as a major structure in the major structure database together with its new confirmed coordinates. This operation gradually increases the number of major structures, and the accuracy of the 3D map improves as the main structure database increases. This rewriting process is also called the second correction process. After that, the process ends.

In step S331, the second correction processing unit 213 writes the uncorrected coordinate information for the object that is not a major structure into the latest map database 22 as calculated coordinates rather than confirmed coordinates. Then, the process ends.

Steps S311 and S312 can be performed for each record in the simplified 3D map database 161 that is determined to be a major structure in step S302. Also, step S321 and the subsequent steps can be performed for each record in the simplified 3D map database 161 that is determined not to be a major structure in step S302.

In this way, with regard to location coordinate data, in addition to location coordinate information from GNSS satellites or highly accurate location calculations using RTK (Real Time Kinematics), the location accuracy is adjusted using buildings, utility poles, manholes, and other ground reference points with fixed coordinates, thereby improving the accuracy of the map data on the server.
Buildings, utility poles, manholes, etc. that serve as ground reference points can be surveyed in advance and a set of their shape and confirmed coordinates can be stored as known data in a reference point data table (major structure database), or a system (beacon) can be created that transmits information about the reference point (the identifier and confirmed coordinates of the reference point) from the reference point via radio waves, etc.
When creating a map or when a mobile object moves, the shape of the reference point is recognized from the camera image and matched with the reference point data table, or an identifier is acquired from the received radio wave and matched with the reference point data, or if the reference point transmits an identifier and location information, it can be used as the location data as it is. The ground reference point group is designed to cover the area efficiently (for example, a mesh structure with one point every 1 km is created), and by measuring it separately, it can be used as a useful base information for unmanned flight and unmanned driving (making it possible to fly in places where GNSS radio waves cannot reach). The location data of the reference point is treated as a confirmed coordinate. The confirmed coordinates are multiplied by step S323.
When a new ground control point is installed on a map and more accurate position information is obtained compared to GNSS data, the server corrects the map data up to the surrounding control points. The position data obtained using the data obtained from the ground control point is stored on the map on the server as confirmed coordinates, and in principle is not corrected thereafter.

To explain again, a definite coordinate refers to position information measured by ground surveying, or coordinates calculated from two or more definite coordinates.
The calculated coordinates are coordinates calculated using one confirmed coordinate and depth-added point cloud data or GNSS, or coordinates calculated using GNSS and depth-added point cloud data. The calculated coordinates have lower accuracy than the confirmed coordinates. The calculated coordinates may be promoted to confirmed coordinates due to an increase in the confirmed coordinate points (step S323).
The accuracy of the final data may be impaired due to ground movements, disasters, etc., so an update mode (step S311) is provided to allow the data to be reset. The resetting can be performed for each region.

Figure 10 shows the process performed by the server 21 to generate parameters (weighting coefficients) of the mathematical model used by the object determination unit 152 to make a determination. In step S401, the data acquisition unit 211 acquires training data. As described above, this training data includes a training image, an object that appears in the training image, and the position of the object within the training image.

In step S402, the parameter calculation unit 214 calculates the parameters of the mathematical model based on the teacher data. In other words, a learned first mathematical model is constructed.

Figure 11 shows the flow of the parameter generation process for the distance estimation model (second mathematical model) performed by the server 21. In step S501, the data acquisition unit 211 acquires a learning image, an object shown in the learning image, and the position of the object within the learning image, similar to step S401.

In step S502, the data acquisition unit 211 acquires data (LiDAR data or depth data) regarding the distance between an object shown in the learning image in step S501 and the point where the image was captured.
Regarding steps S501 and S502, one of the steps may be performed before the other step, or both steps may be performed simultaneously.

In step S503, the parameter calculation unit 214 calculates parameters of a distance estimation model (second mathematical model) using the data acquired in step S501 and the distance data acquired in step S502 as training data. In other words, a learned second mathematical model is constructed.

Unlike the mobile body 1, a mobile body that does not have a LiDAR device 14 can use this second mathematical model to obtain the type of object shown in the input image, the position of the object in the image, and the distance to the object. In other words, even a mobile body that does not have a LiDAR device 14 can perform the process of creating the simplified 3D map database shown in Figure 6.

In addition, a horizontal cross-sectional image at a specified altitude can be created from the latest map database 22. As shown in FIG. 12, a horizontal cross-sectional image IM11 at a first altitude X and a horizontal cross-sectional image IM12 at a second altitude Y can be created from the latest map database 22 by a viewer installed on the server 21. The server 21 can create these cross-sectional images by receiving input of the specified altitude from the user.

Figure 13 shows an example of the hardware configuration of the server 21. The server 21 includes a processor 21a, an interface device 21b, a display device 21c, an input device 21d, a drive device 21e, an auxiliary storage device 21f, and a memory device 21g, which are interconnected by a bus 21h.

The processor 21a includes a CPU and a GPU. The GPU is capable of highly parallel processing.

The program that realizes the functions of the server 21 is provided by a recording medium 21i such as a CD-ROM. When the recording medium 21i on which the program is recorded is set in the drive device 21e, the program is installed from the recording medium 21i via the drive device 21e into the auxiliary storage device 21f. Alternatively, the program does not necessarily have to be installed from the recording medium 21i, but can also be installed via a network. The auxiliary storage device 21f stores the installed program as well as necessary files, data, etc.

When an instruction to start a program is received, the memory device 21g reads out and stores the program from the auxiliary storage device 21f. The processor 21a realizes the functions of the server 21 according to the program stored in the memory device 21g. The interface device 21b is used as an interface for connecting to other computers via a network. The display device 21c displays a GUI (Graphical User Interface) or the like according to the program. The input device 21d is a keyboard, a mouse, etc.

The edge computer 15 also has a hardware configuration similar to that of the server 21.

The embodiments described so far have aspects as an apparatus, as a method, and as a computer program.

According to the embodiment described with reference to Figures 1 to 7 and 10, a three-dimensional map database for unmanned aerial vehicle flight can be efficiently created using a mathematical model based on machine learning. The main space in which unmanned aerial vehicles fly is the space between 5 meters and 150 meters above the ground. A three-dimensional map database with a certain level of accuracy can be created to avoid collisions with objects such as buildings that exist in this space and fly safely.

When flying an unmanned aerial vehicle, it is necessary to recognize objects such as buildings in the airspace. However, it is not necessary to recognize the patterns on the objects or windows and other features. Furthermore, with the exception of takeoff and landing points, there is no need to recognize the detailed unevenness of objects in the airspace. In light of this situation, it is possible to increase the judgment speed by cutting out unnecessary parts, and perform object recognition that focuses on improving real-time performance.

The processes shown in Figures 8 and 9 can improve the accuracy of the three-dimensional map database created by the process shown in Figure 6. This contributes to safe flight of unmanned aerial vehicles. Depending on the performance of the edge computer 15, the process of Figure 9 may be performed by the edge computer 15 instead of the server 21.

The mathematical model created by the process shown in Figure 11 makes it possible to efficiently create a 3D map database even for a moving object that is not equipped with a relatively expensive LiDAR device.

In addition, manholes, as ground reference points, act as nodes between map information of underground structures and aboveground maps. This makes it possible to integrate underground maps and aboveground maps.

Regarding the embodiments described above, the following supplementary notes are disclosed.
[Appendix 1]
A 3D map data creation program for creating 3D map data for an unmanned aerial vehicle flight, comprising:
an acquisition step of acquiring an image of the surroundings of the moving body captured by a camera mounted on the moving body, an imaging direction of the camera, position coordinate information of the moving body at the time of the imaging, and distance information between the moving body and an object captured in the image of the surroundings of the moving body;
a determination step of determining the type of object shown in the surrounding image and the in-image position of the object shown in the surrounding image by using a mathematical model that has been trained using an input image, the type of object shown in the input image, and the in-image position of the object shown in the input image as training data;
a calculation step of calculating position coordinates of the object, the type of which has been determined in the determination step, based on position coordinate information of the moving body, the imaging direction of the camera, and the distance information, and calculating a size of the object, the type of which has been determined in the determination step, based on a size of the object in the surrounding image and the distance information;
a creating step of creating three-dimensional map data in which the type of each object determined in the determining step is combined with the type of the object, the position coordinates of the object, and the size of the object;
and a step of storing the created relief map data group as a relief map database for reuse.
[Appendix 2]
In the calculation step, a size of the object whose type has been determined in the determination step is calculated based on a shape of the object,
The shape of the object is derived from a geometric shape model table in which a relationship between the type of the object and the shape of the object is defined.
2. A 3D map data generating program according to claim 1.
[Appendix 3]
a first determination step of determining whether or not to rewrite coordinate information of an object in the relief map data to the determined coordinates of a structure to be compared with an object in the relief map database, based on relief map data generated by the relief map data generation program according to appendix 1 or 2 and known determined coordinates stored in a coordinate reference point database of the structure to be compared with the object in the relief map database;
a first correction step of rewriting coordinate information of the object in the 3D map data to the confirmed coordinates of the structure when a result of the determination in the first determination step is affirmative;
a history creating step of creating a history of the execution of the correction step;
a second judgment step of judging whether or not a survey of position coordinates of an object in the 3D map data can be performed based on the determined coordinates of the structure when the judgment result of the first judgment step is negative;
a surveying step of performing the survey when the judgment result of the second judgment step is positive;
a second correction step of rewriting coordinate information of an object in the three-dimensional map data to coordinate information obtained in the surveying step;
and adding a combination of the object and the coordinate information obtained by the surveying step as a new confirmed coordinate to a coordinate reference point database.
[Appendix 4]
A first acquisition step of acquiring an image, a type of an object appearing in the image, and a position of the object appearing in the image;
A second acquisition step of acquiring a distance from a shooting point of the image to an object shown in the image;
a generation step of generating parameters of a mathematical model that outputs, when an input image is input, the type of object appearing in the input image, the position of the object in the input image, and the distance to the object, using the image, the type of object, the position of the object in the image, and the distance as training data.

Although the embodiment of the present invention has been described above, the present invention is not limited to the embodiment described above, and various modifications and changes are possible based on the technical concept of the present invention.

S System 1 Mobile unit 1a Unmanned aerial vehicle 1b Vehicle 2 Center

11 Camera 12 GNSS receiver 13 Inertial measurement unit (IMU)
14 LiDAR device 15 Edge computer 16 Recording medium

2 Center 21 Server 22 Database

151 Data acquisition unit 152 Object determination unit 153 Position and size calculation unit 154 3D modeler

211 Data acquisition unit 212 First correction processing unit 213 Second correction processing unit 214 Parameter calculation unit

Claims (4)

A three-dimensional map data creation program for creating three-dimensional map data for an unmanned aerial vehicle flight, comprising:
an acquisition step of acquiring an image of the surroundings of the moving body captured by a camera mounted on the moving body, an imaging direction of the camera, position coordinate information of the moving body at the time of the imaging, and distance information between the moving body and an object captured in the image of the surroundings of the moving body;
a determination step of determining a type of object in the surrounding image by using a mathematical model that has been trained using an input image and a type of object in the input image as training data;
a calculation step of calculating position coordinates of the object, the type of which has been determined in the determination step, based on position coordinate information of the moving body, the imaging direction of the camera, and the distance information, and calculating a size of the object, the type of which has been determined in the determination step, based on a size of the object in the surrounding image and the distance information;
a creating step of creating three-dimensional map data in which the type of each object determined in the determining step is combined with the type of the object, the position coordinates of the object, and the size of the object;
and a step of storing the created relief map data group as a relief map database for reuse. In the calculation step, a size of the object whose type has been determined in the determination step is calculated based on a shape of the object,
The shape of the object is derived from a geometric model table in which a relationship between the type of the object and the shape of the object is defined.
2. The 3D map data generating program according to claim 1. a first determination step of determining whether or not to rewrite coordinate information of an object in the relief map data to the determined coordinates of a structure to be compared with an object in the relief map database, based on the relief map data generated by the relief map data generation program according to claim 1 or 2 and known determined coordinates stored in a coordinate reference point database of the structure to be compared with the object in the relief map database;
a first correction step of rewriting coordinate information of the object in the 3D map data to the confirmed coordinates of the structure when a result of the determination in the first determination step is affirmative;
a history creating step of creating a history of the execution of the correction step;
a second judgment step of judging whether or not a survey of position coordinates of an object in the 3D map data can be performed based on the determined coordinates of the structure when the judgment result of the first judgment step is negative;
a surveying step of performing the survey when the judgment result of the second judgment step is positive;
a second correction step of rewriting coordinate information of an object in the three-dimensional map data to coordinate information obtained in the surveying step;
3. The 3D map data updating method according to claim 1, further comprising the step of: adding a combination of the object and the coordinate information obtained in said surveying step as a new confirmed coordinate to a coordinate reference point database. A first acquisition step of acquiring an image, a type of an object appearing in the image, and a position of the object appearing in the image;
A second acquisition step of acquiring a distance from a shooting point of the image to an object shown in the image;
a generation step of generating parameters of a mathematical model that outputs, when an input image is input, the type of object appearing in the input image, the position of the object in the input image, and the distance to the object, using the image, the type of object, the position of the object in the image, and the distance as training data.