JP6737751B2 - Flight device, management device, flight management method and program - Google Patents

Description

The present invention relates to a flight device, a management device, a flight management method, and a program.

In a flight device such as a drone, a user generally sets a flight route by designating a way point on a map. Among such techniques, there is a technique in which an obstacle that is not reflected on a map is imaged by a drone and a user specifies a flight route using the captured image (for example, Patent Document 1).

Patent No. 6138326

However, in the method described in Patent Document 1, the obstacle information and the obstacle avoidance method are not managed in association with each other, and flight management is insufficient.

Then, this invention is made|formed in view of the said problem, and it aims at providing the technique for managing the avoidance method of an obstacle.

A flight device according to a first aspect of the present invention includes a position information acquisition unit that acquires flight position information indicating a flight position, a detection unit that detects an obstacle in a predetermined area including a first flight route, and the obstacle. A captured image acquisition unit that acquires a captured image of an object, and the flight position information, when flying a second flight route generated based on the flight position information and the captured image to avoid the obstacle, A storage unit that stores a captured image, a capturing direction of the captured image, and the second flight route.

The flight device may further include a generation unit that generates the second flight route based on the flight position information and the captured image. The first flight route may be generated based on map information that does not include information about the obstacle.

The storage unit further includes a distance measuring unit that measures a distance from the flight position to the obstacle, and the storage unit, when flying the second flight route, distance information indicating a distance from the flight position to the obstacle. May be stored. The storage unit may store information about the object when the detection unit detects an object outside the predetermined area, which is not shown in the map information.

The flight device further includes a transmitter that transmits the flight position information, the captured image, a direction of capturing the captured image, and the second flight route to a management device when flying on the second flight route. Good.

A flight device according to a second aspect of the present invention includes a position information acquisition unit that acquires flight position information indicating a flight position, a storage unit that stores map information indicating the position of an object, and the map information. A detection unit that detects the presence or absence of an object that is present, and the storage unit detects the presence of the object when the detection unit detects that the object indicated in the map information does not exist during flight. Remember what you don't.

The management device according to the third aspect of the present invention is directed to flight position information indicating a flight position of a flight device, a predetermined area including a position on a first flight route imaged from the flight device which is flying a first flight route. A captured image of an obstacle, and a management information acquisition unit that acquires an image capturing direction of the captured image, and a second flight route generated to avoid the obstacle based on the flight position information and the captured image. And a storage unit that stores the flight position information, the captured image, and an imaging direction of the captured image.

The first flight route may be generated based on map information that does not include information about the obstacle.

The management information acquisition unit further acquires an image capture date and time of the captured image, a storage unit that stores map information, the flight position information at the time of capturing the captured image, a capturing direction of the captured image, and Based on the distance from the flight position to the obstacle, a specifying unit that specifies the position of the obstacle captured in the captured image, and the map information corresponding to the position of the obstacle specified by the specifying unit. It may further include a map display unit that displays the icon of the obstacle at a position on the map, and associates the captured image and the captured date and time of the obstacle with the icon of the obstacle.

The specifying unit specifies the size of the obstacle based on the captured image and the distance from the flight position to the obstacle at the time of capturing the captured image, and the map display unit specifies the specifying unit. The size of the obstacle may be associated with the obstacle icon.

The management device further includes a schedule acquisition unit that acquires a flight schedule of another flight device, and the obstacle is compared with a device that manages a flight device that is scheduled to fly in an area within a predetermined distance from the obstacle. And a notification unit for notifying the information about the second flight route or the second flight route.

A flight management method according to a fourth aspect of the present invention is directed to a predetermined area including flight position information indicating a flight position of a flight device, the first flight route imaged from the flight device flying a first flight route. A captured image of the obstacle and a direction of capturing the captured image; and a second flight route generated to avoid the obstacle based on the flight position information and the captured image, Storing the flight position information, the captured image, and an imaging direction of the captured image.

A program according to a fifth aspect of the present invention causes a computer to include predetermined flight position information indicating a flight position of a flight device, and a first flight route imaged from the flight device that is flying a first flight route. A step of acquiring a captured image of an obstacle in a region and a direction of capturing the captured image; and a second flight route generated to avoid the obstacle based on the flight position information and the captured image. A step of storing the flight position information, the captured image, and the capturing direction of the captured image.

According to the present invention, it is possible to manage an obstacle avoidance method.

It is a figure which shows the structure of the map management system which concerns on embodiment. It is a figure which shows the structure of a flying device. It is a figure which shows the structure of a management apparatus. The example of the display image of a 3D map which a map display part makes a display device display is shown. It is the flowchart which showed an example of operation|movement of a flying device. 6 is a flowchart showing an example of the operation of the management device.

[Configuration of map management system S]
FIG. 1 is a diagram showing a configuration of a map management system S according to this embodiment. The map management system S manages a 3D map indicating information on obstacles in the airspace in which the flying device 100 flies. This 3D map includes information indicating altitude.

The map management system S includes a management device 200. The flying device 100 can acquire captured images during flight, and is, for example, a drone. The flight device 100 communicates with the management device 200 by wireless communication to receive from the management device 200 first route information indicating a first flight route for the flight device 100 to fly from the flight start position to the flight end position. .. In FIG. 1, the first flight route is indicated by a broken line. The flying device 100 moves along the first flight route received from the management device 200.

When the obstacle A is detected on the first flight route, the flight device 100 generates a second flight route for avoiding the obstacle A and moves along the generated second flight route. .. In FIG. 1, the second flight route is indicated by a solid line. The second flight route is a route after branching from the first flight route to avoid the obstacle A and then joining the original first flight route. The obstacle A is, for example, a lightning rod or a building under construction, and in the example of FIG. 1, is a lightning rod installed on the roof of the building. When the flight device 100 moves along the second flight route, the flight device 100 acquires a captured image of the obstacle A and transmits the acquired captured image to the management device 200.

The management device 200 stores map information indicating a 3D map. The management device 200 refers to the map information and generates the first flight route so that the flying device 100 in flight does not come into contact with each obstacle recorded in the map information. In addition, the management device 200 receives a captured image of the obstacle A from the flight device 100 when the flight device 100 moves on the second flight route.

The management device 200 generates the first flight route so that the flying device 100 in flight does not come into contact with the obstacle recorded in the map information. However, in the example of the present embodiment, the first flight route is generated based on the map information that does not include the information about the obstacle A due to the situation that the obstacle A is installed after the map information is created. Be present. The management device 200 stores information on the obstacle A detected by the flying device 100.

[Configuration of Flight Device 100]
FIG. 2 is a diagram showing the configuration of the flying device 100. The flying device 100 includes a GPS receiving unit 11, a detecting unit 12, a distance measuring unit 13, an image capturing unit 14, a communication unit 15, a storage unit 16 and a control unit 17. The GPS receiver 11 receives signals from a plurality of GPS satellites.

The detection unit 12 detects the obstacle A in a predetermined area including the first flight route. The predetermined area is a permissible value in consideration of a measurement error of positioning by the GPS receiving unit 11, and is, for example, a range within 20 meters from the first flight route. The obstacle A is an object on the first flight route or within a predetermined range from the first flight route. The obstacle A is an object that is expected to come into contact with the flying device 100 when the flying device 100 flies on the first flight route, or a communication obstacle to the flying device 100 when the flying device 100 approaches the flying device 100. It may be an object which is expected to cause the obstacle.

The detection unit 12 is, for example, a stereo camera. The stereo camera is activated continuously or intermittently during the flight on the first flight route, acquires a plurality of images of a subject on the first flight route taken simultaneously from a plurality of different directions, and acquires the plurality of acquired images. The obstacle A is detected by the matching of the images. The closer the object is from the flying device 100, the larger the parallax when the images are taken from a plurality of different directions. On the other hand, parallax does not occur when an object does not exist at a predetermined position which is an imaging target. Therefore, the stereo camera detects as an obstacle A an object whose parallax is equal to or greater than a certain value when images are taken from a plurality of different directions in the traveling direction of the flying device 100. The constant value is, for example, the minimum value of parallax that can be detected by the stereo camera. Further, the detection unit 12 may be an ultrasonic sensor or an infrared distance measuring sensor. The ultrasonic sensor detects the obstacle A by transmitting ultrasonic waves to a target and receiving the reflected waves with a receiver. The infrared distance measuring sensor detects the obstacle A by irradiating the object with infrared light and receiving the reflected light.

The detection unit 12 also acquires detection information for specifying the position and size of the obstacle A in the horizontal direction, the vertical direction, and the depth direction. For example, when the detection unit 12 is a stereo camera, the detection unit 12 acquires a plurality of images obtained by simultaneously capturing an object from a plurality of different directions and divides the acquired images into a large number of small sections. The detection unit 12 obtains the parallax by matching a plurality of images for each small section. Furthermore, the detection unit 12 specifies the position and size of the obstacle A in the horizontal direction, the vertical direction, and the depth direction by calculating the distance to the obstacle A for each small section based on the obtained parallax. The detection unit 12 notifies the recognition unit 174 of the presence/absence of detection of the obstacle A and the detection information.

The distance measuring unit 13 measures the distance from the flight position of the flying device 100 to the obstacle A. The distance measuring unit 13 is, for example, an ultrasonic sensor, emits ultrasonic waves toward the obstacle A, and receives the reflected wave to measure the distance from the flight position to the obstacle A. The distance measuring unit 13 may be a stereo camera or an infrared distance measuring sensor. Further, the detection unit 12 and the distance measurement unit 13 may be the same sensor.

The image capturing unit 14 is a camera for capturing a captured image with an optical element. The communication unit 15 is a communication interface for wireless communication. The storage unit 16 is a storage medium such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The storage unit 16 may be a volatile memory or a non-volatile memory. The storage unit 16 stores a program executed by the control unit 17.

The control unit 17 is, for example, a CPU (Central Processing Unit). The control unit 17 functions as the position information acquisition unit 171, the flight control unit 172, the generation unit 173, the recognition unit 174, the captured image acquisition unit 175, and the communication control unit 176 by executing the program stored in the storage unit 16. To do.

The captured image acquisition unit 175 acquires a captured image by controlling the image capturing unit 14. The captured image acquisition unit 175 acquires a captured image of the obstacle A when the detection unit 12 detects the obstacle A. The captured image acquisition unit 175 also acquires the capturing date and time and the capturing direction of the captured image of the obstacle A.

The position information acquisition unit 171 acquires flight position information indicating the flight position of the flying device 100 based on the signal from the GPS satellite received by the GPS reception unit 11. Further, the position information acquisition unit 171 acquires flight altitude information indicating the flight altitude of the flying device 100 by using a stereo camera, an ultrasonic sensor, a barometer, etc. installed below the flying device 100. For example, the position information acquisition unit 171 acquires flight position information and flight altitude information when the captured image acquisition unit 175 acquires the captured image.

The recognition unit 174 analyzes the detection information acquired by the detection unit 12, the distance information from the flight position acquired by the distance measurement unit 13 to the obstacle A, and the captured image acquired by the captured image acquisition unit 175. The position and size of the obstacle A in the horizontal direction, the vertical direction, and the depth direction are recognized. For example, the recognition unit 174 recognizes the position and size of the obstacle A in the depth direction based on the detection information acquired by the detection unit 12. The recognition unit 174 also recognizes the horizontal position and size of the obstacle A and the vertical position and size of the obstacle A based on the captured image acquired by the captured image acquisition unit 175.

The generation unit 173 generates a second flight route for avoiding the obstacle A. When the detection unit 12 detects the obstacle A, the generation unit 173 refers to the horizontal position, the vertical direction, and the depth direction position and size of the obstacle A recognized by the recognition unit 174, thereby performing the second flight. Generate a route.

For example, the generation unit 173 generates a second flight route that turns around the obstacle A so that the flight device 100 keeps a certain distance or more from the obstacle A. The fixed distance is, for example, 2 meters. The generation unit 173 refers to the first flight route and the current flight position, and after diverging from the first flight route to avoid the obstacle A, the second flight route until the vehicle rejoins the first flight route. To generate. The generation unit 173 may generate a second flight route that avoids the obstacle A so that the flight distance becomes the shortest while keeping a certain distance or more from the obstacle A.

The flight control unit 172 controls flight. For example, the flight control unit 172 specifies the moving direction and the moving speed of the flying device 100 by specifying the rotational speeds of a plurality of rotors (not shown), respectively. The flight control unit 172 acquires the first route information indicating the first flight route from the management device 200 by the communication unit 15. The flight control unit 172 controls the flight to move along the first flight route indicated by the acquired first route information.

The flight control unit 172 specifies the moving direction corresponding to the first flight route by comparing the acquired first route information with the flight position information acquired by the position information acquisition unit 171. In addition, when the detection unit 12 detects the obstacle A on the first flight route, the flight control unit 172 hovers, and after the generation unit 173 generates the second flight route, the flight control unit 172 follows the second flight route. The flight control to move is performed. The flight control unit 172 performs flight control to move along the first flight route again after the second flight route merges with the first flight route.

The communication control unit 176 communicates with the management device 200 via the communication unit 15. When the flight device 100 flies on the second flight route, the communication control unit 176 (1) a device ID for identifying the flight device 100, and (2) flight of the flight device 100 measured by the distance measurement unit 13. Distance information indicating the distance from the position to the obstacle A, (3) captured image of the obstacle A acquired by the image capturing unit 14 and captured date/time information indicating the captured date/time of the captured image, and (4) position information acquisition unit Flight position information and flight altitude information acquired at the time of acquiring the captured image in 171, (5) Direction information indicating the capturing direction of the captured image, and (6) Horizontal direction and vertical direction of the obstacle A recognized by the recognition unit 174. The three-dimensional information indicating the position and size in the direction and the depth direction is stored in the storage unit 16 and transmitted to the management device 200. In addition, the communication control unit 176 transmits (7) second route information indicating the second flight route to the management device 200 after joining the first flight route from the second flight route.

[Configuration of Management Device 200]
FIG. 3 is a diagram showing the configuration of the management device 200. The management device 200 includes a communication unit 21, a storage unit 22, and a control unit 23. The communication unit 21 is a communication interface for transmitting and receiving various kinds of information via the network. The storage unit 22 is a storage medium such as a ROM and a RAM. The storage unit 22 may be a volatile memory or a non-volatile memory. The storage unit 22 stores map information indicating a 3D map, for example. The storage unit 22 also stores a program executed by the control unit 23.

The control unit 23 is, for example, a CPU. The control unit 23 functions as the management information acquisition unit 231, the identification unit 232, the map display unit 233, the schedule acquisition unit 234, and the notification unit 235 by executing the program stored in the storage unit 22. The management information acquisition unit 231 (1) the device ID, (2) distance information, (3) the captured image of the obstacle A and the captured date/time information, and (4) the position information acquisition unit 171 at the time of capturing the captured image. The communication unit 15 acquires the acquired flight position information and flight altitude information, (5) direction information, and (6) three-dimensional information. In addition, the management information acquisition unit 231 acquires (7) second route information by the communication unit 21. The management information acquisition unit 231 stores the acquired information (1) to (7) in the storage unit 22.

The identification unit 232 identifies the position of the obstacle A captured in the captured image acquired by the management information acquisition unit 231. The identifying unit 232 identifies the position of the obstacle A by longitude and latitude, for example. For example, the identifying unit 232 identifies, as the position of the obstacle A, a position that is moved from the flight position at the time of capturing the captured image in the imaging direction by the distance measured by the distance measuring unit 13. The specifying unit 232 specifies the vicinity of the center of the obstacle A as the position of the obstacle A based on the position and the size of the obstacle A in the horizontal direction, the vertical direction, and the depth direction recognized by the recognizing unit 174. Good.

The identifying unit 232 also identifies the type of the obstacle A based on the captured image of the obstacle A. The type of the obstacle A is, for example, a lightning rod, a crane, a high-rise building under construction, or the like. The identifying unit 232 extracts the feature information of the obstacle A from the captured image of the obstacle A, and compares the extracted feature information with the feature information stored in the storage unit 22 for each type of the obstacle, thereby determining the obstacle. The type of the object A is specified.

The map display unit 233 causes the communication unit 21 to transmit the map information stored in the storage unit 22 to a display device (not shown) to display the 3D map indicated by the map information on the display of the display device. The map display unit 233 also adds the information on the obstacle A acquired by the management information acquisition unit 231 to the map information in the storage unit 22. The map display unit 233 displays the icon of the obstacle A at the position on the 3D map corresponding to the position of the obstacle A specified by the specifying unit 232. The map display unit 233 also displays the captured image of the obstacle A and the captured date and time in association with the icon of the obstacle A. For example, when the map display unit 233 receives an operation of selecting the icon of the obstacle A on the display device, the map display unit 233 displays the captured image of the obstacle A and the captured date and time.

[Example of display image of 3D map]
The operation of the map display unit 233 will be described with reference to FIG. FIG. 4 shows an example of a display image of a 3D map displayed on the display device by the map display unit 233. As shown in FIG. 4, the map display unit 233 displays icons corresponding to obstacles such as buildings on the 3D map. Each of the plurality of rectangular parallelepiped icons in FIG. 4 indicates a building. The map display unit 233 also displays the icon of the obstacle A acquired by the management information acquisition unit 231 at a position corresponding to the position specified by the specifying unit 232. The icon of the obstacle A is shown with hatching.

The map display unit 233 displays an icon based on the size of the obstacle A acquired by the management information acquisition unit 231 in the horizontal direction, the vertical direction, and the depth direction. For example, the map display unit 233 displays an icon obtained by reducing the horizontal direction, the vertical direction, and the depth direction of the obstacle A acquired by the management information acquisition unit 231 at a predetermined reduction ratio. The predetermined reduction rate is, for example, 1/1000. In addition, when the map display unit 233 receives an operation of selecting the icon of the obstacle A on the display device, the map display unit 233 displays the captured image of the obstacle A and the captured date and time, as shown in the lower right area of FIG. ..

Returning to the description of FIG. The schedule acquisition unit 234 acquires the flight schedule of another flight device from the one or a plurality of devices (not shown) that manage the other flight device by the communication unit 21. The schedule acquisition unit 234 refers to the acquired flight schedules of the other flight devices, and specifies the flight device that is scheduled to move in an area within a predetermined distance from the obstacle A. The predetermined distance is a distance at which it is assumed that another flight device may come into contact with the obstacle A due to an error in positioning of a signal from a GPS satellite or the influence of wind, for example, 20 meters from the obstacle A. Within.

From the viewpoint of suppressing contact with the obstacle A, the information regarding the obstacle A is useful for other flight devices that are scheduled to move around the obstacle A. The information on the obstacle A is, for example, a captured image of the obstacle, a type, a position, or an altitude. Therefore, the notification unit 235 is acquired by the management information acquisition unit 231 with respect to one or a plurality of devices that manage the flight device identified by the schedule acquisition unit 234 as planning to move within the predetermined distance from the obstacle A. The communication unit 21 notifies the information about the obstacle A that has been deleted or a new flight route for avoiding the obstacle A.

[Processing procedure of flight device 100]
FIG. 5 is a flowchart showing an example of the operation of the flying device 100. The processing procedure of FIG. 5 starts when the detection unit 12 detects the obstacle A in a predetermined area including the first flight route.

First, the flight control unit 172 starts hovering (step S101), and the distance measuring unit 13 measures the distance from the flight position of the flying device 100 to the obstacle A (step S102). Furthermore, the captured image acquisition unit 175 acquires a captured image of the obstacle A (step S103). Further, the captured image acquisition unit 175 acquires the acquisition date and time of the captured image and the capturing direction (step S104). The position information acquisition unit 171 acquires flight position information and flight altitude information when the captured image acquisition unit 175 acquires the captured image (step S105).

The generation unit 173 generates the second flight route for avoiding the obstacle A (step S106). The communication control unit 176 acquires (1) the device ID, (2) the distance information measured by the distance measuring unit 13, (3) the captured image of the obstacle A and the captured date/time information, and (4) at the time of acquiring the captured image. The acquired flight position information and flight altitude information, (5) direction information, and (6) three-dimensional information recognized by the recognition unit 174 are transmitted to the management device 200 (step S107).

After that, the flight control unit 172 performs flight control to fly along the second flight route generated by the generation unit 173 (step S108). Further, the communication control unit 176 transmits the second route information indicating the second flight route to the management device 200 after the flight device 100 returns to the first flight route along the second flight route (step S109). The communication control unit 176 may transmit the second route information to the management device 200 before or during the flight of the flight device 100 on the second flight route.

[Processing Procedure of Management Device 200]
FIG. 6 is a flowchart showing an example of the operation of the management device 200. The processing procedure of FIG. 6 starts when the communication control unit 176 of the flying device 100 transmits various information items (1) to (6) in step S107 of FIG. 5 to the management device 200.

First, the management information acquisition unit 231 acquires various information shown in (1) to (6) of step S107 of FIG. 5 from the flying device 100 (step S201). Further, the management information acquisition unit 231 acquires the second route information by the communication unit 15 (step S202). The identification unit 232 identifies the position of the obstacle A captured in the captured image acquired by the management information acquisition unit 231 (step S203). The map display unit 233 displays the icon of the obstacle A at the position on the 3D map corresponding to the position of the obstacle A specified by the specifying unit 232 (step S204). Further, the map display unit 233 displays the captured image of the obstacle A and the captured date and time in association with the icon of the obstacle A (step S205).

In addition, the schedule acquisition unit 234 acquires the flight schedule of the other flight device from the one or more devices that manage the other flight device by the communication unit 21 (step S206). The schedule acquisition unit 234 refers to the acquired flight schedules of the other flight devices to identify the flight device that is scheduled to move within a predetermined distance from the obstacle A. Furthermore, the notification unit 235 detects a failure acquired by the management information acquisition unit 231 with respect to one or a plurality of devices that manage the flight device identified by the schedule acquisition unit 234 as having a plan to move within a predetermined distance from the obstacle A. The communication unit 21 notifies the information about the object A (step S207).

[Effects of the Present Invention]
According to the present embodiment, the generation unit 173 sets the second flight route based on the flight position information acquired by the position information acquisition unit 171 and the captured image of the obstacle A acquired by the captured image acquisition unit 175. To generate. Therefore, the flight control unit 172 can avoid the obstacle A along the second flight route when the detection unit 12 detects the obstacle A in a predetermined region including the first flight route. ..

Further, according to the present embodiment, the management information acquisition unit 231 acquires the shooting date and time of the captured image from the flying device 100, and the identification unit 232 determines the flight position information at the time of capturing the captured image and the captured image. The position of the obstacle captured in the captured image is specified based on the direction of the arrow and the distance from the flight position to the obstacle. Further, the map display unit 233 displays the icon of the obstacle A at the position on the map of the map information corresponding to the position of the obstacle A specified by the specifying unit 232, and the captured image of the obstacle A and the captured date and time. Is associated with the icon of the obstacle A. Therefore, the user can easily understand the position of the obstacle A, the captured image, and the captured date and time on the map.

Further, according to the present embodiment, the map display unit 233 associates the size of the obstacle A specified by the specifying unit 232 with the icon of the obstacle A. Therefore, the user can easily grasp the size of the obstacle A on the map.

Further, according to the present embodiment, the notification unit 235 informs the device that manages the flight device that is scheduled to fly in the area within the predetermined distance from the obstacle A to the information about the obstacle A or the second flight route. To notify. Therefore, a device that manages another flight device can set a flight route that avoids the obstacle A in advance.

[Modification 1]
In this embodiment, the communication control unit 176 manages the flight position information, the captured image of the obstacle A, the direction information, and the second route information when the flying device 100 flies on the second flight route. The example of the case of sending to the has been described. However, the present invention is not limited to this. For example, when the flying device 100 flies on the second flight route, the flight position information, the captured image, the direction information, and the second route information are stored in the storage unit 16 without being transmitted to the management device 200. These pieces of information may be read from the storage unit 16 to the outside of the flying device 100 after the flight of the device 100 is completed. By adopting such a configuration, even if communication failure occurs between the flying device 100 and the management device 200, it is possible to read and use the captured image of the obstacle A detected by the detection unit 12 and the like.

[Modification 2]
Further, the present embodiment has described the configuration in which the detection unit 12 detects the obstacle A in a predetermined area including the first flight route. However, the present invention is not limited to this. For example, the detection unit 12 may detect an object outside the predetermined area.

For example, the storage unit 16 of the flying device 100 stores map information indicating the same 3D map as the storage unit 22 of the management device 200. When the detection unit 12 detects an object outside the predetermined area, the recognition unit 174 specifies the position of the object by the same method as the method by which the specifying unit 232 of the management device 200 specifies the position of the obstacle A, for example. .. The recognition unit 174 refers to the map information stored in the storage unit 22 and determines whether or not an object such as an obstacle is shown at the position on the 3D map corresponding to the specified position. The recognition unit 174 determines that the object detected by the detection unit 12 is a new object not shown in the 3D map when the object is not shown at the position on the 3D map corresponding to the specified position. On the other hand, the recognition unit 174 determines that the object detected by the detection unit 12 is a known object shown in the 3D map when the object is shown at the position on the 3D map corresponding to the specified position. To do.

When the recognition unit 174 determines that the object detected by the detection unit 12 is a new object not shown in the 3D map, the communication control unit 176 (1) uses the device ID for identifying the flying device 100. , (2) distance information indicating the distance from the flight position of the flying device 100 to the object measured by the distance measuring unit 13, and (3) a captured image of the object acquired by the image capturing unit 14 and an image capturing date and time of the captured image. Date and time information, (4) Flight position information and flight altitude information acquired at the time of acquisition of the captured image by the position information acquisition unit 171, (5) Direction information indicating the imaging direction of the captured image, and (6) Recognition unit 174. The three-dimensional information indicating the position and size of the object in the horizontal direction, the vertical direction, and the depth direction recognized by (1) and (7) the position information indicating the position of the object identified by the recognition unit 174 are transmitted to the management device 200.

Further, the communication control unit 176 may store the information (1) to (7) in the storage unit 22 without transmitting the information to the management device 200. The map display unit 233 also displays the icon of the object at the position on the 3D map corresponding to the position of the object identified by the recognition unit 174. By adopting such a configuration, it is possible to share the information about the object not stored in the map information stored in the storage unit 16 for the flight of another flight device.

[Modification 3]
The detection unit 12 may detect that the object indicated in the map information stored in the storage unit 16 does not exist. For example, the recognition unit 174 reads objects existing within a certain range from the flight position of the flying device 100 in the map information, and determines whether the detection unit 12 has detected these objects. The certain range is, for example, a range in which the detection unit 12 can detect an object.

When the recognition unit 174 determines that there is no object in the map information that is within a certain range from the flight position of the flying device 100 in the map information, the communication control unit 176 identifies this object in the map information. The specific information to be displayed and the information indicating that this object does not exist are transmitted to the management device 200. The management information acquisition unit 231 acquires the specific information and the information indicating that the object does not exist by the communication unit 21, and deletes the object specified by the specific information in the map information stored in the storage unit 22. Update the map information. Further, the communication control unit 176 may store the information in the storage unit 16 instead of transmitting the information to the management device 200.

By adopting such a configuration, when the detection unit 12 detects that the object shown in the map information does not exist, the absence of this object is shared for the flight of another flight device. be able to.

[Modification 4]
Further, in the present embodiment, an example in which the generation unit 173 of the flight device 100 generates the second flight route has been described. However, the present invention is not limited to this. For example, the flight device 100 may generate the second flight route for avoiding the obstacle A in the specifying unit 232 of the management device 200.

When the detection unit 12 detects the obstacle A, the communication control unit 176 of the flying device 100 (1) the device ID, (2) the distance information measured by the distance measurement unit 13, and (3) the obstacle A. Captured image and imaging date and time information, (4) flight position information and flight altitude information acquired when the captured image was acquired, (5) direction information, and (6) horizontal direction of obstacle A recognized by the recognition unit 174. , And the three-dimensional information indicating the position and size in the vertical direction and the depth direction to the management device 200.

The identifying unit 232 identifies the size of the obstacle A based on the captured image and the distance information indicating the distance from the flight position at the time of capturing the captured image to the obstacle A. The identifying unit 232 also creates the second flight route by the same method as the creating unit 173. The storage unit 22 stores the information on the above-mentioned (1) to (6) and the second flight route in association with each other. In this case, the notification unit 235 notifies the flight device 100 of the second flight route generated by the identification unit 232. In addition, the map display unit 233 associates the size of the obstacle A specified by the specifying unit 232 with the icon of the obstacle A.

By adopting such a configuration, the second flight route can be generated by the processor in the specifying unit 232 of the management device 200. Since the processor in the management device 200 has a small design constraint, it is possible to increase the processing capacity as compared with the processor in the flight device 100. Therefore, the time taken to generate the second flight route can be shortened.

[Modification 5]
Further, the specifying unit 232 may recognize the three-dimensional information indicating the position and size of the obstacle A in the horizontal direction, the vertical direction, and the depth direction by the same method as the recognition unit 174. In this case, the recognition unit 174 in the flying device 100 is not necessary, and the three-dimensional information indicating the position and size of the obstacle A acquired by the flying device 100 in the horizontal direction, the vertical direction, and the depth direction is transmitted to the management device. Similarly to the generation unit 173, the identification unit 232 generates the second flight route by referring to the position and size of the obstacle A recognized by the identification unit 232 in the horizontal direction, the vertical direction, and the depth direction. Good.

[Modification 6]
Further, the captured image acquisition unit 175 has been described with respect to an example of the case where the captured image of the obstacle A is acquired during hovering. However, the present invention is not limited to this. For example, the captured image acquisition unit 175 may acquire captured images of the obstacle A at a plurality of flight positions on the second flight route.

In this case, the distance measuring unit 13 measures the distance information for each flight position at which the captured image is acquired, the captured image acquisition unit 175 acquires the captured date/time information and direction information of each captured image, and the position information acquisition unit. 171 acquires flight position information and flight altitude information at the time of acquisition of each captured image. The communication control unit 176, in addition to the device ID of the flight device 100 and the three-dimensional information of the obstacle A recognized by the recognition unit 174, respectively, the distance information, the imaging date/time information, the direction information, the flight position information, and the flight altitude information. It is transmitted to the management device 200.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist thereof. is there. For example, the specific embodiment of device distribution/integration is not limited to the above embodiment, and all or part of the device may be functionally or physically distributed/integrated in arbitrary units. You can Further, a new embodiment that occurs due to an arbitrary combination of a plurality of embodiments is also included in the embodiment of the present invention. The effect of the new embodiment produced by the combination also has the effect of the original embodiment.

11 GPS receiving unit 12 Detecting unit 13 Distance measuring unit 14 Imaging unit 15 Communication unit 16 Storage unit 17 Control unit 21 Communication unit 22 Storage unit 23 Control unit 100 Flight device 171 Position information acquisition unit 172 Flight control unit 173 Generation unit 174 Recognition unit 175 Captured image acquisition unit 176 Communication control unit 200 Management device 231 Management information acquisition unit 232 Specification unit 233 Map display unit 234 Schedule acquisition unit 235 Notification unit

Claims (14)

A position information acquisition unit that acquires flight position information indicating a flight position,
A detection unit that detects an obstacle in a predetermined area including the first flight route;
A captured image acquisition unit that acquires a captured image of the obstacle,
When flying the second flight route generated based on the flight position information and the captured image in order to avoid the obstacle, the flight position information, the captured image, the direction of capturing the captured image, and the second A storage unit that stores 2 flight routes,
A flying device. The flight device according to claim 1, further comprising: a generation unit that generates the second flight route based on the flight position information and the captured image. The flight device according to claim 1 or 2, wherein the first flight route is generated based on map information that does not include information about the obstacle. Further comprising a distance measuring unit for measuring the distance from the flight position to the obstacle,
The flight device according to claim 1, wherein the storage unit stores distance information indicating a distance from the flight position to the obstacle when flying the second flight route. When the detection unit detects an object outside the predetermined area, which is not shown in the map information, the storage unit stores information about the object,
The flight device according to any one of claims 1 to 4. When flying on the second flight route, the system further includes a transmission unit that transmits the flight position information, the captured image, a direction of capturing the captured image, and the second flight route to a management device.
The flight device according to any one of claims 1 to 5. The storage unit further stores map information indicating the position of the object ,
The detection unit detects the presence or absence of the object shown in the map information,
The storage unit, and wherein the object shown in the map information that does not exist, when the detecting unit during flight of the flying device detects, stores that the object does not exist The flight device according to claim 1 or 2 . Flight position information indicating a flight position of the flight device, a captured image of an obstacle in a predetermined area including the first flight route captured from the flight device flying the first flight route, and the captured image Management information acquisition unit that acquires the direction of
A second flight route generated for avoiding the obstacle based on the flight position information and the captured image, the flight position information, the captured image, and an imaging direction of the captured image are stored. Storage part,
A management device comprising: The management device according to claim 8, wherein the first flight route is generated based on map information that does not include information about the obstacle. The management information acquisition unit further acquires the imaging date and time of the captured image,
A storage unit that stores map information,
Identification for identifying the position of the obstacle captured in the captured image based on the flight position information when capturing the captured image, the direction of capturing the captured image, and the distance from the flight position to the obstacle Department,
The obstacle icon is displayed at a position on the map of the map information corresponding to the position of the obstacle specified by the specifying unit, and the captured image of the obstacle and the imaging date and time are displayed as an icon of the obstacle. And a map display section associated with
The management device according to claim 8. The identifying unit identifies the size of the obstacle based on the distance from the flight position to the obstacle when the captured image and the captured image are captured,
The map display unit associates the size of the obstacle identified by the identifying unit with the obstacle icon,
The management device according to claim 10. Further comprising a schedule acquisition unit for acquiring flight schedules of other flight devices,
Further, a notification unit for notifying information regarding the obstacle or the second flight route to a device that manages a flight device that is scheduled to fly in an area within a predetermined distance from the obstacle,
The management device according to any one of claims 8 to 11. Flight position information indicating a flight position of the flight device, a captured image of an obstacle in a predetermined area including the first flight route captured from the flight device flying the first flight route, and the captured image To obtain the direction of
A second flight route generated for avoiding the obstacle based on the flight position information and the captured image, the flight position information, the captured image, and an imaging direction of the captured image are stored. And a step,
Flight management method. On the computer,
Flight position information indicating a flight position of the flight device, a captured image of an obstacle in a predetermined area including the first flight route captured from the flight device flying the first flight route, and the captured image To obtain the direction of
A second flight route generated for avoiding the obstacle based on the flight position information and the captured image, the flight position information, the captured image, and an imaging direction of the captured image are stored. Steps,
A program to execute.

Images