JP7442467B2 - Photographed image classification device and photographed image classification method - Google Patents

Description

The present invention relates to an apparatus and method for classifying image data obtained by photographing inspection parts of various structures such as gantry cranes, steel towers, and bridges.

With advances in digital camera technology and storage device technology, it has become easy to store large amounts of captured images (image data) captured by digital cameras in storage devices. For this reason, for example, images obtained by photographing construction sites and objects to be inspected with a digital camera are accumulated and used for managing the work status of construction sites and inspecting various objects to be inspected. things are being done. However, the number of photographed images obtained by photographing for work status management and inspection work may number in the thousands. Therefore, it is necessary to appropriately classify and manage a large amount of photographed images, and various inventions related to this have been made.

For example, in Patent Document 1 and Patent Document 2, when photographing a work scene for each work process according to a photographing plan, scene numbers and classification items are added to the photographed images, and the photographed images are classified according to the photographing plan. An invention is disclosed that allows for organization. Further, Patent Document 3 discloses an invention in which a photographed image is stored in association with a photographing point ID uniquely assigned to a position. Further, Patent Document 4 discloses an invention that generates classification information for classifying a plurality of image files according to attribute information, and controls display of the plurality of image files using the generated classification information. ing.

Further, Patent Document 5 discloses an invention related to an image management device and the like. In the invention disclosed in Patent Document 5, when a grouping condition for classifying one or more images as the same case based on location information is attached to an image, the images are classified for each case according to the condition. However, if the condition is not attached to the image, the image is classified according to the grouping condition that makes the image the same as other images within a predetermined distance from the image. That is what it is. Further, Patent Document 6 discloses an invention in which one building is divided into a plurality of management units and inspection result data is stored for each management unit.

Japanese Patent Application Publication No. 2005-033448 Japanese Patent Application Publication No. 2007-140738 Japanese Patent Application Publication No. 2011-070632 Japanese Patent Application Publication No. 2014-219993 Japanese Patent Application Publication No. 2016-019166 Japanese Patent Application Publication No. 2016-125846

However, there is a demand for categorizing captured images even more easily and flexibly. For example, consider a case where a structure is inspected. Inspections of structures are often performed repeatedly on a regular basis. Furthermore, there are various types of structures that are objects to be inspected. For example, there are some types of cranes, such as gantry cranes and transfer cranes, which are movable and have a plurality of similar structures (mobile common structures). In addition, there are structures such as power transmission line towers that are located at fixed locations but have multiple structures with the same structure (fixed common structures). Additionally, there are structures that are located at fixed locations such as bridges and are unique in themselves (fixed unique structures).

In the case of a fixed unique structure such as a bridge, the photographing position (longitude, latitude, altitude (height)) for photographing the specified inspection area is determined, the inspection area is photographed from the photographing position, and the photographed image is It is conceivable to classify based on the photographing position. However, in the case of movable common structures, they move even within a limited area such as a port area or a container yard, so the photographing position cannot be uniquely determined. For this reason, it is necessary to determine the photographing position each time a moving common structure is inspected.

In addition, if there are multiple moving common structures, the positions of each of them are different, so even if the common structures have similar structures and the inspection parts are common, the photographing position for each moving common structure will be different. must be determined and inspected, which is time-consuming. Similarly, in the case of fixed common structures, because they are installed in different locations, even though they are common structures with similar structures and the same inspection parts, the photographing location is different for each fixed common structure. must be determined and inspected, which is time-consuming.

In view of the above, the present invention is capable of capturing images obtained by photographing any structure for inspection, regardless of the installation position of the structure to be inspected. The purpose is to enable appropriate classification and management without the hassle of human intervention.

In order to solve the above problem, a photographed image classification device according to the invention according to claim 1 includes:
For each of one or more inspection parts of the inspection target, setting information for each inspection part is stored, which associates a scheduled imaging position in a relative coordinate space with a predetermined reference point as the origin and scheduled movement information of the imaging means. a setting information storage means for
Reference point storage means for storing the longitude, latitude, and altitude of the reference point;
Image data obtained by photographing through the imaging means, which is information acquired when photographing the inspection part of the inspection object, longitude, latitude, and altitude indicating the photographing position, and the time of photographing by the imaging means. Photographic data storage means for storing photographic data associated with movement information;
normalizing means for converting the longitude, latitude, and altitude of the photographic data into a photographing position in the relative coordinate space using the longitude, latitude, and altitude of the reference point storage means;
Based on the converted imaging position of the imaging data and the motion information at the time of imaging of the imaging data, the inspection area-specific setting information in the setting information storage means is referred to, and the inspection area is specified and inspected. and an image classification means for recording the image data of the photographic data on a recording medium for each image data.

According to the present invention, images are taken for inspection appropriately without depending on the actual position of the object to be inspected, and photographed images (images) obtained by photographing each inspection part of the object to be inspected are taken. data) can be classified.

FIG. 2 is a diagram for explaining a usage environment of a photographed image classification device according to an embodiment. FIG. 1 is a block diagram for explaining a configuration example of a UAV (unmanned aerial vehicle) according to an embodiment. FIG. 3 is a diagram for explaining an example of data stored in an imaging data file of a UAV according to an embodiment. FIG. 2 is a diagram for explaining settings of inspection parts, scheduled photographing positions, and scheduled movement information of a gantry crane, which is an example of an inspection object. FIG. 6 is a diagram for explaining the association between inspection parts, scheduled imaging positions, and scheduled camera movement information. FIG. 1 is a block diagram for explaining a configuration example of a photographed image classification device according to an embodiment. FIG. 3 is a diagram for explaining an example of data stored in a normalized data file of the captured image classification device according to the embodiment. FIG. 3 is a diagram for explaining an example of an image data file classified by inspection site in an image data folder classified by inspection site of the photographed image classification device according to the embodiment. It is a flowchart for explaining a series of processing when inspecting an object to be inspected. 7 is a flowchart for explaining a photographed image classification process executed by the photographed image classification device according to the embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of a photographed image classification device and a photographed image classification method according to the present invention will be described below with reference to the drawings. The photographed image classification device and photographed image classification method described below are based on a large number of photographed images (images in the hundreds or even thousands) obtained by photographing each of a large number of inspection parts of a large inspection target. data) can be easily and appropriately classified.

There are various large inspection objects, such as gantry cranes, bridges, steel towers, wind turbine towers for wind power generation, and Ferris wheels. In addition, photographic means that can be used for photographing include cameras mounted on unmanned aircraft such as drones and UAVs, cameras mounted on various mobile robots, and so-called gimbals held in the hands of inspection workers. There are fixed cameras.

In the embodiment described below, the inspection target is a gantry crane used for loading and unloading containers in a port area, and the imaging means is a digital camera mounted on a UAV. Gantry cranes are large steel structures used in port areas, measuring several tens of meters in height and length, and it is important to periodically inspect them in order to properly understand the state of rust and deterioration over time. Additionally, UAVs are suitable for use in inspecting high places.

[Usage environment of photographed image classification device]
FIG. 1 is a diagram for explaining a usage environment of a captured image classification device 1 according to an embodiment. The UAV 2 includes a current position measuring means and a camera section 210. The current position measuring means of the UAV 2 uses the RTK (Real Time Kinematic) method, which will be described in detail later, and is capable of positioning the current position with high accuracy within an error of several centimeters. The camera unit 210 is fixed to a support 210P, as shown surrounded by a dotted line in FIG. 1, and is capable of a tilt operation as shown by arrow a and a roll action as shown by arrow b.

The tilt operation is an operation that changes the direction of the camera unit 210 in the vertical (up and down) direction. It is possible to tilt the camera up to ±45 degrees. Further, the roll operation is an operation of rotating the camera unit 210 around the optical axis of the camera unit 210, and the camera unit 210 of the UAV 2 of this embodiment moves 45 degrees up and down from the 0 degree position (reference value). It is possible to roll in degrees (±45 degrees).

When the UAV 2 photographs the inspection area of the gantry crane, which is the object to be inspected, the UAV 2 records the photographed image (image data) with the photographing position (longitude, latitude, altitude) and movement information of the camera unit 210 at the time of photographing. are acquired to form photographic data in which they are associated with each other. In this embodiment, the movement information of the camera unit 210 at the time of photographing includes a tilt angle θ at the time of photographing and a roll angle φ at the time of photographing. This photographic data is provided to the photographed image classification device 1 and is held in a photographic data file 104 as shown in FIG.

The inspection site matching device 3 is configured of, for example, a CAD (Computer Aided Design) device. The inspection part associating device 3 sets a scheduled photographing position and scheduled movement information of the camera section 210 for each inspection part of the gantry crane to be inspected in a relative coordinate space with a predetermined reference point as the origin. The planned movement information of the camera unit 210 is a planned tilt angle α and a planned roll angle β. The inspection area associating device 3 forms an image data file for each inspection area with header information including identification information of the inspection area, a scheduled photographing position set for each inspection area, and scheduled movement information of the camera unit 210, and identifies the inspection area for each inspection area. Register in the image data folder 106.

Note that when photographing the inspection site using the UAV 2, it is necessary to take into account in advance the influence of wind and variations in the skill of operators for items that are remotely operated. For this reason, the scheduled imaging position and scheduled movement information of the camera section 210 of each inspection site-specific imaging data file registered in the inspection site-specific image data folder 106 are set with a certain range. That is, the planned photographing position, the planned tilt angle α, and the planned roll angle β of the camera unit 210 are set to have a range.

The reference point surveying device 4 accurately measures the longitude, latitude, and altitude of the above-described predetermined reference point, and sets this in the photographed image classification device 1. Specifically, the reference point surveying device 4 includes current position measuring means and a semiconductor laser, irradiates a laser beam onto a target reference point, receives the reflected light, and accurately determines the distance to the reference point. The reference point surveying device 4 accurately measures the longitude, latitude, and altitude of the reference point based on the laser beam irradiation direction, the current position of the aircraft, and the distance to the measured reference point, and uses the captured image classification device to measure the longitude, latitude, and altitude of the reference point. Set to 1. Note that the current position positioning means of the reference point surveying device 4 is capable of positioning the current position with high precision, such as by adopting the RTK method, similar to the current position positioning means of the UAV 2.

The UAV 2 photographs the inspection site from the scheduled photography position set by the inspection site association device 3. In this case, the scheduled photographing position is a relative coordinate in a relative coordinate space whose origin is a predetermined reference point. However, the longitude, latitude, and altitude of the reference point are accurately measured and specified by the reference point surveying device 4. For this reason, for example, by setting the longitude, latitude, and altitude of the reference point surveyed by the reference point surveying device 4 to the inspection part matching device 3, the longitude, latitude, and altitude of the reference point can be set. , you can specify the longitude, latitude, and altitude of the planned shooting location.

Therefore, when the UAV 2 is operated by remote control, the operator positions the UAV 2 at a scheduled photographing position specified by longitude, latitude, and altitude through the remote control device. Further, the operator remotely controls the camera unit 210 of the UAV 2 to face the inspection site, photographs the inspection site, and forms and stores photographic data as described above.

Further, it is assumed that the UAV 2 is capable of autonomous flight. In this case, as described above, the longitude, latitude, and altitude of the scheduled photographing position specified by the inspection region matching device 3 are set in the UAV 2. Further, necessary information such as the direction of the camera unit 210 (the direction in which the front of the UAV 2 faces) and the number of images to be taken for each inspection site is set for the UAV 2. According to this information, the UAV 2 autonomously flies, photographs the inspection site from a set scheduled photographing position, photographs the inspection site, and forms and stores photographic data as described above. Therefore, in this case, the inspection part associating device 3 also functions as a UAV navigation device.

As described above, the photographed data file 104 of the photographed image classification device 1 stores and holds the photographed data (photographed image, photographing position, information on the movement of the camera unit 210 when photographing) obtained by photographing the inspection site through the UAV 2. do. The photographed image classification processing unit 110 of the photographed image classification device 1 converts the longitude, latitude, and altitude indicating the photographing position of each photographic data in the photographic data file 104 into a reference point based on the longitude, latitude, and altitude of the set reference point. Convert to a position (x, y, z) in a relative coordinate space with the point as the origin. That is, the longitude, latitude, and altitude indicating the photographing position are normalized to relative coordinates in a relative coordinate space with the reference point as the origin.

Next, the photographed image classification processing unit 110 uses the normalized photographing position of the photographic data and the movement information at the time of photographing of the camera unit 210 to refer to the inspection site-specific image data folder 106 and Identify another image data file. That is, in the inspection part image data folder 106, an inspection part image data file is registered for each inspection part of the gantry crane to be inspected. As mentioned above, the inspection site-specific image data file includes the scheduled imaging position (range) with a designated range and the scheduled movement of the camera unit 210 with a designated range in a relative coordinate space with a predetermined reference point as the origin. information as header information.

For this reason, the photographed image classification processing unit 110 combines the normalized photographing position, the scheduled photographing position (planned photographing area) specified by the range to which the movement information at the time of photographing of the camera unit 210 belongs, and the scheduled movement of the camera unit 210 specified by the range. The inspection site-specific image data file having the information is identified. This makes it possible to specify which inspection site the photographed image (image data) in which the photographing position and motion information at the time of photographing are associated is of which inspection site. The photographed image classification processing unit 110 stores the photographed image (image data) of the photographed data to be classified in the inspection site-specific image data file of the specified inspection site. Thereby, the captured images of the captured data can be classified and managed by inspection site.

As described above, in this embodiment, (1) the planned photographing position and the planned movement information (tilt angle α, roll angle β) are set in the photographed image classification device 1. (2) Measure the longitude, latitude, and altitude of a predetermined reference point and set them in the photographed image classification device 1.

(3) Using the UAV 2, each inspection site is photographed, and photographing data consisting of a photographed image, a photographing position, and movement information of the camera unit 210 at the time of photographing (tilt angle θ at the time of photographing, roll angle φ at the time of photographing) is formed. In this case, the planned imaging position (longitude, latitude, altitude) in the absolute coordinate space is specified according to the planned imaging position in the relative coordinate space, and the inspection area is photographed from the planned imaging position. (4) Photographed using UAV2 The longitude, latitude, and altitude of the imaging position of the imaging data of each inspection site formed by this are converted into a position (x, y, z) in a relative coordinate space with the reference point as the origin. (5) Each file in the image data folder 106 for each inspection site is referred to based on the converted imaging position and the movement information of the camera unit 210 at the time of imaging. That is, the image data file for each inspection area having the scheduled imaging position and scheduled movement information to which the imaging position and movement information at the time of imaging belong of the imaging data is specified, and the imaging image (image data) of the imaging data is stored in this. . This completes the classification of images by inspection site, and enables management of captured images by inspection site.

In order to facilitate the explanation of the captured image classification device 1 used in the usage environment described using FIG. ) settings will be explained. Thereafter, a configuration example of the photographed image classification device 1 according to the present invention will be specifically explained.

[Configuration example of UAV2]
FIG. 2 is a block diagram for explaining a configuration example of the UAV 2. As shown in FIG. There are various types of UAVs, and they can be roughly divided into fixed wing types with fixed wings like a general airplane, helicopter types with one rotor blade, and helicopter types with three or more rotor blades. There is a multicopter type. Helicopter-type and multicopter-type UAVs are suitable for inspection work at high places because they are capable of hovering, in which they fly while remaining at a fixed position in the air. As shown in FIG. 1, the UAV 2 of this embodiment is of a quadcopter type with four rotary blades.

As shown in FIG. 2, the UAV 2 includes a flight mechanism section 21 consisting of a rotor and its drive unit, and a drive control unit 22. The UAV 2 is capable of both flight by remote control and autonomous flight according to set flight instruction data. In FIG. 2, a transmitting/receiving antenna 201A and a wireless communication unit 201 mainly communicate with a remote control device. Further, during autonomous flight, the transmitting/receiving antenna 201A and the wireless communication unit 201 are also used, for example, when transmitting and receiving flight instruction data provided from a UAV navigation device or the like.

Although not shown, the control unit 202 is a microprocessor configured by connecting a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), nonvolatile memory, etc. through a bus, and is Control each part. The storage device 203 consists of a storage medium and its driver, and stores various programs and various data required for processing, and is also used as a work area for temporarily storing intermediate results of various processing. Furthermore, as described above, the storage device 203 is also capable of storing flight instruction data provided from the inspection area associating device 3 that functions as a UAV navigation device.

The photographic data file 204 is created in a storage medium of a predetermined storage device, and stores and holds photographic data including photographed images (image data) obtained by photographing through a camera unit 210, which will be described later. FIG. 3 is a diagram for explaining an example of data stored in the photographing data file of the UAV 2. As shown in FIG. 3, the photographing data file includes image data, longitude, latitude, and altitude as the photographing position, direction of the camera unit 210 at the time of photographing, tilt angle θ at the time of photographing, and tilt pitch at the time of photographing as motion information at the time of photographing. angle, a roll angle φ at the time of photographing, and a roll pitch angle at the time of photographing.

The image data is a photographed image of the inspection site obtained by photographing through the camera unit 210 of the UAV 2. The photographing position is the longitude, latitude, and altitude indicating the current position of the UAV 2 at the time of photographing, which is determined with high precision by a current position positioning means to be described later. The current position of the UAV 2 can be considered as the current position of the camera unit 210. The tilt angle θ at the time of photographing, which is movement information of the camera unit 210 at the time of photographing, is the angle when the direction of the camera unit 210 is changed in the vertical (up and down) direction. Further, the roll angle φ at the time of photographing, which is movement information of the camera section 210 at the time of photographing, is an angle when the camera section 210 is rotated about the optical axis of the camera section 210. This movement information at the time of photographing can be acquired from the camera unit 210.

Furthermore, the direction of the camera unit 210 is information used when the UAV 2 flies autonomously, and is the direction in which the camera unit 210 is facing when both the tilt angle θ and the roll angle φ are 0°, in other words, This shows the direction in which the front of the UAN 2 is facing. The tilt pitch angle is the front-to-back tilt of the UAV 2 (tilt in the pitch direction), and the roll pitch angle is the left-right tilt of the UAV 2 (tilt in the roll direction). The direction, tilt pitch angle, and roll pitch angle of the camera unit 210 can be acquired through a sensor mounted on the UAV 2. Note that each of the tilt pitch angle and the roll pitch angle can be used when correcting the tilt angle θ at the time of photographing and the roll angle φ at the time of photographing, taking into consideration the inclination of the UAV 2 itself.

The power supply unit 205 includes a battery and supplies necessary power to each part of the UAV 2. The sensor unit 206 includes various sensors such as a gyro sensor, an atmospheric pressure sensor, an acceleration sensor, an ultrasonic sensor, and a geomagnetic sensor. The gyro sensor is used for attitude control, and the barometric pressure sensor is used for altitude detection. The acceleration sensor is used to detect speed, and the ultrasonic sensor is used to detect distance to an object. Furthermore, the geomagnetic sensor is used for direction detection. Note that the gyro sensor and the acceleration sensor constitute a 6-axis sensor for front and rear, left and right, and up and down, so that the posture of the UAV 2 can be finely and appropriately controlled.

The autonomous attitude control unit 207 utilizes detection outputs from various sensors mounted on the sensor unit 206 and captured images from the camera unit 210 to control flight so that the UAV 2 flies stably in an appropriate attitude. Controls the drive unit 209. Note that since the ultrasonic sensor is used, it is not necessarily necessary to use the image information from the camera section 210, but the image information from the camera section 210 can also be used to confirm obstacles. This information is particularly important during takeoff and landing.

The GNSS (Global Navigation Satellite System) unit 208 and GNSS antenna 208A have a function of accurately detecting (positioning) the current position of the aircraft by receiving and analyzing transmission signals (positioning information) from multiple artificial satellites. This is the part that realizes this. Note that GNSS means Global Positioning Satellite System, and is a general term for satellite positioning systems such as GPS (Global Positioning System), GLONASS (Global Navigation Satellite System), Galileo, and Quasi-Zenith Satellite (QZSS). The GNSS unit 208 is capable of detecting longitude, latitude, and altitude. The flight drive section 209 supplies drive control signals to each of the four engine sections of the four propeller mechanisms of the flight mechanism section 21 under the control of the autonomous attitude control section 207 . Thereby, various movement operations, posture control, and hovering position control can be performed for the UAV 2.

As described above, the camera section 210 is attached to the support column 210P so as to face in one direction (forward), and is capable of tilting and rolling operations. In addition, in the case of the UAV 2 of this embodiment, the panning operation (the operation of changing the direction in the horizontal direction (left and right directions)) is realized not by the function of the camera section 210 but by changing the direction of the UAV 2 itself. The camera unit 210 can change the tilt angle and roll angle and take pictures under the control of the control unit 202.

The flight control unit 211 cooperates with the autonomous attitude control unit 207, and when the UAV 2 performs autonomous flight, the flight control unit 211 receives flight instruction data stored in the storage device 203, for example, by receiving supply from a UAV navigation device or the like. The flight drive unit 209 is controlled to fly the corresponding flight route. While the autonomous attitude control unit 207 mainly controls the attitude of the UAV 2, the flight control unit 211 controls the flight so that the flight follows a sequence of coordinate points including latitude, longitude, and altitude instructed by flight instruction data. Controls the drive unit 209.

The current position correction unit 212 realizes positioning of the current position using the RTK method. The current position correction unit 212 transmits the current position of the aircraft measured through the GNSS unit 208 to a predetermined correction station through a network communication unit (not shown), obtains correction information from the correction station, and calculates the current position of the aircraft. By making corrections, the current position of the own aircraft can be determined more accurately. Note that the correction station receives information such as the current position of the fixed station measured by other fixed stations, and receives information such as the current position measured by the mobile station from the mobile station. It is possible to form and provide correction information for correcting errors included in the current position. Further, a network notification section (not shown) is for communicating through a wide area network mainly consisting of the Internet.

Furthermore, although a case has been described here in which a predetermined correction station on a wide area network is used, the present invention is not limited to this. A fixed station is placed near the UAV2, the fixed station also measures the current position of the aircraft, and information is exchanged by wireless communication between the UAV2 and the fixed station, and the error included in the measured current position is calculated. It is also possible to use a method of correcting the current position to obtain a more accurate current position.

When a photograph is taken through the camera section 210, the photographing log forming section 213 forms log data including the photographing position (current position at the time of photographing) and movement information of the camera section 210 under the control of the control section 202. The photographing position is the accurate current position (longitude, latitude, altitude) of the own aircraft acquired by the current position correction unit 212 at the time of photographing. The movement information of the camera unit 210 includes a tilt angle θ and a roll angle φ at the time of shooting acquired from the camera unit 210, an orientation of the camera unit 210 at the time of photography (direction of the UAV 2), and a tilt pitch angle acquired from the sensor unit 206. , the roll pitch angle. As a result, the control unit 202 forms shooting data with a shooting log added to the shot image (image data) shot through the camera unit 210, and saves it to the shooting data file 204 in the manner described using FIG. Can be recorded.

The UAV 2 having such a configuration can fly around the gantry crane that is the object to be inspected, photograph the inspection site, obtain a photographed image, and record it in the photographed data file 204. The image data (photographed images) of the photographic data recorded in the photographic data file 204 are to be classified and managed.

[Setting of planned shooting position (planned shooting airspace) in relative image space]
FIG. 4 is a diagram for explaining the setting of inspection parts, scheduled photographing positions, and scheduled movement information of a camera (imaging means) of a gantry crane, which is an example of an inspection object. Note that in FIG. 4, descriptions of facilities such as an operator room and a machine room provided in the gantry crane are omitted. Further, FIG. 5 is a diagram for explaining the association between inspection parts, scheduled imaging positions, and scheduled camera movement information. Gantry cranes load and unload containers onto and from container ships, and there are 17-row types, 22-row types, etc. based on the number of containers that can be loaded in the width direction of the container ship. The 22-row gantry crane is a large steel structure with a total height of approximately 80 meters, total length of approximately 148 meters, and an outreach portion extending toward the sea that is approximately 63 meters long.

There are many inspection points for the gantry crane, but here, as shown in Fig. 4, we will inspect inspection points a1 and a2 at high places on the right side when viewed from the sea side indicated by arrow f, and inspection points at high places on the left side. Consider the case where parts b1 and b2 are inspected. Note that, as described using FIG. 1, the scheduled imaging position and scheduled movement information can be set through the inspection site matching device 3 realized by a CAD device or the like. While looking at the three-dimensional drawing of the gantry crane displayed on the display of the inspection site matching device 3, the user can operate a pointing device such as a mouse or a keyboard to set the planned imaging position and planned movement angle information.

When setting the planned photographing position and planned movement information, first, a reference point P for the gantry crane to be inspected is determined. In this embodiment, as shown in FIG. 4, a reference point P is set at the lowest end of the right front leg of the gantry crane. In a relative coordinate space with this reference point P as the origin (0, 0, 0), a scheduled photographing position and scheduled movement information of the camera unit 210 are set.

Specifically, as shown in FIG. 4, the user instructs the planned photographing position PR for the inspection parts a1 and a2 at a high place on the right side of the gantry crane. The inspection region matching device 3 specifies the coordinates of the designated planned imaging position PR in the relative coordinate space. In this example, the planned photographing position PR is on the left side (-side) of the reference point P, in front of the reference point P (-side), and above the reference point P (+ side). Therefore, relative coordinate positions (x, y, z) in the relative coordinate space of the planned photographing position PR are specified as, for example, (-25 m, -15 m, +75 m).

However, since the inspection site is photographed by the UAV2, it may be affected by the wind, or if the UAV2 is remotely controlled, it may be difficult to position it accurately at the scheduled inspection location, so the planned photographing location may be , set the width. In this embodiment, a width of 5 m is provided on the left and right (x-axis direction), front and back (y-axis direction), and above and below (z-axis direction) of the selected scheduled photographing position. Therefore, the scheduled imaging positions for the inspection sites a1 and a2 are specified as areas having width, depth, and height, as shown by the scheduled imaging area ArR in FIG. 4.

Furthermore, the planned movement information (planned tilt angle α, planned roll angle β) is set. These scheduled motion information are also set within a range, similar to the scheduled shooting position.

Similarly, the user indicates the planned photographing position PL for the inspection parts b1 and b2 on the left side of the gantry crane shown in FIG. 4. The inspection site matching device 3 specifies the coordinates of the designated planned imaging position PL in the relative coordinate space. In this example, the planned photographing position PL is on the left side (-side) of the reference point P, on the back side (+ side) of the reference point P, and above (+ side) than the reference point P. Therefore, relative coordinate positions (x, y, z) in the relative coordinate space of the scheduled photographing position PL are specified as, for example, (-25 m, +35 m, +75 m).

Further, as described above, in this embodiment, the scheduled imaging positions are set to have a width of 5 m in the left and right (x-axis direction), front and back (y-axis direction), and up and down (z-axis direction). Therefore, the scheduled imaging positions for the inspection parts b1 and b2 are specified as areas having width, depth, and height, as shown by the scheduled imaging area ArL in FIG. 4. Furthermore, the planned movement information (planned tilt angle α, planned roll angle β) is set. These scheduled motion information are also set within a range, similar to the scheduled shooting position.

As a result, as shown in FIG. 5, the scheduled imaging area ArR including the scheduled imaging positions PR of inspection areas a1 and a2 has an x coordinate in a range of -20 m to -30 m, a y coordinate in a range of -10 m to -20 m, It can be specified that the area has a z coordinate in a range of 70 m to 80 m. In addition, the planned movement information when photographing the inspection area a1 from the planned photographing area ArR is that the planned tilt angle α is in the range of 0° to +35°, and the planned roll angle β is in the range of -45° to +45°. Set. Similarly, the planned movement information when photographing the inspection area a2 from the planned photographing area ArR is that the planned tilt angle α is in the range of -35° to +0°, and the planned roll angle β is in the range of -45° to +45°. That is set.

In addition, as shown in FIG. 5, the scheduled imaging area ArL including the scheduled imaging positions PL of the inspection areas b1 and b2 has an x coordinate in the range of -20 m to -30 m, a y coordinate in the range of +30 m to +40 m, and a z coordinate in the range of -20 m to -30 m. It can be specified that the area is within a range of 70m to 80m. In addition, the planned movement information when photographing the inspection area b1 from the planned photographing area ArL is that the planned tilt angle α is in the range of 0° to +35°, and the planned roll angle β is in the range of -45° to +45°. Set. Similarly, the planned movement information when photographing the inspection area a2 from the planned photographing area ArL is that the planned tilt angle α is in the range of -35° to +0°, and the planned roll angle β is in the range of -45° to +45°. That is set.

Further, as shown in FIG. 4, in order to photograph the inspection site at the seaward end of the gantry crane, a scheduled photographing area ArF is specified, and scheduled movement information of the camera unit 210 is specified. In this way, the planned imaging position (planned imaging area) and planned movement information (planned tilt angle α, planned roll angle β) are specified for each inspection part of the gantry clay, and these are sent to the photographed image classification device 1 for each inspection part. It will be set for. Based on the information set in this way, the photographed images (image data) taken through the camera unit 210 of the UAV 2 are automatically classified by inspection area and analyzed to determine the appropriate state of rust and deterioration. be understood.

[Example of configuration of photographed image classification device]
FIG. 6 is a block diagram for explaining a configuration example of the photographed image classification device 1. As shown in FIG. Although not shown, the control unit 101 is a microprocessor configured by connecting a CPU, ROM, RAM, and nonvolatile memory through a bus, and controls each unit of the photographed image classification device 1 . The storage device 102 is a device unit consisting of a recording medium such as an SSD (Solid State Drive) and its driver, and records, changes, deletes, reads, etc. various data. The storage device 102 stores and holds necessary data and programs, and is also used as a work area to temporarily store intermediate data formed in various processes.

The reference point data file 103 is created in a storage medium of a predetermined storage device, and stores information indicating the longitude, latitude, and altitude of the reference point measured by the reference point surveying device 4, as explained using FIG. Hold. The photographic data file 104 is created in a storage medium of a predetermined storage device, and stores and holds the photographic data accumulated in the photographic data file 204 described using FIG. 3 from the UAV 2. A normalized data file 105 is also created on a storage medium of a predetermined storage device. The normalized data file 105 stores the longitude, latitude, and altitude indicating the shooting position of the shooting data stored in the shooting data file 104 as a coordinate position (x, y, z) in a relative coordinate space with the reference point P as the origin. The converted data (normalized data) is stored and retained.

FIG. 7 is a diagram for explaining an example of data stored in the normalized data file of the photographed image classification device 1. As shown in FIG. 7, the data stored in the normalized data file 105 includes image data (photographed image) and the longitude, latitude, and altitude of the photographing position at the time of photographing in a relative coordinate space with the reference point P as the origin. and data converted to coordinate positions (x, y, z). Further, the normalized data includes information such as the orientation of the camera unit 210 at the time of photographing the corresponding image data, the tilt angle θ at the time of photographing, and the roll angle φ at the time of photographing. There is no need to convert the movement information at the time of shooting, such as the direction of the camera unit 210 at the time of shooting, the tilt angle θ at the time of shooting, and the roll angle φ at the time of shooting, and the information from the UAV 2 can be used as is. Further, if necessary, information indicating the tilt pitch angle at the time of photographing and the roll pitch angle at the time of photographing can also be held.

The inspection site image data folder 106 is created in a storage medium of a predetermined storage device, and stores and holds a plurality of inspection site image data files. Each inspection site image data file formed in the inspection site image data folder 106 stores and holds image data (photographed images) included in the photographic data accumulated in the normalized data file 105, for each inspection site. . FIG. 8 is a diagram for explaining an example of an image data file classified by inspection part that is formed in the image data folder classified by inspection part 106 of the photographed image classification device 1. As shown in FIG. 8, the inspection site-specific image data file of this embodiment has identification information of the inspection site and name of the inspection site as header information. Based on the inspection part identification information and the inspection part name, it is possible to recognize which inspection part the information is about.

In addition, as shown in FIG. 8, the image data file for each inspection site indicates, as header information, the scheduled imaging position (planned imaging area) specified for each inspection site as explained using FIGS. 4 and 5. information and information indicating planned movement information (range of planned movement information). The information specified through the above-mentioned inspection area matching device 3 is provided for each inspection area, and an image data file for each inspection area is formed accordingly and recorded in the image data folder for each inspection area 106. be done. The inspection site-specific image data file can store and hold a plurality of image data (photographed images) for each inspection site. In FIG. 8, descriptions such as "0001.jpg" and "0002.jpg" indicate captured images (still image data) data compressed using the JPEG (Joint Photographic Experts Group) method.

Therefore, based on the normalized imaging position (x, y, z) of the normalized data file 105 and the movement information at the time of imaging (tilt angle θ at the time of imaging, roll angle φ at the time of imaging), the image data folder 106 is created for each inspection site. Refer to the header section of the image data file for each inspected area. In this case, the planned shooting area and the planned movement information range (planned tilt angle α and the range of the planned roll angle β). In this case, it can be identified that the image data file by inspection site corresponds to the image data (captured image) included in the normalized data to be processed, and the photographed image is stored in the image data file by inspection site. Such classification processing of photographed images is performed by an image classification processing unit 112 for each inspection site, which will be described later.

Note that each of the reference point data file 103, imaging data file 104, normalized data file 105, and image data folder by inspection site 106 is stored in an area on a recording medium of a predetermined storage device consisting of a recording medium such as an SSD and its driver. It can be formed by changing. Furthermore, each file or folder can be created on a different recording medium, or files and folders can be distributed and created on several recording media. Furthermore, it is of course possible to create files and folders in the semiconductor memory included in the photographed image classification device 1. Further, by removing the semiconductor memory mounted on the UAV 2 and in which the photographed data file 204 is formed and connecting it to the photographed image classification device 1, it is also possible to use this as the photographed data file 104.

The connection terminal 107T and the external I/F (Inter Face) 107 constitute a connection end with an external device such as the inspection site association device 3 or the reference point surveying device 4, for example. Thereby, through the connection terminal 107T and the external I/F 107, information on the planned imaging area and planned movement for each inspection area is provided from the inspection area association device 3, and an image data file for each inspection area is formed. It can be recorded (registered) in the data folder 106. Similarly, measurement values can be received from the reference point surveying device 4 through the connection terminal 107T and the external I/F 107 and recorded in the reference point data file 103. Further, through the connection terminal 107T and the external I/F 107, it is possible to receive photographic data from the UAV 2 and record it in the photographic data file 104.

Further, the photographed image classification device 1 includes a photographed image classification processing section 110 that includes a normalization processing section 111 and an image classification processing section 112 for each inspection site. The normalization processing unit 111 sets the longitude, latitude, and altitude indicating the shooting position of the shooting data (FIG. 3) of the shooting data file 104 to the reference point P as the origin using the reference point data of the reference point data file 103. Convert to relative coordinates (x, y, z) in relative coordinate space. The normalization processing unit 111 uses the captured image (image data), the converted relative coordinates (x, y, z), the orientation of the camera unit 210 of the captured data, the tilt angle θ at the time of shooting, and the roll angle φ at the time of shooting. The normalized data (FIG. 7) is formed and recorded in the normalized data file 105.

The inspection part-specific image classification processing unit 112 classifies each of the inspection part-specific image data folders 106 based on the normalized imaging position, tilt angle θ at the time of imaging, and roll angle φ at the time of imaging of the normalized data of the normalized data file 105. Refer to the header section of the image data file for each inspection area. As explained using FIG. 8, the image data file for each area to be inspected includes information indicating the planned imaging area consisting of the specified range of x, y, and z coordinates as header information, and the planned tilt of the specified range. Angle α and planned roll angle β are recorded.

For this reason, it is assumed that there exists a scheduled photographing area to which the normalized photographing position, tilt angle θ at the time of photographing, roll angle φ at the time of photographing belong, a range of the planned tilt angle α, and a range of the planned roll angle β. In this case, the image data file for each inspection area is specified as the image data file for each inspection area of the corresponding inspection area, and the image of the normalized data to be processed is added to the specified image data file for each inspection area. Record data (photographed images). Thereby, based on the imaging position and the movement information of the camera unit 210 at the time of imaging, the imaging region can be specified and the captured images can be classified for each imaging region.

[Summary of processing of photographed image classification device 1]
Next, the classification process of photographed images performed by the photographed image classification apparatus 1 will be summarized, including preprocessing such as setting information to the photographic image classification apparatus 1 and photographing the inspected region by the UAV 2. FIG. 9 is a flowchart for explaining a series of processes when inspecting an inspection part of an object to be inspected (gantry crane).

First, for the gantry clay that is the object to be inspected, the planned photographing position and planned movement information are associated with the inspected parts (step S101). In step S101, as explained using FIG. 1, FIG. 4, and FIG. For each inspection site, a scheduled imaging position and scheduled camera movement information are specified. As mentioned above, the planned shooting position is specified as the planned shooting area that includes the planned shooting position, and the planned movement information of the camera is specified as the range of the planned tilt angle α and the range of the planned roll angle β of the camera unit 210. be done.

Next, the inspection area associating device 3 classifies the scheduled imaging area including the scheduled imaging position specified for each inspection area, the range of the planned tilt angle α and the range of the planned roll angle β of the camera unit 210 into the captured image classification. Provided to device 1. In the captured image classification device 1, the control unit 101 functions to perform a process of forming an image classification file for each inspection site (FIG. 8) and registering it in the image data folder for each inspection site 106 (step S102).

Next, the longitude, latitude, and altitude of the determined reference point P are measured using the reference point surveying device 4 (step S103), and the longitude, latitude, and altitude of the reference point P obtained by surveying are photographed. It is set (registered) in the reference point data file 103 of the image classification device 1 (step S104). Note that the surveying process in step S103 can be performed in parallel with the processes in steps S101 to S102 once the reference point P is determined. Through the processing of steps S101 to S104, preparations for photographing the inspection site of the gantry crane, which is the object to be inspected, and classifying the photographed images are completed.

Next, the UAV 2 is used to take an aerial photograph of the inspection site and output a photographed image (image data) and log information (step S105). In the UAV 2, the control unit 202 functions to capture images (image data) taken through the camera unit 210, the shooting position at the time of shooting (longitude, latitude, altitude), and movement information of the camera unit 210 at the time of shooting (tilt at the time of shooting). The angle θ and the roll angle φ at the time of photographing are obtained. The control unit 202 forms photographic data (FIG. 3) having this information and records it in the photographic data file 204. As a result, photographic data of various inspection parts are accumulated in the photographic data file 204.

Thereafter, when the UAV 2 finishes photographing the inspection site, the photographic data accumulated in the photographic data file 204 of the UAV 2 is input to the photographic data file 104 of the photographed image classification device 1 (step S106). Thereby, the photographed image classification device 1 is ready for classification processing of the photographed images. Thereafter, the photographed image classification process in the photographed image classification device 1 is executed (step S107), and the photographed images (image data) of the photographic data are classified for each photographed part, and the series of processes shown in the flowchart shown in FIG. 9 are performed. finish.

[Classification processing of photographed images in photographed image classification device 1]
FIG. 10 is a flowchart for explaining the photographed image classification process executed by the photographed image classification apparatus 1 in step S107 of the process shown in FIG. The process shown in the flowchart shown in FIG. 10 is a process executed by the control unit 101 of the photographed image classification device 1. First, the control unit 101 controls the normalization processing unit 111 to execute normalization processing in steps S201 to S203.

That is, the normalization processing unit 111 of the photographed image classification device 1 reads photographic data from the photographic data file 104 under the control of the control unit 101 (step S201), and determines whether all photographic data has been read and processed. It is determined whether or not (step S202). In the determination process of step S202, when it is determined that not all photographic data has been read out (unprocessed photographic data exists), normalization processing is executed (step S203).

In the normalization process in step S203, the photographing position (longitude, latitude, altitude) of the read photographic data is converted into relative coordinates (x, y, z) in a relative coordinate space with the reference point P as the origin, and normalized. The data (FIG. 7) is formed and recorded in the normalized data file 105. After step S203, the processing from step S201 is repeated. Assume that in the determination processing in step S202, it is determined that all photographic data has been read and processed. In this case, the control unit 101 controls the image classification processing unit 112 by inspection site to perform the classification processing of the captured images in steps S204 to S209.

Under the control of the control unit 101, the inspection site-based image classification processing unit 112 reads normalized data from the normalized data file 105 (step S204), and determines whether all normalized data has been read and processed. (Step S205). In the determination process of step S205, when it is determined that not all the normalized data has been read out (unprocessed normalized data exists), the captured image classification process is executed (step S206).

In step S206, based on the normalized imaging position (x, y, Refer to the image data file for each inspection site in the image data folder 106. The inspection part-specific image classification processing unit 112 determines whether or not there is an inspection part-specific image data file in which a scheduled imaging area including the imaging position and movement information at the time of imaging and a range of the planned movement information are set (step S207 ).

Assume that in the determination processing in step S207, it is determined that a corresponding image data file by inspection site exists. In this case, the inspection site-specific image classification processing unit 112 adds the captured image (image data) of the normalized data to be processed to the inspection site-specific image data file (step S208). Further, assume that in the determination processing in step S207, it is determined that the corresponding image data file by inspection site does not exist. In this case, the inspection site-specific image classification processing unit 112 adds the photographed image (image data) of the normalized data to be processed to the non-target inspection site-specific image data file that aggregates the non-target photographed images (step S209).

After the processing in steps S208 and S209, the processing from step S204 is repeated. In the determination process of step S205, when it is determined that all normalized data have been processed, the process of the flowchart shown in FIG. 10 is ended. Thereby, the photographed images photographed by the UAV 2 can be automatically classified by inspection site without requiring the operator to visually classify the images.

In this way, as a preliminary preparation, (1) the planned photographing position and the planned movement information of the camera unit 210 are identified in a relative coordinate space with a predetermined reference point as the origin in association with the inspected part of the gantry crane, and the photograph is taken. The image classification device 1 is set. As described above, the planned photographing position is set as a scheduled photographing area with a range, and the planned tilt angle α and planned roll angle β as planned movement information are also set with a range. Information consisting of the set scheduled imaging area and range of scheduled movement information functions as inspection site-specific setting information. (2) Accurately measure the longitude, latitude, and altitude of a predetermined reference point and set them in the photographed image classification device 1.

The photographing process includes (3) identifying a photographing position (longitude, latitude, altitude) according to the planned photographing position in the relative coordinate space, positioning the UAV 2 at the photographing position, and photographing the inspection site. In this case, the photographic data that is formed includes the longitude, latitude, and altitude indicating the photographing position for the photographed image (image data), the tilt angle θ at the time of photographing, and the roll angle at the time of photographing as movement information of the camera unit 210 at the time of photographing. φ is associated.

The classification process of photographed images is performed by (4) The photographed image classification device 1 calculates the longitude, latitude, and altitude of the photographing position of the photographed data of each inspection site formed by photographing through the UAV 2, with the reference point P as the origin. Convert to relative coordinates (x, y, z) in the relative coordinate space. (5) Based on the imaging position converted to relative coordinates and the movement information of the camera unit 210 at the time of imaging, each inspection region image data file in the inspection region image data folder 106 is referred to, and the corresponding inspection region image data file is , and store the captured image (image data).

In this case, there is an image data file for each inspection area that has the planned shooting position (planned shooting area) to which the shooting position, tilt angle θ and roll angle φ belong, the range of the planned tilt angle α, and the range of the planned roll angle β. This is the file that should be specified. In this way, photographed images can be appropriately classified by inspection site. In addition, by performing image analysis processing using artificial intelligence on the captured images after classification, we can analyze the state of rust and aging, identify areas that require immediate repair, and perform repairs quickly and accurately. It becomes possible to take appropriate measures.

[Effects of embodiment]
In the case of the photographed image classification device 1 of the embodiment described above, the inspection site is photographed assuming that the inspection target is located in a relative coordinate space whose origin is a predetermined reference point, and photographed data including the photographed image is generated. form. The photographed images of the inspection site photographed in this manner can be classified by inspection site based on the photographing position in the relative coordinate space and the movement information of the camera unit 210 at the time of photographing. Therefore, regardless of the installation location of the inspection object, no matter what structure it is, the inspection area can be photographed using the same method, and the photographed images can be edited appropriately without the need for human intervention. can be classified and managed.

In addition, the planned shooting position and camera's planned movement information can be set as a range, such as the planned shooting area and the range of the planned movement information, so it is possible to prevent the influence of the natural environment such as wind and the remote control (manual operation) of the UAV2. It can absorb variations in position and camera movement information. Therefore, it is possible to classify photographed images by inspection site with a certain degree of latitude.

Further, since the captured images can be classified by inspection site based on the imaging position in the relative coordinate space and the movement information of the camera, there is no restriction on the order of imaging.

For these reasons, especially when inspecting a plurality of objects to be inspected that have similar structures, images can be taken for each inspection site using the same method, and the captured images can be classified according to the inspection site. This makes it possible to significantly reduce the number of man-hours involved in classifying, managing, and analyzing captured images. Of course, when inspecting fixed structures such as bridges, radio towers, and Ferris wheels, the process of photographing the inspected area, classifying, managing, and analyzing the photographed images can be simplified, making it possible to significantly reduce man-hours. Become.

[Modifications, etc.]
In the above-described embodiment, the case where the object to be inspected is a gantry crane has been described as an example, but the invention is not limited to this. The present invention can be used when inspecting any of movable common structures such as gantry cranes and transfer cranes, fixed common structures such as power transmission line towers, and fixed unique structures such as bridges. Moreover, the object to be inspected is not limited to steel structures, and the present invention can be used when inspecting structures formed of various materials, such as wooden structures and reinforced concrete structures.

Further, as described above, the UAV 2 is also capable of detecting the tilt pitch angle and roll pitch angle at the time of photographing. Therefore, if the inspection site is photographed tilted forward or backward or left or right due to the influence of wind or remote control operations, the tilt angle θ at the time of photographing must be corrected by taking into account the tilt pitch angle. Alternatively, the roll angle φ at the time of photographing can be corrected in consideration of the roll pitch angle. Thereby, the tilt angle θ at the time of photographing and the roll angle φ at the time of photographing can be grasped as the angles when the UAV 2 is photographed while keeping it horizontal, and the photographed images can be classified more appropriately. Note that the correction processing of the tilt angle θ at the time of photographing and the roll angle φ at the time of photographing can be performed by the control unit 101 of the photographed image classification device 1, for example. Of course, apart from the control unit 101, a correcting means for motion information at the time of photographing may be provided.

Furthermore, in the embodiment described above, the tilt angle θ and the roll angle φ are used as the movement information of the camera unit 210 during shooting, but the present invention is not limited to this. For example, if the camera unit 210 is capable of changing the panning angle (action of changing direction in the horizontal direction (left and right direction)), in addition to the tilt angle θ during shooting and the roll angle φ during shooting, It is also possible to configure the camera to take into account the panning angle ω when photographing.

For example, if you want to photograph not only an inspection area that is located at a different position vertically from the scheduled imaging position (planned imaging area) but also an inspection area that is located at a different position horizontally, the planned tilt angle α is used as the scheduled movement information. In addition to the planned roll angle β, the planned pan angle γ is also set. Furthermore, when shooting with the UAV2, the pan angle ω at the time of shooting is also detected as movement information at the time of shooting, and the pan angle ω at the time of shooting is included in the shooting data as movement information at the time of shooting, in addition to the tilt angle θ at the time of shooting and the roll angle φ at the time of shooting. The angle ω is also detected and added to the photographic data.

Therefore, when classifying photographed images, in addition to the photographing position, the tilt angle θ at the time of photographing, and the roll angle φ at the time of photographing, the pan angle ω at the time of photographing can also be taken into consideration. Thereby, it becomes possible to classify the photographed images, which are obtained by photographing inspection parts at different positions to the left and right from the scheduled imaging position (planned imaging area), by inspection part. In this case, the panning angle ω as the planned movement information can also be set within a range.

Note that, unlike the camera unit 210 of the UAV 2 in the embodiment described above, the camera unit itself is not capable of changing its direction in the horizontal direction, and the direction of the UAV 2 may be changed. In this case, the pan angle can be understood as the direction of the UAV 2.

Further, the planned enlargement/reduction ratio (zoom magnification) can also be used as judgment information for classifying photographed images. That is, in addition to the planned photographing area and the planned movement angle range of the camera section, a planned enlargement/reduction ratio is set for each inspection site. When the inspection site is photographed using the camera section 210 of the UAV 2, the magnification/reduction ratio at the time of photographing of the camera section 210 is acquired and added to the photographic data. Thereby, when classifying the photographed images by the inspected part image classification processing unit 112 of the photographed image classification device 1, it is possible to perform classification processing of the photographed images that also takes into consideration the enlargement/reduction ratio at the time of photographing. In this case, the planned enlargement/reduction ratio (zoom magnification) can also be set within a range.

In this way, the planned tilt angle α, planned roll angle β, planned pan angle γ can be used as the planned movement information of the camera unit, and furthermore, the planned enlargement/reduction ratio (zoom magnification) can also be taken into consideration. That is, the inspection site-specific setting information includes a scheduled imaging position (planned imaging area) and scheduled camera movement information. As the planned movement information of the camera, one or more of a planned tilt angle α, a planned roll angle β, a planned pan angle γ, and a scaling ratio can be used.

At the time of shooting, one or more of the following: tilt angle θ during shooting, roll angle φ during shooting, pan angle ω during shooting, scaling ratio during shooting is set according to the planned shooting position (planned shooting area) and the planned movement information of the camera. All you have to do is acquire it and add it to the photographic data. This makes it possible to classify photographed images according to the photographing position and the movement information of the camera unit. In addition to the angle information and scaling ratio described above, various types of information that can be used to classify captured images can be used as camera movement information. For example, information included in EXIF information acquired when the camera unit 210 takes a picture is used as camera movement information for classifying the shot image, and scheduled camera movement information is determined accordingly. It is also possible to keep it.

Further, in the photographed image classification device 1 of the above-described embodiment, the image data file for each inspection site, which includes a scheduled imaging area and scheduled movement information for each inspection site, is prepared in advance in the inspection site image data folder 106. Photographed images are classified and recorded for each inspection site image data file in the inspection site image data folder 106. However, it is not limited to this.

For example, a classification data file is prepared in advance that stores and holds classification data in which inspection part identification information, scheduled imaging areas, and scheduled movement information are associated with each other. In this case, based on the imaging position and movement information at the time of imaging of the imaging data normalized by the normalization processing unit 111, inspection part identification information is identified with reference to the classification data. This specified inspection part identification information is added to the photographed image (image data) to be classified, and each different inspection part identification information is recorded in a different image data file. Even in this case, image data can be classified according to the inspection site. In short, in the relative coordinate space, various methods can be used to classify captured images based on scheduled imaging positions and scheduled movement information for each inspection site.

In addition, as mentioned above, in preparation for the photographing process of the inspected parts, the scheduled photographing position and planned movement information are set for each inspected part, and accordingly, flight instruction data for the autonomously flying UAV 2 is created, It can be provided to UAV2. That is, the flight instruction data can be formed using the scheduled imaging position and the scheduled movement information of the camera unit for each inspection area set in preparation for the inspection area photography process, and the UAV 2 can also fly autonomously using this. .

[others]
As can be seen from the above description of the embodiment, the function of the setting information storage means in the claims is realized by the image data folder by inspection site 106 of the photographed image classification device 1. Further, the function of the reference point storage means in the claims is realized by the reference point data file 103 of the photographed image classification device 1, and the function of the photographic data storage means in the claims is realized by the photographic data file 104 of the photographed image classification device 1. It has been realized. Further, the function of the normalization means in the claims is realized by the normalization processing section 111 of the photographed image classification device 1, and the function of the image classification means in the claims is realized by the image classification processing section for each inspection part of the photographic image classification device 1. 112 has been realized. Further, the correction means in the claims can be realized by, for example, the control unit 101 of the photographed image classification device 1.

Further, the method described using the flowchart of FIG. 10 corresponds to one embodiment of the photographed image classification method. Further, by creating a program that executes the processing shown in the flowchart of FIG. 10 and making it executable on, for example, a personal computer, the personal computer can function as the photographed image classification apparatus according to the present invention. That is, the functions of the normalization processing unit 111 and the image classification processing unit 112 by inspection site of the captured image classification device 1 shown in FIG. 6 can be realized as the functions of the control unit 101 by a program executed by the control unit 101. You can also do it.

1... Captured image classification device, 101... Control unit, 102... Storage device, 103... Reference point data file, 104... Captured data file, 105... Normalized data file, 106... Image data folder by inspection site, 107... External I/ F, 107T... Connection terminal, 110... Captured image classification processing section, 111... Normalization processing section, 112... Image classification processing section by inspection part, 2... UAV, 210... Camera section, 210P... Support column, 202... Control section, 204...Photographing data file, 3...Inspection area matching device, 4...Reference point surveying device, P...Reference point, a1, a2, b1, b2...Inspection area, PL, PR...Planned imaging position, ArL, ArR...Plan Shooting area

Claims (7)

For each of one or more inspection parts of the inspection target, setting information for each inspection part is stored, which associates a scheduled imaging position in a relative coordinate space with a predetermined reference point as the origin and scheduled movement information of the imaging means. a setting information storage means for
Reference point storage means for storing the longitude, latitude, and altitude of the reference point;
Image data obtained by photographing through the imaging means, which is information acquired when photographing the inspection part of the inspection object, longitude, latitude, and altitude indicating the photographing position, and the time of photographing by the imaging means. Photographic data storage means for storing photographic data associated with movement information;
normalizing means for converting the longitude, latitude, and altitude of the photographic data into a photographing position in the relative coordinate space using the longitude, latitude, and altitude of the reference point storage means;
Based on the converted imaging position of the imaging data and the motion information at the time of imaging of the imaging data, the inspection area-specific setting information in the setting information storage means is referred to, and the inspection area is identified and inspected. A photographed image classification device comprising: image classification means for recording the image data of the photographed data on a recording medium for each image data. The photographed image classification device according to claim 1,
The photographed image classification device, wherein the scheduled motion information and the motion information at the time of photographing are at least a tilt angle and a roll angle of an imaging means. The photographed image classification device according to claim 2,
The photographing data is formed in a UAV (unmanned aerial vehicle) equipped with the imaging means, and the motion information at the time of photographing includes a tilt pitch angle indicating the front and rear tilt of the UAV , and a left and right tilt. and a roll pitch angle indicated;
A photographed image classification device comprising: a correction unit that corrects the tilt angle using the tilt pitch angle, and corrects the roll angle using the roll pitch angle. The photographed image classification device according to claim 2 or 3,
The photographed image classification device, wherein the scheduled motion information and the motion information at the time of photographing include a panning angle of an imaging means. The photographed image classification device according to claim 2, claim 3, or claim 4,
The photographed image classification device, wherein the scheduled motion information and the motion information at the time of photographing include an enlargement/reduction ratio of an imaging means. A photographed image classification device according to any one of claims 1, 2, 3, 4, or 5,
The photographed image classification device, wherein the scheduled photographing position and the scheduled movement information are information that specifies a range. For each of one or more inspection parts of the inspection target, setting information for each inspection part is stored, which associates a scheduled imaging position in a relative coordinate space with a predetermined reference point as the origin and scheduled movement information of the imaging means. a reference point storage means for storing the longitude, latitude, and altitude of the reference point; and information obtained when photographing the inspection part of the inspection object through the imaging means. and a photographic data storage means for storing photographic data in which the image data obtained by the photographing method is associated with the longitude, latitude, altitude indicating the photographing position, and movement information of the imaging means at the time of photographing. A photographed image classification method,
a normalization step in which the normalization means converts the longitude, latitude, and altitude of the photographic data into a photographing position in the relative coordinate space using the longitude, latitude, and altitude of the reference point storage means;
The image classification means refers to the setting information for each inspection part in the setting information storage means based on the converted imaging position of the imaging data and the movement information at the time of imaging of the imaging data, and selects the inspection part. A photographic image classification method comprising: an image classification step of specifying and recording the image data of the photographic data on a recording medium for each inspection site.

Images