JP7664130B2 - Satellite image analysis system and satellite image analysis method - Google Patents

Description

The present invention generally relates to the analysis of satellite images.

As a result of the large amount of satellite images being taken and improvements in image analysis technology, it is now possible to analyze satellite images using deep learning. For such analysis, it is desirable to use high-resolution satellite images.

In this regard, there is known an image processing device that can perform super-resolution processing to obtain a high-resolution image using a low-resolution image (see Patent Document 1).

JP 2010-161760 A

According to the technology described in Patent Document 1, by taking the difference between the analysis results of low-resolution satellite imagery and super-resolution satellite imagery, it is possible to detect analysis results that could not be confirmed in low-resolution satellite imagery, as well as unexpected changes that occur due to super-resolution processing. However, with the technology described in Patent Document 1, when many differences are detected, the user needs to check all of the differences, which leads to human error, high human costs, etc.

The present invention was made in consideration of the above points, and aims to propose a satellite image analysis system etc. that can obtain the differences in the extracted objects.

In order to solve this problem, the present invention provides an input unit for inputting a satellite image, a processing unit for generating a super-resolution satellite image of the satellite image input by the input unit using a super-resolution model for generating a super-resolution satellite image of the satellite image, an analysis unit for analyzing the satellite image input by the input unit and the super-resolution satellite image of the satellite image generated by the processing unit using an analysis model for performing analysis to obtain predetermined information from the satellite image, an acquisition unit for acquiring the difference of the extraction target from the simple difference between the analysis result of the satellite image analyzed by the analysis unit and the analysis result of the super-resolution satellite image of the satellite image analyzed by the analysis unit using a difference acquisition model for acquiring the difference of a predetermined extraction target from the simple difference between the analysis result of the satellite image and the analysis result of the super-resolution satellite image of the satellite image, and an output unit for outputting information indicating the difference of the extraction target acquired by the acquisition unit.

In the above configuration, for example, the difference of a predetermined extraction target is automatically extracted, so that the user can easily understand the difference of the predetermined extraction target. With the above configuration, the difference of the extraction target can be automatically extracted, reducing user confirmation errors, confirmation costs, etc.

The present invention makes it possible to realize a highly convenient satellite image analysis system.

FIG. 1 is a diagram illustrating an example of a hardware configuration of a satellite image analysis system according to a first embodiment. 1 is a diagram illustrating an example of a functional configuration of a satellite image analysis system according to a first embodiment. FIG. 11 is a diagram showing an example of a super-resolution model table according to the first embodiment; FIG. 4 is a diagram illustrating an example of an analysis model table according to the first embodiment. FIG. 11 is a diagram illustrating an example of a difference acquisition model table according to the first embodiment. FIG. 2 is a diagram illustrating an example of a processing flow according to the first embodiment. FIG. 11 is a diagram showing an example of a result output GUI according to the first embodiment. FIG. 11 is a diagram showing an example of a result output GUI according to the first embodiment. FIG. 11 is a diagram showing an example of a result output GUI according to the first embodiment. FIG. 11 is a diagram showing an example of a result output GUI according to the first embodiment. FIG. 11 is a diagram illustrating an example of a functional configuration of a satellite image analysis system according to a second embodiment. FIG. 11 is a diagram illustrating an example of a target resolution table according to the second embodiment. FIG. 11 is a diagram illustrating an example of a processing flow according to the second embodiment.

(I) First Embodiment Hereinafter, one embodiment of the present invention will be described in detail. However, the present invention is not limited to the embodiment.

The satellite image analysis system of this embodiment includes a processor and a storage device. The satellite image analysis system stores one or more super-resolution models, one or more analysis models, and one or more difference acquisition models in the storage device. For example, the satellite image analysis system receives a satellite image. The satellite image analysis system generates a super-resolution satellite image from the satellite image using the super-resolution model. The satellite image analysis system analyzes the satellite image and the super-resolution satellite image using the analysis model. The satellite image analysis system obtains the difference of the extraction target from the simple difference between the analysis result of the satellite image and the analysis result of the super-resolution satellite image using the difference acquisition model. The satellite image analysis system outputs information indicating the difference of the extraction target.

Next, an embodiment of the present invention will be described with reference to the drawings. The following description and drawings are examples for explaining the present invention, and some parts have been omitted or simplified as appropriate for clarity of explanation. The present invention can also be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

In the following explanation, the same elements in the drawings are given the same numbers and the explanation is omitted as appropriate. When explaining similar elements without distinguishing between them, the common part (part excluding the branch number) of the reference code including the branch number is used, and when explaining similar elements with distinction, the reference code including the branch number may be used. For example, when explaining analysis information without making a distinction, it may be written as "analysis information 711", and when explaining each piece of analysis information with distinction, it may be written as "analysis condition 711-1" and "display analysis result selection section 711-2".

The designations "first," "second," "third," and the like in this specification are used to identify components and do not necessarily limit the number or order. Furthermore, numbers for identifying components are used in different contexts, and a number used in one context does not necessarily indicate the same configuration in another context. Furthermore, this does not prevent a component identified by a certain number from also serving the function of a component identified by another number.

In FIG. 1, 100 indicates the satellite image analysis system according to the first embodiment as a whole.

Figure 1 is a diagram showing an example of the hardware configuration of a satellite image analysis system 100. The satellite image analysis system 100 is one or more general-purpose computers, one or more server devices, etc. The satellite image analysis system 100 is configured to include, as components, an input device 110, a processor 120 (e.g., a CPU: Central Processing Unit), a graphics processor 130 (e.g., a GPU: Graphics Processing Unit), a storage device 140, a memory 150, and an output device 160. These components are connected via a communication medium 170 such as a bus. The program of this embodiment and data required to execute the program are stored in the storage device 140.

The satellite image analysis system 100 accepts instructions from a user via an input device 110 such as a mouse or keyboard. In response to the user's instructions, the processor 120 reads the program and data required for processing from the storage device 140 into the memory 150 and executes the program. Since satellite images are typically large-scale image data, the image processing load is large. In this regard, in this embodiment, image processing can be assisted by transferring image data to the memory of the graphics processor 130.

The output device 160 is, for example, a display (display device), and outputs the difference between the extracted objects before and after super-resolution of the satellite image analysis results, which are the processing results of this embodiment, the analysis results of the super-resolution satellite image, the analysis results of the satellite image, the super-resolution satellite image, the satellite image, etc.

Note that FIG. 1 merely shows a representative hardware configuration, and various modifications are possible. For example, the programs and data may be stored in multiple storage devices, or may be stored in a storage device external to the satellite image analysis system 100 via a network. In addition, a user may access the satellite image analysis system 100 from an external terminal via the network.

Figure 2 is a diagram showing an example of the functional configuration of the satellite image analysis system 100. The satellite image analysis system 100 includes an input/output unit 210, a storage unit 220, and a calculation unit 230.

The input/output unit 210 includes an input unit 211 and an output unit 212. The input unit 211 is a function related to the input device 110, and the output unit 212 is a function related to the output device 160. The input unit 211 receives inputs such as satellite images, types of analysis (analysis methods), and types of differences of the extracted objects from the user via the input device 110. The input unit 211 performs the processing of steps S610 to S630 described below. The output unit 212 displays the differences of the extracted objects before and after super-resolution of the analysis results of the satellite images on the output device 160. Note that the output unit 212 may display, in addition to the differences of the extracted objects, for example, simple differences, analysis results of super-resolution satellite images, analysis results of satellite images, super-resolution satellite images, satellite images, etc. The output unit 212 performs the processing of step S690 described below.

The storage unit 220 is configured to include a super-resolution model DB 221, an analysis model DB 222, a difference acquisition model DB 223, a satellite image DB 224, and a super-resolution satellite image DB 225. The super-resolution model DB 221 stores a super-resolution model and a super-resolution model table. The super-resolution model table will be described using FIG. 3. The analysis model DB 222 stores an analysis model and an analysis model table. The analysis model table will be described using FIG. 4. The difference acquisition model DB 223 stores a difference acquisition model table. The difference acquisition model table will be described using FIG. 5. The satellite image DB 224 stores satellite images and information related to the satellite images (satellite image ID, super-resolution model ID, satellite image acquisition date and time, satellite image path, etc.). Super-resolution satellite image DB225 stores super-resolution satellite images and information related to the super-resolution satellite images (ID of the super-resolution satellite image, ID of the satellite image used to generate the super-resolution satellite image, path of the super-resolution satellite image, etc.).

The calculation unit 230 includes a processing unit 231, an analysis unit 232, and an acquisition unit 233. The processing unit 231 receives the satellite image input by the input unit 211 and determines whether or not a super-resolution satellite image of the satellite image exists in the super-resolution satellite image DB 225. If the super-resolution satellite image exists, the processing unit 231 reads the super-resolution satellite image from the super-resolution satellite image DB 225. If the super-resolution satellite image does not exist, the processing unit 231 super-resolutions the satellite image using the super-resolution model acquired from the super-resolution model DB 221 to generate a super-resolution satellite image. The processing unit 231 stores the generated super-resolution satellite image in the super-resolution satellite image DB 225 in association with the ID of the satellite image used to generate the super-resolution satellite image. The processing unit 231 performs the processes of steps S640 to S660 of the processing flow of this embodiment.

The analysis unit 232 receives the satellite image and the type of analysis input by the input unit 211, receives the super-resolution satellite image from the processing unit 231, obtains an analysis model corresponding to the satellite image, the type of analysis, and the super-resolution satellite image from the analysis model DB 222, analyzes the satellite image and the super-resolution satellite image using the analysis model, and generates an analysis result of the satellite image and an analysis result of the super-resolution satellite image. The analysis unit 232 performs the process of step S670 described below.

The acquisition unit 233 receives the type of analysis and the type of difference to be extracted input by the input unit 211, and receives the analysis results of the satellite image and the analysis results of the super-resolution satellite image from the analysis unit 232. The acquisition unit 233 acquires a difference acquisition model corresponding to the type of analysis and the type of difference to be extracted from the difference acquisition model DB 223, and acquires the difference to be extracted between the analysis results of the satellite image and the analysis results of the super-resolution satellite image using the difference acquisition model. The acquisition unit 233 performs the process of step S680 described below.

The functions of the satellite image analysis system 100 (the input/output unit 210, the storage unit 220, the calculation unit 230, etc.) may be realized, for example, by the processor 120 reading a program stored in the storage device 140 into the memory 150 and executing it (software), or may be realized by hardware such as the graphic processor 130 or a dedicated circuit, or may be realized by a combination of software and hardware. Note that one function of the satellite image analysis system 100 may be divided into multiple functions, or multiple functions may be combined into one function. Also, some of the functions of the satellite image analysis system 100 may be provided as a separate function, or may be included in another function. Also, some of the functions of the satellite image analysis system 100 may be realized by another computer that can communicate with the satellite image analysis system 100.

Figure 3 is a diagram showing an example of a super-resolution model table (super-resolution model table 300) stored in the super-resolution model DB 221. The super-resolution model table 300 stores data such as a super-resolution model ID 310, a type of satellite image 320, a band of the satellite image 330, a super-resolution magnification 340, and a super-resolution model path 350. Each record in the super-resolution model table 300 represents information on a super-resolution model (e.g., a trained neural network) for super-resolution processing.

The data of the super-resolution model ID 310 is an identifier that can identify the super-resolution model. The data of the satellite image type 320 is, for example, "WorldView-3" or "GeoEye-1". The data of the satellite image band 330 is, for example, "panchromatic band" or "multispectral band". The data of the super-resolution magnification 340 is, for example, "2" or "3". The data of the super-resolution model path 350 is data that indicates the location where the super-resolution model is stored.

Figure 4 is a diagram showing an example of an analytical model table (analysis model table 400) stored in the analytical model DB 222. The analytical model table 400 stores data such as an analytical model ID 410, a type of satellite image 420, a band of the satellite image 430, a super-resolution magnification 440, an analysis type 450, and an analytical model path 460. Each record in the analytical model table 400 represents information about an analytical model (e.g., a trained neural network) for the analysis process.

The data in the analysis model ID 410 is an identifier that can identify the analysis model. The satellite image type 420, satellite image band 430, and super-resolution magnification 440 are the same as those stored in the super-resolution model table 300. The data in the analysis type 450 is, for example, "semantic segmentation" or "object detection." The data in the analysis model path 460 is data that indicates the location where the analysis model is stored.

Figure 5 is a diagram showing an example of a difference acquisition model table (difference acquisition model table 500) stored in the difference acquisition model DB 223. The difference acquisition model table 500 stores data such as a difference acquisition model ID 510, an analysis type 520, and a type of difference to be extracted 530. Each record in the difference acquisition model table 500 represents a difference acquisition model for difference acquisition processing (e.g., a processing method included in a program).

The data in the difference acquisition model ID 510 is an identifier capable of identifying the difference acquisition model. The data in the analysis type 520 is the same as that stored in the analysis model table 400. The data in the type of difference to be extracted 530 is, for example, "newly detected object", "sharpened contour", etc. when the analysis type is "semantic segmentation", and "newly detected object", "bounding box whose size has changed", etc. when the analysis type is "object detection".

The difference acquisition model may be a trained neural network. In this case, the difference acquisition model DB 223 includes the difference acquisition model, and the difference acquisition model table 500 includes path data of the difference acquisition model (data indicating the location where the difference acquisition model is stored).

Figure 6 is a diagram showing an example of the processing flow of this embodiment. The operation based on the processing flow of Figure 6 is as follows.

Step S610: The satellite image analysis system 100 inputs a satellite image, the type of satellite image, and the band of the satellite image. The satellite image may be any image that indicates the geographical spatial position of each pixel of the image. Examples of satellite images include WorldView-3 panchromatic images and WorldView-3 multispectral images.

For example, the satellite image analysis system 100, in response to the user's operation of the input device 110, accesses a storage location (e.g., a cloud service) where a satellite image specified by a URL or the like is stored, and acquires the satellite image. In this case, the satellite image analysis system 100 presents the user with options for the type of satellite image and the band of the satellite image. The satellite image analysis system 100 inputs the type of satellite image and the band of the satellite image corresponding to the satellite image selected by the user. The options for the type of satellite image and the band of the satellite image are acquired from the type of satellite image 320 and the band of the satellite image 330 stored in the super-resolution model table 300. Additionally, the acquired satellite image may be stored in the satellite image DB 224.

For example, the satellite image analysis system 100 may obtain a satellite image from the satellite image DB 224 for which the file name of the satellite image, the ID of the satellite image, etc. have been specified by the user. When obtaining a satellite image from the satellite image DB 224, the type of satellite image and the band of the satellite image are automatically input from the satellite image DB 224.

Step S620: The satellite image analysis system 100 inputs the type of analysis. For example, the satellite image analysis system 100 presents a choice of analysis types to the user. The choice of analysis types is obtained from the analysis types 450 stored in the analysis model table 400. At this time, the analysis types presented in the choices must correspond to the type of satellite image and the band of the satellite image input in step S610. The satellite image analysis system 100 inputs the type of analysis selected by the user. The types of analysis are "semantic segmentation", "object detection", etc.

Step S630: The satellite image analysis system 100 inputs the type of difference to be extracted. For example, the satellite image analysis system 100 presents the user with options for the type of difference to be extracted. The options for the type of difference to be extracted are acquired from the type of difference to be extracted 530 stored in the difference acquisition model table 500. At this time, the type of difference to be extracted presented in the options must correspond to the type of analysis input in step S620. The satellite image analysis system 100 inputs the type of difference to be extracted selected by the user. For example, if the type of analysis input in step S620 is "semantic segmentation", the type of difference to be extracted is "newly detected object", "sharpened contour", etc.

Step S640: The satellite image analysis system 100 determines whether or not a super-resolution satellite image (e.g., a super-resolution satellite image in which the satellite image ID matches the type of satellite image, satellite image band, and super-resolution magnification linked to the super-resolution model ID) obtained by super-resolution of the satellite image input in step S610 exists in the super-resolution satellite image DB 225. If the satellite image analysis system 100 determines that the super-resolution satellite image does not exist, it proceeds to step S650, and if it determines that the super-resolution satellite image exists, it proceeds to step S660. Super-resolution is a process that places a high load on the system, but step S640 can reduce the number of times the super-resolution process is performed.

Step S650: The satellite image analysis system 100 obtains a super-resolution model corresponding to the type of satellite image and the band of the satellite image input in step S610 from the super-resolution model DB 221 based on the super-resolution model table 300. The satellite image analysis system 100 uses the super-resolution model to super-resolve the satellite image input in step S610 and generate a super-resolution satellite image. If there are multiple corresponding super-resolution models, the satellite image analysis system 100 uses the super-resolution model with the highest super-resolution magnification. Super-resolution is performed, for example, by inputting a satellite image into a trained neural network. The super-resolution satellite image has the same band as the satellite image. The resolution of the super-resolution satellite image is higher than that of the input satellite image, but the super-resolution satellite image and the input satellite image show the same geographical area. The generated super-resolution satellite image may be stored in the super-resolution satellite image DB 225 together with the ID of the super-resolution satellite image, the ID of the input satellite image, and the ID of the super-resolution model used.

Step S660: The satellite image analysis system 100 obtains a super-resolution satellite image, which is a super-resolution version of the satellite image input in step S610, from the super-resolution satellite image DB 225.

Step S670: The satellite image analysis system 100 acquires an analysis model corresponding to the type of satellite image and band of the satellite image input in step S610 and the type of analysis input in step S620 from the analysis model DB 222 based on the analysis model table 400. The satellite image analysis system 100 uses the analysis model to analyze the satellite image input in step S610 and the super-resolution satellite image generated in step S650 or the super-resolution satellite image acquired in step S660, respectively, and generates an analysis result of the satellite image and an analysis result of the super-resolution satellite image. The analysis is performed, for example, by inputting the satellite image and the super-resolution satellite image to a trained neural network. For example, when the type of analysis is "semantic segmentation," the analysis result of the satellite image and the analysis result of the super-resolution satellite image are class maps in which each pixel is classified into a class.

Step S680: The satellite image analysis system 100 acquires the ID of the difference acquisition model corresponding to the type of analysis input in step S620 and the type of difference of the extraction target input in step S630 from the difference acquisition model table 500. The satellite image analysis system 100 uses the difference acquisition model to acquire the difference of the extraction target from the simple difference between the analysis result of the satellite image generated in step S660 and the analysis result of the super-resolution satellite image.

Since the analysis results of the satellite image and the super-resolution satellite image have different resolutions, when acquiring the difference, the satellite image analysis system 100 upsamples the analysis results of the satellite image using, for example, the nearest neighbor method to make the resolution uniform. An example of a method for acquiring the difference of the extraction target will be described. Note that the acquired difference of the extraction target has geospatial information, just like the satellite image and the super-resolution satellite image.

For example, if the type of analysis is "semantic segmentation" and the type of difference to be extracted is "newly detected object," the satellite image analysis system 100 first obtains the simple difference between the analysis results of the satellite image with the same resolution and the analysis results of the super-resolution satellite image. Next, the satellite image analysis system 100 examines the surroundings of the pixel where a difference exists in the analysis results for both the satellite image analysis results and the super-resolution satellite image analysis results. If continuity between the class of the pixel and the classes of the surrounding pixels exists only in one of the analysis results, the satellite image analysis system 100 determines that "the pixel is a newly detected object" and obtains the pixel as the difference to be extracted.

For example, if the type of analysis is "semantic segmentation" and the type of difference to be extracted is "sharpened contours," the satellite image analysis system 100 first obtains the simple difference between the analysis results of the satellite image with the same resolution and the analysis results of the super-resolution satellite image. Next, the satellite image analysis system 100 examines the surroundings of a pixel where a difference exists in the analysis results for both the satellite image analysis results and the super-resolution satellite image analysis results. If continuity exists between the class of the pixel and the classes of the surrounding pixels in both analysis results, the satellite image analysis system 100 determines that "the pixel in question is a sharpened contour" and obtains the pixel as the difference to be extracted.

For example, if the type of analysis is "object detection" and the type of difference to be extracted is "newly detected object," the satellite image analysis system 100 first compares the analysis results of the satellite image with the same resolution with the analysis results of the super-resolution satellite image, and pairs bounding boxes whose degree of match is higher than a predetermined value. If there are any remaining bounding boxes that are not paired, the satellite image analysis system 100 determines that "the bounding box in question is a newly detected object," and obtains the pixels related to that bounding box as the difference to be extracted.

For example, if the type of analysis is "object detection" and the type of difference to be extracted is "bounding box of changed size," the satellite image analysis system 100 first compares the analysis results of the satellite image with the same resolution with the analysis results of the super-resolution satellite image, and pairs bounding boxes whose degree of agreement is higher than a predetermined value. The satellite image analysis system 100 compares the bounding boxes of each pair, and if there is a misalignment in the coordinates (pixels of the bounding boxes), the satellite image analysis system 100 determines that the bounding box is "a bounding box of changed size," and acquires the pixels related to the bounding box as the difference to be extracted.

Step S690: The satellite image analysis system 100 outputs information indicating the differences in the extracted objects obtained in step S680 (e.g., the result output GUI shown in Figures 7 to 10).

Figure 7 is a diagram showing an example of a result output GUI (a screen when the difference of the extraction target is displayed) in this embodiment. This result output GUI is configured to include a window 700. The window 700 is configured to include a display result selection section 710 and a result display section 720.

The display result selection section 710 includes analysis information 711. The analysis information 711 includes analysis conditions 711-1 and a display analysis result selection section 711-2. By operating the check boxes in the analysis information 711 and the display analysis result selection section 711-2, the user can select the analysis results to be displayed in the result display section 720. In FIG. 7, "Result 1" is selected in the analysis information 711, and "Difference of extracted subject" is selected in the display analysis result selection section 711-2, so the result display section 720 displays the difference of the extracted subject of result 1.

In the analysis conditions 711-1, the path of the satellite image input by the user, the type of analysis, and the type of difference to be extracted are displayed. In the display analysis result selection section 711-2, an item for displaying the difference to be extracted is displayed. In the display analysis result selection section 711-2, an item for displaying the analysis results of the satellite image and/or the analysis results of the super-resolution satellite image may be displayed. In addition, in the display analysis result selection section 711-2, an item for displaying the satellite image and/or the super-resolution satellite image may be displayed.

In the result display section 720 of FIG. 7, the difference of the extracted object in result 1 is displayed. When the difference of the extracted object is displayed, the result display section 720 is configured to include a set of pixels 721 where there is a difference of the extracted object and a set of pixels 722 where there is no difference of the extracted object. In FIG. 7, the type of difference of the extracted object is "newly detected object", so by checking the set of pixels 721 where there is a difference of the extracted object, it is possible to check the object newly detected by super-resolution. Since the difference has geospatial information, it may be displayed superimposed on a satellite image, a super-resolution satellite image, etc., which also has geospatial information.

Figure 8 is a diagram showing an example of a result output GUI (a screen when the analysis results of a super-resolution satellite image are displayed) in this embodiment. The differences from Figure 7 will be described below. In Figure 8, "Analysis results of super-resolution satellite image" is selected in the display analysis result selection unit 711-2, so the analysis results of the super-resolution satellite image of result 1 are displayed in the result display unit 720. In Figure 8, the type of analysis is "semantic segmentation", so the result display unit 720 is configured to include, for example, a set 801 of pixels classified as buildings, a set 802 of pixels classified as roads, and a set 803 of pixels classified as ground.

Figure 9 is a diagram showing an example of a result output GUI (a screen when the analysis results of a satellite image are displayed) in this embodiment. The differences from Figure 7 will be described. In Figure 9, since "Satellite Image Analysis Results" is selected in the display analysis result selection unit 711-2, the analysis results of the satellite image of result 1 are displayed in the result display unit 720. Note that Figure 9 displays an image in which the analysis results of the satellite image are upsampled using the nearest neighbor method. In Figure 9, since the type of analysis is "semantic segmentation", the result display unit 720 includes, for example, a set 901 of pixels classified as a building, a set 902 of pixels classified as a road, and a set 903 of pixels classified as the ground. Comparing Figures 8 and 9, it can be seen that the analysis results of the satellite image and the analysis results of the super-resolution satellite image are different.

Figure 10 is a diagram showing an example of a result output GUI (a screen in which a simple difference and an extraction target difference are displayed in a distinguishable manner) in this embodiment. The result output GUI in Figure 10 may be adopted instead of the result output GUI in Figure 7. In Figure 10, "Result 1" is selected in the analysis information 711, and "Extraction target difference" is selected in the display analysis result selection unit 711-2, so that the result display unit 720 displays the extraction target difference of result 1. In Figure 10, the type of analysis is "semantic segmentation" and the type of extraction target difference is "newly detected object", so that the result display unit 720 includes, for example, a set 1001 of pixels with an extraction target difference (a newly detected object), a set 1002 of pixels with a simple difference, and a set 1003 of pixels with no difference.

(II) Second embodiment The satellite image analysis system of this embodiment is characterized in that it accepts input of a target object to be analyzed (analysis target object) and an analysis purpose (analysis purpose), holds a target resolution DB, and selects and determines the super-resolution magnification. In the following, the same components as those in the first embodiment are given the same reference numerals, and their explanations are omitted as appropriate.

Figure 11 is a diagram showing an example of the functional configuration of a satellite image analysis system 1100 according to this embodiment. An input unit 1111 of the input/output unit 1110 receives inputs of a satellite image, a type of analysis, a type of difference to be extracted, an analysis target, and an analysis purpose from a user. The input unit 1111 carries out steps S610 to S630 and step S1310 of the processing flow according to this embodiment.

The storage unit 1120 includes a super-resolution model DB 221, an analysis model DB 222, a difference acquisition model DB 223, a satellite image DB 224, a super-resolution satellite image DB 225, and a target resolution DB 1121. A target resolution table is stored in the target resolution DB 1121. The target resolution table will be described with reference to FIG. 12.

The calculation unit 1130 includes a processing unit 231, an analysis unit 232, an acquisition unit 233, and a determination unit 1131. The determination unit 1131 receives the satellite image input by the input unit 1111, acquires a target resolution corresponding to the analysis target and the analysis purpose from the target resolution DB 1121, and determines the super-resolution magnification from the resolution of the satellite image and the target resolution. The determination unit 1131 performs step S1320 of the processing flow of this embodiment.

Figure 12 is a diagram showing an example of a target resolution table (target resolution table 1200) stored in target resolution DB 1121. Target resolution table 1200 is a table showing the resolution (target resolution) required when analyzing an analysis target in a satellite image for a specific analysis purpose, and stores the target resolution for each analysis target 1210 and analysis purpose 1220. Data for analysis target 1210 is, for example, "vehicle", "airplane", and "ship". Data for analysis purpose 1220 is, for example, "detection", "overview", and "classification". In the example of Figure 12, for example, the target resolution for detecting a vehicle is "1.5m".

Figure 13 shows an example of the processing flow of this embodiment.

Step S1310: The satellite image analysis system 1100 inputs an analysis target and an analysis purpose. For example, the satellite image analysis system 1100 presents a selection of analysis targets and analysis purposes to the user. The selection of analysis targets and analysis purposes is obtained from the analysis targets 1210 and analysis purposes 1220 stored in the target resolution table 1200. The satellite image analysis system 1100 inputs the analysis target and analysis purpose selected by the user. An example of an analysis target and analysis purpose is vehicle detection.

Step S1320: The satellite image analysis system 1100 determines the super-resolution magnification. For example, the satellite image analysis system 1100 obtains a target resolution corresponding to the analysis target and analysis purpose input in step S1310 from the target resolution table 1200. The satellite image analysis system 1100 determines the super-resolution magnification from the resolution of the satellite image input in step S610 and the target resolution. For example, if the analysis target is a "vehicle" and the analysis purpose is "detection", the target resolution is "1.5 m". In this case, for example, when the resolution of the satellite image is "6.0 m", the satellite image analysis system 1100 determines "4 (= 6.0 ÷ 1.5)" as the super-resolution magnification.

Step S1330: The satellite image analysis system 1100 generates a super-resolution satellite image using the super-resolution model. For example, the satellite image analysis system 1100 acquires a super-resolution model corresponding to the type of satellite image and band of the satellite image input in step S610 and the super-resolution magnification determined in step S1320 from the super-resolution model DB 221 based on the super-resolution model table 300. The satellite image analysis system 1100 uses the super-resolution model to super-resolve the satellite image input in step S610 and generate a super-resolution satellite image.

The present invention is not limited to the above-described embodiments, but includes various modified examples. For example, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace part of the configuration of each embodiment with another configuration. Furthermore, each of the above configurations may be configured in part or in whole as hardware, or may be configured to be realized by executing a program on a processor.

(III) Supplementary Notes The above-described embodiment includes, for example, the following contents.

In the above embodiment, the present invention has been described as being applied to a satellite image analysis system, but the present invention is not limited to this and can be widely applied to a variety of other systems, devices, methods, and programs.

In addition, in the above-described embodiment, a part or all of the program may be installed from a program source into a device such as a computer that realizes the satellite image analysis system. The program source may be, for example, a program distribution server connected via a network or a computer-readable recording medium (e.g., a non-transitory recording medium). In the above description, two or more programs may be realized as one program, and one program may be realized as two or more programs.

In addition, in the above-described embodiment, the configuration of each table is an example, and one table may be divided into two or more tables, or all or part of two or more tables may be one table.

In addition, in the above embodiment, for convenience of explanation, the information related to the satellite image analysis system is described using a table, but the data structure is not limited to a table. The information related to the satellite image analysis system may be expressed using a data structure other than a table, such as XML (Extensible Markup Language), YAML (YAML Ain't a Markup Language), a hash table, a tree structure, etc.

In addition, the screens shown and described in the above embodiment are merely examples, and any design may be used as long as the information received is the same.

In addition, the screens shown and described in the above embodiment are merely examples, and any design may be used as long as the information presented is the same.

In addition, in the above-described embodiment, the output of information is not limited to display on a display. The output of information may be audio output from a speaker, output to a file, printing on paper media using a printing device, projection on a screen using a projector, or other forms.

In addition, in the above description, information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC card, an SD card, or a DVD.

The above-described embodiment has the following characteristic configurations:

(1)
A satellite image analysis system (e.g., satellite image analysis system 100, 1100) includes an input unit (e.g., input unit 211, 1111, input/output unit 210, 1110, circuit, computer capable of communicating with satellite image analysis system 100, 1100) that inputs a satellite image, a processing unit (e.g., processing unit 231, calculation unit 230, 1130, circuit, computer capable of communicating with satellite image analysis system 100, 1100) that generates a super-resolution satellite image of the satellite image input by the input unit using a super-resolution model that generates a super-resolution satellite image of the satellite image, and an analysis unit (e.g., analysis unit 232, calculation unit 230, 1130, circuit, computer capable of communicating with satellite image analysis system 100, 1100) that analyzes the satellite image input by the input unit and analyzes the super-resolution satellite image of the satellite image generated by the processing unit using an analysis model that performs analysis to obtain predetermined information from the satellite image. an acquisition unit (e.g., acquisition unit 233, calculation unit 230, 1130, circuit, computer capable of communicating with satellite image analysis system 100, 1100) that acquires a difference of the extraction target from a simple difference between the analysis result of the satellite image analyzed by the analysis unit and the analysis result of the super-resolution satellite image of the satellite image analyzed by the analysis unit using a difference acquisition model that acquires a difference of a predetermined extraction target from a simple difference between the analysis result of the satellite image and the analysis result of a super-resolution satellite image of the satellite image; and an output unit (e.g., output unit 212, input/output unit 210, 1110, circuit, computer capable of communicating with satellite image analysis system 100, 1100) that outputs information indicating the difference of the extraction target acquired by the acquisition unit (e.g., see Figures 7 to 10).

In the above configuration, for example, the difference of a predetermined extraction target is automatically extracted, so that the user can easily understand the difference of the predetermined extraction target. With the above configuration, the difference of the extraction target can be automatically extracted, reducing user confirmation errors, confirmation costs, etc.

(2)
The satellite image analysis system includes a memory unit (e.g., memory unit 220, 1120, difference acquisition model DB 223, circuit, computer capable of communicating with satellite image analysis system 100, 1100) that stores a difference acquisition model for each type of difference to be extracted, and the input unit inputs the type of difference to be extracted specified by the user (e.g., step S630), and the acquisition unit uses the difference acquisition model corresponding to the type of difference to be extracted input by the input unit to acquire the difference to be extracted from a simple difference between the analysis result of the satellite image analyzed by the analysis unit and the analysis result of the super-resolution satellite image of the satellite image analyzed by the analysis unit (e.g., step S680).

According to the above configuration, for example, the differences in the extraction target that interests the user are automatically extracted, so the user can easily understand the differences in the extraction target that interests him/her.

(3)
The types of differences to be extracted are newly detected objects, sharpened edges, or bounding boxes that have changed size (see, for example, FIG. 5).

In the above configuration, the acquisition unit acquires a newly detected object, acquires a sharpened contour, or acquires a bounding box whose size has changed as the difference of the extraction target. With the above configuration, for example, the user can easily grasp a newly detected object, a sharpened contour, or a bounding box whose size has changed.

(4)
The satellite image analysis system includes a memory unit (e.g., memory unit 220, 1120, analysis model DB 222, circuit, computer capable of communicating with satellite image analysis system 100, 1100) that stores an analysis model for each type of analysis, and the input unit inputs the type of analysis specified by the user (e.g., step S620), and the analysis unit uses the analysis model corresponding to the type of analysis input by the input unit to analyze the satellite image input by the input unit and to analyze the super-resolution satellite image of the satellite image generated by the processing unit (e.g., step S670).

In the above configuration, for example, the user can specify the type of analysis, and can understand the differences in the extracted objects in the analysis results of the desired analysis.

(5)
The type of analysis is semantic segmentation or object detection (see, for example, FIG. 4).

In the above configuration, for example, a user can understand the differences in extracted objects in the analysis results obtained by semantic segmentation, and can understand the differences in extracted objects in the analysis results obtained by object detection.

(6)
The memory unit stores a first difference acquisition model corresponding to semantic segmentation and a second difference acquisition model corresponding to object detection (for example, see Figure 5), and the first difference acquisition model is a model that evaluates continuity between a pixel that has a simple difference between the analysis result of a satellite image on which semantic segmentation has been performed by the analysis unit and the analysis result of a super-resolution satellite image of the satellite image on which semantic segmentation has been performed by the analysis unit and the pixels surrounding the pixel, and acquires the difference of the extraction target based on the presence or absence of continuity, and the second difference acquisition model is a model that compares a first bounding box of a satellite image on which object detection has been performed by the analysis unit and a second bounding box of a super-resolution satellite image of the satellite image on which object detection has been performed by the analysis unit, and acquires the difference of the extraction target based on the degree of match between the first bounding box and the second bounding box.

(7)
The satellite image analysis system (for example, satellite image analysis system 1100) includes a determination unit (for example, determination unit 1131, calculation unit 1130, circuit, computer capable of communicating with satellite image analysis system 1100) that determines the magnification of super-resolution, and a storage unit (for example, storage unit 1120, super-resolution model DB 221 and target resolution DB 1121, circuit, computer capable of communicating with satellite image analysis system 1100) that stores a super-resolution model (for example, see FIG. 3) corresponding to the magnification of super-resolution and stores resolution information (for example, see FIG. 12) indicating the resolution corresponding to the target object to be analyzed and the purpose of the analysis, and the input unit stores a resolution information specified by a user. The determination unit inputs the target of analysis and the purpose of the analysis input by the input unit from the resolution information stored in the memory unit, and determines a super-resolution magnification from the specified resolution and the resolution of the satellite image input by the input unit (e.g., step S1320). The analysis unit uses a super-resolution model corresponding to the super-resolution magnification determined by the determination unit to analyze the satellite image input by the input unit and the super-resolution satellite image of the satellite image generated by the processing unit (e.g., step S1330).

For example, if the magnification of super-resolution is increased too much, unnecessary objects that should not be seen may become visible. In this regard, in the above configuration, the magnification of super-resolution is determined according to the target of analysis, thereby making such unnecessary objects invisible. In addition, if the magnification of super-resolution is increased, processing time is also increased. In this regard, according to the above configuration, the magnification of super-resolution is determined according to the target of analysis, thereby making it possible to reduce processing time. In addition, according to the above configuration, for example, when the processing unit determines that the resolution of the satellite image satisfies the conditions of analysis from the target to be analyzed and the magnification of super-resolution required from the purpose of the analysis, the processing unit may not perform super-resolution.

(8)
The satellite image analysis system includes a memory unit (e.g., memory unit 220, 1120, satellite image DB 224, circuit, computer capable of communicating with satellite image analysis system 100, 1100) that stores a plurality of satellite images, and the input unit inputs information for identifying a satellite image to be analyzed that is specified by a user from the plurality of satellite images, and retrieves the satellite image from the memory unit (e.g., step S610).

In the above configuration, there is no need to input satellite images from outside the satellite image analysis system, and satellite images are input from a storage unit inside the satellite image analysis system, which can reduce the time it takes to input satellite images, for example.

(9)
The satellite image analysis system includes a memory unit (e.g., memory unit 220, 1120, super-resolution satellite image DB 225, circuit, computer capable of communicating with satellite image analysis system 100, 1100) that stores the super-resolution satellite image generated by the processing unit, and the processing unit determines whether or not the super-resolution satellite image of the satellite image input by the input unit is stored in the memory unit (e.g., step S640), and if it determines that it is not stored, generates a super-resolution satellite image of the satellite image (e.g., step S650, step S1330).

With the above configuration, it is not necessary to generate a super-resolution satellite image every time, so for example, the processing time can be shortened for satellite images for which a super-resolution satellite image has already been generated.

(10)
The output unit displays information indicating the difference between the extracted objects acquired by the acquisition unit from a simple difference between the analysis result of the satellite image analyzed by the analysis unit and the analysis result of the super-resolution satellite image of the satellite image analyzed by the analysis unit on a display device (e.g., output device 160, a computer capable of communicating with satellite image analysis systems 100, 1100) superimposed on the satellite image, the super-resolution satellite image, the analysis result of the satellite image, or the analysis result of the super-resolution satellite image (e.g., the difference between the extracted objects in FIG. 7 may be superimposed on the analysis result of the super-resolution satellite image in FIG. 8, or the difference between the extracted objects in FIG. 7 may be superimposed on the analysis result of the satellite image in FIG. 9).

In the above configuration, the difference in the extracted object is displayed superimposed on the satellite image, the super-resolution satellite image, the analysis result of the satellite image, or the analysis result of the super-resolution satellite image, so that, for example, the user can more easily grasp the difference in the extracted object.

(11)
The output unit displays the simple difference and the difference to be extracted on a display device (for example, the output device 160, a computer capable of communicating with the satellite image analysis systems 100, 1100) in a distinguishable manner (for example, see FIG. 10).

In the above configuration, simple differences and differences to be extracted are distinguished and displayed, so that, for example, the user can understand the difference between simple differences and differences to be extracted.

Furthermore, the above-mentioned configurations may be modified, rearranged, combined, or omitted as appropriate without departing from the spirit and scope of the present invention.

It should be understood that an item in a list in the format "at least one of A, B, and C" can mean (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C). Similarly, an item in a list in the format "at least one of A, B, or C" can mean (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C).

100: satellite image analysis system, 211: input unit, 212: output unit, 231: processing unit, 232: analysis unit, 233: acquisition unit.

Claims (12)

an input unit for inputting satellite images;
a processing unit that generates a super-resolution satellite image of the satellite image input by the input unit using a super-resolution model that generates a super-resolution satellite image of the satellite image;
an analysis unit that uses an analysis model to perform analysis to obtain predetermined information from a satellite image, and analyzes the satellite image input by the input unit and a super-resolution satellite image of the satellite image generated by the processing unit;
an acquisition unit that acquires a difference of an extraction target from a simple difference between an analysis result of a satellite image analyzed by the analysis unit and an analysis result of a super-resolution satellite image of the satellite image, using a difference acquisition model that acquires a difference of a predetermined extraction target from a simple difference between an analysis result of a satellite image and an analysis result of a super-resolution satellite image of the satellite image;
an output unit that outputs information indicating the difference between the extraction objects acquired by the acquisition unit;
A satellite image analysis system comprising: a storage unit that stores a difference acquisition model for each type of difference to be extracted,
The input unit inputs a type of difference to be extracted that is designated by a user,
the acquisition unit acquires the difference of the extraction target from a simple difference between an analysis result of the satellite image analyzed by the analysis unit and an analysis result of a super-resolution satellite image of the satellite image analyzed by the analysis unit, using a difference acquisition model corresponding to the type of difference of the extraction target input by the input unit;
2. The satellite image analysis system according to claim 1. The types of differences to be extracted are newly detected objects, sharpened edges, or bounding boxes that have changed in size.
The satellite image analysis system according to claim 2. A storage unit is provided for storing an analysis model for each type of analysis,
The input unit inputs a type of analysis designated by a user,
The analysis unit analyzes the satellite image input by the input unit and analyzes the super-resolution satellite image of the satellite image generated by the processing unit, using an analysis model corresponding to the type of analysis input by the input unit.
The satellite image analysis system according to claim 1 . The type of analysis is semantic segmentation or object detection.
5. The satellite image analysis system according to claim 4. The storage unit stores a first difference acquisition model corresponding to semantic segmentation and a second difference acquisition model corresponding to object detection;
the first difference acquisition model is a model that evaluates continuity between a pixel having a simple difference between an analysis result of a satellite image on which semantic segmentation has been performed by the analysis unit and an analysis result of a super-resolution satellite image of the satellite image on which semantic segmentation has been performed by the analysis unit and pixels surrounding the pixel, and acquires a difference to be extracted based on the presence or absence of continuity;
the second difference acquisition model is a model that compares a first bounding box of a satellite image on which object detection has been performed by the analysis unit with a second bounding box of a super-resolution satellite image of the satellite image on which object detection has been performed by the analysis unit, and acquires a difference of an extraction target based on a degree of coincidence between the first bounding box and the second bounding box.
The satellite image analysis system according to claim 5. A determination unit that determines a magnification of super-resolution;
A storage unit stores a super-resolution model corresponding to a magnification of the super-resolution, and stores resolution information indicating a resolution corresponding to a target object to be analyzed and a purpose of the analysis,
The input unit inputs an analysis target and an analysis purpose designated by a user,
The determination unit specifies a resolution corresponding to the target object of the analysis and the purpose of the analysis input by the input unit from the resolution information stored in the storage unit, and determines a super-resolution magnification from the specified resolution and the resolution of the satellite image input by the input unit;
The analysis unit analyzes the satellite image input by the input unit and analyzes the super-resolution satellite image of the satellite image generated by the processing unit, using a super-resolution model corresponding to the super-resolution magnification determined by the determination unit.
2. The satellite image analysis system according to claim 1. A storage unit for storing a plurality of satellite images,
the input unit inputs information for identifying a satellite image to be analyzed that is designated by a user from the plurality of satellite images, and acquires the satellite image from the storage unit;
2. The satellite image analysis system according to claim 1. a storage unit that stores the super-resolution satellite image generated by the processing unit,
the processing unit determines whether or not a super-resolution satellite image of the satellite image input by the input unit is stored in the storage unit, and when determining that the super-resolution satellite image of the satellite image is not stored, generates a super-resolution satellite image of the satellite image.
2. The satellite image analysis system according to claim 1. the output unit displays, on a display device, information indicating a difference between the extracted objects acquired by the acquisition unit from a simple difference between the analysis result of the satellite image analyzed by the analysis unit and the analysis result of the super-resolution satellite image of the satellite image analyzed by the analysis unit, superimposing the information on the satellite image, the super-resolution satellite image, the analysis result of the satellite image, or the analysis result of the super-resolution satellite image.
2. The satellite image analysis system according to claim 1. the output unit displays the simple difference and the extraction target difference on a display device in a distinguishable manner.
The satellite image analysis system according to claim 1 . An input unit inputs a satellite image;
A processing unit generates a super-resolution satellite image of the satellite image input by the input unit using a super-resolution model that generates a super-resolution satellite image of the satellite image;
an analysis unit, using an analysis model that performs analysis to obtain predetermined information from a satellite image, analyzes the satellite image input by the input unit, and analyzes a super-resolution satellite image of the satellite image generated by the processing unit;
an acquisition unit acquires a difference of an extraction target from a simple difference between an analysis result of a satellite image analyzed by the analysis unit and an analysis result of a super-resolution satellite image of the satellite image analyzed by the analysis unit, using a difference acquisition model that acquires a difference of a predetermined extraction target from a simple difference between an analysis result of a satellite image and an analysis result of a super-resolution satellite image of the satellite image;
an output unit outputs information indicating the difference between the extraction objects acquired by the acquisition unit;
A method for analyzing satellite images, including:

Images