JP7614130B2 - High resolution system, high resolution method, and high resolution program - Google Patents

Description

This disclosure relates to a high resolution system, a high resolution method, and a high resolution program.

Traditionally, ground sensors have been used to grasp the damage caused by disasters such as floods, heavy rains, and earthquakes. Methods for grasping damage caused by disasters using ground sensors are useful for grasping the damage situation at spots such as houses and power plants. However, when damage occurs over a wide area, it is difficult to obtain area-wide information such as which areas have suffered the most damage.

In recent years, analytical methods using remote sensing data have been proposed as a method for grasping the state of disaster damage occurring over a wide area. Specific examples of remote sensing include aircraft and UAVs (drones), as well as optical satellites and synthetic aperture radar (SAR) satellites.
The means for acquiring SAR images is a means for observing the earth's surface regardless of weather and time of day, and is used as a means for monitoring various objects on the earth's surface.

There is a technology that improves the ground resolution of an image by increasing the resolution of the image using a high-resolution model trained by machine learning. Here, the high-resolution model is also called a super-resolution model.
Patent Document 1 discloses a technology for increasing the resolution of an image using a high-resolution model, in which a high-resolution image is prepared as a teacher image, a low-resolution image generated by downsampling the high-resolution image is prepared as an input image, and a high-resolution model is generated by training a machine learning model using the data pair of the prepared high-resolution image and low-resolution image.

International Publication No. 2020/175446

In the technology disclosed in Patent Document 1, a high-resolution technology using machine learning generates a low-resolution image using downsampling as a degradation process. However, the degradation process that corresponds to an actual low-resolution image is often different from simple downsampling. Therefore, this technology has an issue in that the accuracy of the generated high-resolution image is reduced when an image actually acquired at low resolution is input to a trained model generated by this technology.

The present disclosure aims to provide a learning method for a high-resolution model for SAR images that generates high-resolution images with relatively high accuracy by inputting actual low-resolution images into a trained model.

The resolution enhancing system according to the present disclosure comprises:
The learning device includes a model generation unit that uses learning data consisting of a plurality of high-resolution full aperture images generated by performing an image reproduction process using pulses indicated by each of a plurality of complex radar images indicating pulses reflected in each of a plurality of irradiation areas, and a plurality of low-resolution sub-aperture images generated by performing an image reproduction process using some of the pulses used in generating each of the plurality of high-resolution full aperture images, to learn a learned model that infers an estimated high-resolution full aperture image, which is an image showing the target irradiation area and has a resolution higher than the resolution of the low-resolution full aperture image, from a low-resolution full aperture image, which is an image corresponding to pulses reflected in the target irradiation area and has a resolution equivalent to the resolution corresponding to the plurality of low-resolution sub-aperture images.

In the present disclosure, the model generation unit trains a trained model using training data consisting of a high-resolution full aperture image generated by performing an image reproduction process using a pulse reflected in the irradiation area, and a low-resolution sub-aperture image generated by performing an image reproduction process using a portion of the pulse used to generate the high-resolution full aperture image. Here, the low-resolution sub-aperture image can be said to be an image based on an actual degradation process. Therefore, according to the present disclosure, a training method for a high-resolution model for SAR images can be provided in which the accuracy of the generated high-resolution image is relatively high by inputting an actual low-resolution image into the trained model.

1 is a block diagram illustrating a schematic configuration of a high resolution system 1 according to a first embodiment. 1 is a block diagram illustrating a schematic configuration of an input image generating device 10 according to a first embodiment. 1A and 1B are diagrams for explaining image synthesis according to the first embodiment, where FIG. 1A is a diagram for explaining a full aperture image, and FIG. 1B is a diagram for explaining a sub-aperture image. FIG. 1 is a block diagram illustrating a schematic configuration of a learning device 20 according to a first embodiment. FIG. 1 is a diagram explaining a neural network. FIG. 1 is a block diagram showing a schematic configuration of an inference device 30 according to a first embodiment. FIG. 2 is a diagram showing an example of a hardware configuration of a learning device 20 according to the first embodiment. 4 is a flowchart showing the operation of the learning device 20 according to the first embodiment. 4 is a flowchart showing the operation of the inference device 30 according to the first embodiment. FIG. 13 is a diagram showing an example of a hardware configuration of a learning device 20 according to a modified example of the first embodiment.

In the description of the embodiments and the drawings, the same elements and corresponding elements are given the same reference numerals. Descriptions of elements given the same reference numerals are omitted or simplified as appropriate. Arrows in the drawings primarily indicate data flow or processing flow. In addition, each of "device" and "part" may be read as "circuit," "process," "procedure," "processing," or "circuitry" as appropriate.

Embodiment 1.
Hereinafter, the present embodiment will be described in detail with reference to the drawings.

***Configuration Description***
Fig. 1 is a block diagram showing an outline of a configuration example of a high resolution system 1 according to embodiment 1. As shown in Fig. 1, the high resolution system 1 includes an input image generating device 10 that generates an input image for learning, a learning device 20 that functions as a model generating device, and an inference device 30 that functions as an utilization device. The multiple devices that make up the high resolution system 1 may be configured integrally as appropriate.
In the resolution improvement system 1, the inference device 30 uses a trained model trained by the learning device 20 to improve the resolution of the low-resolution full aperture image DIN3.

FIG. 2 is a block diagram illustrating an example of the configuration of the input image generating device 10. As shown in FIG.
The input image generating device 10 includes an image reproducing unit 101 .
The image reconstructing unit 101 receives a complex radar image DIN0 as an input, and performs image reconstruction on the input complex radar image DIN0 to generate two types of amplitude images, a low-resolution sub-aperture image DIN1 and a high-resolution full aperture image DIN2. The image reconstructing unit 101 typically generates a plurality of high-resolution full aperture images DIN2 by performing image reconstruction processing using pulses indicated by the complex radar images DIN0, which indicate pulses reflected in each of a plurality of irradiation regions. Also, the image reconstructing unit 101 typically generates a plurality of low-resolution sub-aperture images DIN1 by performing image reconstruction processing using some of the pulses used when generating each of the plurality of high-resolution full aperture images DIN2.
The complex radar image DIN0 is an image showing the pulse reflected in the illuminated area and is an image showing the phase. The complex radar image may be an image captured by using either a synthetic aperture radar satellite or an airborne vehicle.

SAR is a technology that achieves the same resolution as when radio waves are transmitted and received using a single huge antenna by continuously emitting microwaves from a flying object as it moves and appropriately combining the waves reflected back from the same target. With SAR, a single point on the ground receives multiple pulses from a moving flying object.

Here, as shown in Fig. 3(a), a full aperture image is defined as an image generated by synthesizing all of the pulses reflected from the radar irradiation area and received by the antenna. In Fig. 3, an aircraft equipped with an antenna is moving in the direction of travel, and the antenna receives a pulse at the position where the antenna is depicted. A full aperture image does not necessarily have to be an image generated using all of the pulses strictly speaking.
Also, as shown in FIG. 3B, an image generated by synthesizing an image using a part of the pulses reflected from the radar irradiation area and received by the antenna is defined as a sub-aperture image. In FIG. 3B, of the three antenna positions shown, only the pulses received by the antenna at the antenna position filled in black are shown to synthesize an image. As a specific example, the part of the pulse indicates about 1/10 to 1/2 of the entire band of the pulses received by the antenna. The irradiation area is also called the irradiation surface.

Unlike a low-resolution image generated by downsampling a high-resolution SAR image, a sub-aperture image generated by image reproduction processing can be said to be an actually acquired SAR image. Therefore, generating a sub-aperture image is considered appropriate as a degradation process of a SAR image. As a specific example, the shape of the point image of the SAR image is maintained in the sub-aperture image, so the sub-aperture image can be said to be an actual SAR image. On the other hand, the point image of the SAR image is crushed in the downsampled image, so the downsampled image differs from the actual SAR image.

Fig. 4 is a block diagram showing an example of the configuration of the learning device 20. As shown in Fig. 4, the learning device 20 includes a data acquisition unit 201, a model generation unit 202, and a learned model storage unit 203. The learning method executed by the learning device 20 corresponds in particular to a learning method for high-resolution processing of remote sensing data.

The data acquisition unit 201 acquires learning data, and provides the acquired learning data to the model generation unit 202 .
The learning data is teacher data indicating each of a low-resolution sub-aperture image DIN1 and a high-resolution full-aperture image DIN2. The high-resolution full-aperture image DIN2 corresponds to a correct answer to be inferred from the low-resolution sub-aperture image DIN1. The learning data typically consists of a plurality of low-resolution sub-aperture images DIN1 and a plurality of high-resolution full-aperture images DIN2.

The model generation unit 202 learns a high-resolution full aperture image DIN2 corresponding to the low-resolution sub-aperture image DIN1 by using the learning data provided by the data acquisition unit 201. In other words, the model generation unit 202 executes machine learning by using a combination of the low-resolution sub-aperture image DIN1 and the high-resolution full aperture image DIN2 indicated by the learning data, thereby generating a learned model that infers an optimal high-resolution image corresponding to the low-resolution full aperture image DIN3.
The model generation unit 202 stores the generated trained model in the trained model storage unit 203.

The model generation unit 202 may use a known algorithm such as supervised learning, unsupervised learning, or reinforcement learning as a learning algorithm. Here, supervised learning is a method in which a pair of data indicating an input and data indicating a result (label) is provided to a learning device as learning data, and the device learns the features contained in the learning data and infers a result from the input. As an example, a case in which the model generation unit 202 uses a neural network will be described here.

Here, when the model generation unit 202 uses supervised learning, the learning data must include a combination of data consisting of a low-resolution sub-aperture image DIN1 and a high-resolution full-aperture image DIN2, which contains the same subject. Note that when the model generation unit 202 uses unsupervised learning, the low-resolution sub-aperture image DIN1 and the high-resolution full-aperture image DIN2 do not need to contain the same subject.

As a specific example, the model generating unit 202 learns a high-resolution full-aperture image DIN2 corresponding to the low-resolution sub-aperture image DIN1 by so-called supervised learning according to a neural network model.
A neural network is composed of an input layer made up of a plurality of neurons, an intermediate layer (hidden layer) made up of a plurality of neurons, and an output layer made up of a plurality of neurons. The number of intermediate layers may be one or more.

FIG. 5 shows a schematic diagram illustrating an example of a three-layer neural network.
In FIG. 5, when a plurality of input values are input from the input layer X1 to the input layer X3, each of the plurality of input values is multiplied by a first weight w11 to a first weight w16 (hereinafter also referred to as a first weight W1). Each calculated value, which is a value obtained by multiplying the input value by the first weight w11 to the first weight w16, is input to one of the intermediate layers Y1 and Y2. Each calculated value is multiplied by a second weight w21 to a second weight w26 (hereinafter also referred to as a second weight W2). Each output value, which is a value obtained by multiplying the calculated value by the second weight w21 to the second weight w26, is output from one of the output layers Z1 to Z3. Each output value is determined according to the value of the first weight W1 and the value of the second weight W2.

In this embodiment, the neural network learns a learned model for inferring an optimal high-resolution image corresponding to the low-resolution full-aperture image DIN3 by so-called supervised learning, according to learning data created based on a combination of a low-resolution sub-aperture image DIN1 and a high-resolution full-aperture image DIN2 indicated by the learning data acquired by the data acquisition unit 201.
That is, the model generation unit 202 uses a neural network to learn a learned model by adjusting the first weight W1 and the second weight W2 so that when a low-resolution sub-aperture image DIN1 is input to the input layer, the result output from the output layer approaches the correct high-resolution full-aperture image DIN2.

As shown in FIG. 4, the trained model storage unit 203 stores the trained model provided by the model generation unit 202.

Fig. 6 is a block diagram showing a schematic configuration example of the inference device 30. As shown in Fig. 6, the inference device 30 includes a learned model storage unit 203, a data acquisition unit 301, and an inference unit 302. The learned model storage unit 203 included in the inference device 30 may be the same as the learned model storage unit 203 included in the learning device 20. The inference device 30 executes the utilization method.
The inference device 30 uses the trained model provided by the learning device 20 to infer an estimated high-resolution full-aperture image DOUT from the low-resolution full-aperture image DIN3.
The low-resolution full aperture image DIN3 is an image generated based on an actually observed pulse, corresponds to a pulse reflected in a target irradiation area, and has a resolution equivalent to the resolution corresponding to the multiple low-resolution sub-aperture images DIN1. The resolution corresponding to the multiple low-resolution sub-aperture images DIN1 is, as a specific example, a resolution equal to or greater than the lowest resolution and equal to or less than the highest resolution among the resolutions possessed by each of the multiple low-resolution sub-aperture images DIN1. The resolution equivalent to the resolution corresponding to the multiple low-resolution sub-aperture images DIN1 is, as a specific example, a resolution indicated by the resolution corresponding to the multiple low-resolution sub-aperture images DIN1, or a resolution close to the resolution corresponding to the multiple low-resolution sub-aperture images DIN1.
The estimated high-resolution full aperture image DOUT is an image showing the target illumination area corresponding to the low-resolution full aperture image DIN3, is an image inferred from the low-resolution full aperture image DIN3 by the learned model, and has a higher resolution than the resolution of the low-resolution full aperture image DIN3.

The data acquisition unit 301 acquires a low-resolution full aperture image DIN3 and provides the acquired low-resolution full aperture image DIN3 to the inference unit 302. Here, the low-resolution full aperture image DIN3 represents the target subject.

The inference unit 302 infers an estimated high-resolution full-aperture image DOUT showing the target subject using the learned model stored in the learned model storage unit 203 and the low-resolution full-aperture image DIN3. In other words, the inference unit 302 obtains the estimated high-resolution full-aperture image DOUT by inputting the low-resolution full-aperture image DIN3 to the learned model.

Figure 7 shows an example of the hardware configuration of the learning device 20 according to this embodiment. The learning device 20 is composed of a computer. The learning device 20 may be composed of multiple computers.

As shown in the figure, the learning device 20 is a computer equipped with hardware such as a processor 51, a memory 52, an auxiliary storage device 53, an input/output IF (Interface) 54, and a communication device 55. These pieces of hardware are appropriately connected via signal lines 59.

The processor 51 is an integrated circuit (IC) that performs arithmetic processing and controls the hardware of the computer. Specific examples of the processor 51 include a central processing unit (CPU), a digital signal processor (DSP), and a graphics processing unit (GPU).
The learning device 20 may include a plurality of processors that replace the processor 51. The plurality of processors share the role of the processor 51.

The memory 52 is typically a volatile storage device, and a specific example is a RAM (Random Access Memory). The memory 52 is also called a primary storage device or a main memory. The data stored in the memory 52 is saved in the auxiliary storage device 53 as necessary.

The auxiliary storage device 53 is typically a non-volatile storage device, and specific examples thereof include a read only memory (ROM), a hard disk drive (HDD), or a flash memory. Data stored in the auxiliary storage device 53 is loaded into the memory 52 as necessary.
The memory 52 and the auxiliary storage device 53 may be integrated into one unit.

The input/output IF 54 is a port to which an input device and an output device are connected. As a specific example, the input/output IF 54 is a USB (Universal Serial Bus) terminal. As a specific example, the input device is a keyboard and a mouse. As a specific example, the output device is a display.

The communication device 55 is a receiver and a transmitter. A specific example of the communication device 55 is a communication chip or a NIC (Network Interface Card).

Each part of the learning device 20 may use the input/output IF 54 and the communication device 55 as appropriate when communicating with other devices, etc.

The auxiliary storage device 53 stores a high resolution program. The high resolution program is a program that causes a computer to realize the functions of each part of the learning device 20. The high resolution program is loaded into the memory 52 and executed by the processor 51. The functions of each part of the learning device 20 are realized by software.

Data used when executing the high resolution program and data obtained by executing the high resolution program are appropriately stored in the storage device. Each part of the learning device 20 uses the storage device as appropriate. As a specific example, the storage device is composed of at least one of the memory 52, the auxiliary storage device 53, a register in the processor 51, and a cache memory in the processor 51. The storage device may be independent of the computer.
The functions of the memory 52 and the auxiliary storage device 53 may be realized by other storage devices.

The high resolution program may be recorded in a computer-readable non-volatile recording medium. Specific examples of the non-volatile recording medium include an optical disk and a flash memory. The high resolution program may be provided as a program product.
The hardware configuration of each device in the high resolution system 1 may be similar to the hardware configuration of the learning device 20.

*** Operation Description ***
The operating procedure of the high resolution system 1 corresponds to a high resolution method. Moreover, a program that realizes the operation of the high resolution system 1 corresponds to a high resolution program. Moreover, the high resolution method is also a general term for methods executed in each device included in the high resolution system 1. The high resolution program is also a general term for programs executed in each device included in the high resolution system 1.

Figure 8 is a flowchart showing an example of the learning process executed by the learning device 20. The learning process will be explained with reference to this figure.

(Step S11)
The data acquisition unit 201 acquires learning data via the input image generation device 10 , and provides the acquired learning data to the model generation unit 202 .

(Step S12)
The model generation unit 202 generates a learned model by learning the high-resolution full aperture image DIN2 corresponding to the low-resolution sub-aperture image DIN1 using so-called supervised learning based on a combination of a low-resolution sub-aperture image DIN1 and a high-resolution full aperture image DIN2 indicated by the learning data.

(Step S13)
The trained model storage unit 203 stores the generated trained model.

Figure 9 is a flowchart showing an example of the inference process executed by the inference device 30. The inference process will be explained with reference to this figure. Hereinafter, it is assumed that the inference device 30 infers an estimated high-resolution full-aperture image DOUT corresponding to the low-resolution full-aperture image DIN3.

(Step S21)
The data acquisition unit 301 acquires a low-resolution full-aperture image DIN3, and provides the acquired low-resolution full-aperture image DIN3 to the inference unit 302.

(Step S22)
The inference unit 302 inputs the low-resolution full aperture image DIN3 to the learned model stored in the learned model memory unit 203, and obtains an estimated high-resolution full aperture image DOUT corresponding to the low-resolution full aperture image DIN3.

(Step S23)
The inference device 30 outputs the generated estimated high resolution full aperture image DOUT.

Note that, although the case where supervised learning is applied as the learning algorithm used by the model generation unit 202 has been described, the first embodiment is not limited to this case. As a specific example, the model generation unit 202 may use reinforcement learning, unsupervised learning, semi-supervised learning, or the like as the learning algorithm instead of supervised learning.

The model generation unit 202 may also learn a high-resolution full aperture image DIN2 corresponding to the low-resolution sub-aperture image DIN1 according to learning data created for multiple high-resolution systems 1 including the high-resolution system 1 having the model generation unit 202. The model generation unit 202 may learn using learning data acquired from multiple high-resolution systems 1 used in the same area, or may learn using learning data acquired from multiple high-resolution systems 1 operating independently in different areas.

In addition, the model generation unit 202 may add the high resolution system 1 that collects learning data to the targets from which learning data is obtained along the way, or may remove it from the targets from which learning data is obtained along the way.
Furthermore, the model generation unit 202 may apply a trained model that has been trained on a high-resolution full-aperture image DIN2 corresponding to a low-resolution sub-aperture image DIN1 for a certain high-resolution system 1 to a high-resolution system 1 other than the certain high-resolution system 1, and update the trained model by re-training the high-resolution full-aperture image DIN2 corresponding to the low-resolution sub-aperture image DIN1 for the other high-resolution system 1.

The model generation unit 202 may also use, as a learning algorithm, deep learning, which is a method for learning to extract features themselves. The model generation unit 202 may also perform machine learning according to other known methods, such as genetic programming, functional logic programming, or support vector machines, as specific examples.

Here, each of the multiple high-resolution full aperture images DIN2 is a target high-resolution full aperture image, and the low-resolution sub-aperture image DIN1 corresponding to the target high-resolution full aperture image is a target low-resolution sub-aperture image. At this time, it is desirable to determine the ratio between the bandwidth in the image reproduction process corresponding to the target high-resolution full aperture image and the bandwidth in the image reproduction process corresponding to the target low-resolution sub-aperture image based on the ratio between the spatial resolution of the target high-resolution full aperture image and the spatial resolution of the low-resolution full aperture image DIN3. In general, the ratio of the spatial resolution is equal to the ratio of the bandwidth. As a specific example, if the ratio between the spatial resolution of the target high-resolution full aperture image and the spatial resolution of the low-resolution full aperture image DIN3 is 1:4, it is desirable to generate the target low-resolution sub-aperture image corresponding to the target high-resolution full aperture image using a bandwidth of 1/4. However, the relationship between the bandwidth and the spatial resolution may differ depending on the satellite.

In addition, noise characteristics such as side lobes may differ between the low-resolution sub-aperture image DIN1 and the high-resolution full aperture image DIN2 corresponding to the low-resolution sub-aperture image DIN1. Therefore, each of the low-resolution sub-aperture image DIN1 and the high-resolution full aperture image DIN2 may be decomposed into images of finer size, and when a certain or greater difference occurs in the signal components between the decomposed images, the low-resolution sub-aperture image DIN1 and the high-resolution full aperture image DIN2 may not be used when generating a trained model, thereby improving the learning efficiency. Side lobes are noises specific to radar images. That is, in order to improve the learning efficiency, when the difference between the signal components in the target area of the target high-resolution full aperture image and the signal components in the target area of the target low-resolution sub-aperture image is equal to or greater than a threshold value, the target high-resolution full aperture image and the target low-resolution sub-aperture image are not used when learning the trained model. The target area is at least a part of each of the target high-resolution full aperture image and the target low-resolution sub-aperture image. Furthermore, the target area of the target high-resolution full aperture image and the target area of the target low-resolution sub-aperture image indicate the same area. In this case, as a specific example, the user sets a tolerance parameter indicating an acceptable difference in signal components while visually checking the images. Specifically, the user checks whether or not a side lobe is occurring by visually comparing the two images, and sets the tolerance parameter according to the difference in signal components when a side lobe is occurring.

***Description of Effect of First Embodiment***
As described above, in the high resolution system 1 according to the present embodiment, a trained model is generated using learning data based on the actual degradation process of SAR images, and super-resolution processing is performed using the trained model. Therefore, according to the present embodiment, in the method for training a high resolution model for SAR images, when an actual low resolution image is input to the trained model, super-resolution processing can be performed with relatively high accuracy.

***Other configurations***
<Modification 1>
FIG. 10 shows an example of the hardware configuration of a learning device 20 according to this modified example.
The learning device 20 includes a processing circuit 58 in place of the processor 51 , the processor 51 and a memory 52 , the processor 51 and an auxiliary storage device 53 , or the processor 51 , the memory 52 , and the auxiliary storage device 53 .
The processing circuitry 58 is hardware that realizes at least a portion of each unit of the learning device 20 .
The processing circuitry 58 may be dedicated hardware, or may be a processor that executes programs stored in the memory 52 .

When processing circuitry 58 is dedicated hardware, processing circuitry 58 may be, for example, a single circuit, a multiple circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof.
The learning device 20 may include a plurality of processing circuits that replace the processing circuit 58. The plurality of processing circuits share the role of the processing circuit 58.

In the learning device 20, some functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

Processing circuitry 58 is illustratively implemented in hardware, software, firmware, or a combination thereof.
The processor 51, the memory 52, the auxiliary storage device 53, and the processing circuit 58 are collectively referred to as the “processing circuitry.” In other words, the functions of the functional components of the learning device 20 are realized by the processing circuitry.
The hardware configuration of each device included in the high resolution system 1 may be similar to the hardware configuration of the learning device 20 according to this modified example.

***Other embodiments***
Although the first embodiment has been described, it is possible to combine a plurality of parts of the present embodiment. Alternatively, it is possible to partially implement the present embodiment. In addition, the present embodiment may be modified in various ways as necessary, and may be implemented in any combination, either as a whole or in part.
The above-described embodiment is essentially a preferred example, and is not intended to limit the scope of the present disclosure, its applications, and uses. The procedures described using the flowcharts and the like may be modified as appropriate.

1 High resolution system, 10 Input image generation device, 101 Image reproduction unit, 20 Learning device, 201 Data acquisition unit, 202 Model generation unit, 203 Learned model storage unit, 30 Inference device, 301 Data acquisition unit, 302 Inference unit, 51 Processor, 52 Memory, 53 Auxiliary storage device, 54 Input/output IF, 55 Communication device, 58 Processing circuit, 59 Signal line, DIN0 Complex radar image, DIN1 Low resolution sub-aperture image, DIN2 High resolution full aperture image, DIN3 Low resolution full aperture image, DOUT Estimated high resolution full aperture image, W1 First weight, W2 Second weight.

Claims (9)

A high-resolution system including a learning device having a model generation unit that uses learning data consisting of a plurality of high-resolution full aperture images generated by performing an image reproduction process using pulses indicated by each of a plurality of complex radar images indicating pulses reflected in each of a plurality of irradiation areas, and a plurality of low-resolution sub-aperture images generated by performing an image reproduction process using some of the pulses used in generating each of the plurality of high-resolution full aperture images, to learn a learned model that infers an estimated high-resolution full aperture image, which is an image showing a target irradiation area and has a resolution higher than the resolution of the low-resolution full aperture image, from a low-resolution full aperture image, which is an image corresponding to pulses reflected in a target irradiation area and has a resolution equivalent to the resolution corresponding to the plurality of low-resolution sub-aperture images. an inference device including: a trained model trained using training data consisting of a plurality of high-resolution full aperture images generated by performing an image reproduction process using pulses indicated by each of a plurality of complex radar images indicating pulses reflected in each of a plurality of irradiation areas; and a plurality of low-resolution sub-aperture images generated by performing an image reproduction process using some of the pulses used in generating each of the plurality of high-resolution full aperture images; and an inference unit configured to infer an estimated high-resolution full aperture image, which is an image showing the target irradiation area and has a resolution higher than that of the low-resolution full aperture image, using a low-resolution full aperture image, which is an image having a resolution equivalent to the resolution of the plurality of low-resolution sub-aperture images and corresponds to pulses reflected in the target irradiation area;
A resolution improvement system, wherein the learned model is a model that infers the estimated high-resolution full-aperture image from the low-resolution full-aperture image. The high resolution system according to claim 1 or 2, wherein the complex radar image is an image captured using either a synthetic aperture radar satellite or an aircraft. When each of the plurality of high-resolution full aperture images is a target high-resolution full aperture image, and a low-resolution sub-aperture image corresponding to the target high-resolution full aperture image is a target low-resolution sub-aperture image,
A high resolution system described in any one of claims 1 to 3, wherein the ratio between the bandwidth in the image reproduction process corresponding to the target high resolution full aperture image and the bandwidth in the image reproduction process corresponding to the target low resolution sub-aperture image is determined based on the ratio between the spatial resolution of the target high resolution full aperture image and the spatial resolution of the low resolution full aperture image. When each of the plurality of high-resolution full aperture images is a target high-resolution full aperture image, and a low-resolution sub-aperture image corresponding to the target high-resolution full aperture image is a target low-resolution sub-aperture image,
A high-resolution system as described in any one of claims 1 to 4, wherein when a difference between a signal component in a target region of the target high-resolution full aperture image and a signal component in the target region of the target low-resolution sub-aperture image is equal to or greater than a threshold, the target high-resolution full aperture image and the target low-resolution sub-aperture image are not used when training the trained model. A method for increasing resolution in which a computer uses learning data consisting of multiple high-resolution full aperture images generated by performing an image reproduction process using pulses indicated by each of multiple complex radar images indicating pulses reflected in each of multiple illumination regions, and multiple low-resolution sub-aperture images generated by performing an image reproduction process using some of the pulses used in generating each of the multiple high-resolution full aperture images, to learn a learned model that infers an estimated high-resolution full aperture image, which is an image showing the target illumination region and has a higher resolution than the low-resolution full aperture image, from a low-resolution full aperture image, which is an image corresponding to the pulses reflected in the target illumination region and has a resolution equivalent to the resolution corresponding to the multiple low-resolution sub-aperture images. A high resolution method for inferring an estimated high resolution full aperture image, which is an image showing a target illumination area and has a higher resolution than the low resolution full aperture image, using a trained model trained using training data consisting of a plurality of high resolution full aperture images generated by performing an image reproduction process using pulses indicated by each of a plurality of complex radar images indicating pulses reflected in each of a plurality of illumination areas, and a plurality of low resolution sub aperture images generated by performing an image reproduction process using some of the pulses used in generating each of the plurality of high resolution full aperture images, and a low resolution full aperture image, which is an image having a resolution equivalent to a resolution corresponding to the plurality of low resolution sub aperture images and corresponds to pulses reflected in a target illumination area,
A high resolution method, in which the learned model is a model that infers the estimated high resolution full aperture image from the low resolution full aperture image. A high-resolution program that causes a learning device, which is a computer, to execute a model generation process to learn a learned model that infers an estimated high-resolution full aperture image, which is an image showing a target illumination area and has a higher resolution than the low-resolution full aperture image, from a low-resolution full aperture image, which is an image corresponding to pulses reflected in a target illumination area and has a resolution equivalent to the resolution corresponding to the low-resolution sub-aperture images, using learning data consisting of a plurality of high-resolution full aperture images generated by executing an image reproduction process using pulses indicated by each of a plurality of complex radar images indicating pulses reflected in each of a plurality of illumination areas, and a plurality of low-resolution sub-aperture images generated by executing an image reproduction process using some of the pulses used when generating each of the plurality of high-resolution full aperture images. A high-resolution program for causing an inference device, which is a computer, to execute an inference process for inferring an estimated high-resolution full aperture image, which is an image showing a target illumination area and has a higher resolution than the low-resolution full aperture image, using a trained model trained using learning data consisting of a plurality of high-resolution full aperture images generated by executing an image reproduction process using pulses indicated by each of a plurality of complex radar images indicating pulses reflected in each of a plurality of illumination areas, and a plurality of low-resolution sub-aperture images generated by executing an image reproduction process using some of the pulses used in generating each of the plurality of high-resolution full aperture images, and a low-resolution full aperture image, which is an image having a resolution equivalent to a resolution corresponding to the plurality of low-resolution sub-aperture images and corresponds to pulses reflected in a target illumination area,
The learned model is a resolution improvement program that is a model that infers the estimated high-resolution full-aperture image from the low-resolution full-aperture image.

Images