JP7699489B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

In recent years, machine learning technology has been used in a variety of fields, and it is expected to be used in the agricultural field as well. When farmland is paddy field, in order to create a production plan, it is necessary to identify the flooded areas and ridge areas and estimate the costs according to the area, etc. However, surveying to identify the flooded areas and ridge areas takes a lot of time and effort, and it is not realistic to conduct manual surveys over a vast area.

On the other hand, to utilize machine learning technology, it is essential to prepare training data for constructing a learning model. A large amount of data is generally required to construct a learning model, and the task of labeling this data with correct answers takes a lot of time and effort. Patent Document 1 discloses a technology for generating large amounts of training data at low cost to be used in training a device that identifies or detects target objects from images.

JP 2014-178957 A

We will consider using publicly available data to generate learning data for building a learning model to identify flooded and ridge areas from images. The Ministry of Agriculture, Forestry and Fisheries publishes brush polygons, which are plot information for farmland created based on satellite images, etc. The Geospatial Information Authority of Japan also publishes digital elevation models (DEMs), which are a collection of elevation data in which the earth's surface is divided into equally spaced squares, with each square assigned an elevation value for its center point.

Brush polygons are extremely useful as basic information on agricultural land. However, because brush polygons determine whether agricultural land exists or not from satellite images, etc., they do not always match the actual information depending on the time the image was taken. In addition, the satellite images used to create brush polygons are produced by photographing and processing visible light reflected from the earth's surface by sunlight from outer space. Although the effects of terrain undulations and altitude are corrected, it is not possible to completely correct deviations and distortions caused by errors in the satellite's position and attitude at the time of shooting, or errors in the terrain data used for correction, so there may be a misalignment between the corresponding background image and the brush polygon. Furthermore, while many polygons include water-filled areas and ridge areas, there are also some polygons that do not include ridge areas.

The present invention aims to provide an information processing device, information processing method, and program that can easily and accurately extract water-filled areas in a visible image by using partition information that does not necessarily match the actual information.

An information processing device according to one aspect of the present invention includes a data acquisition unit that acquires a visible image, partition information corresponding to the visible image, and a digital elevation model, an area extraction unit that extracts corresponding areas in the visible image, partition information, and digital elevation model, and a water-filled area extraction unit that extracts water-filled areas from the area of the partition information and the area of the digital elevation model, where the water-filled area extraction unit plots a first point of the area of the partition information on the area of the digital elevation model, and extracts a set of pixels whose elevation difference from the elevation of the plotted point is within a predetermined range as the water-filled area.

According to this aspect, water-covered areas in a visible image can be easily extracted by using the area information corresponding to the visible image and the digital elevation model information.

The information processing device may further include a ridge area extraction unit that classifies pixels belonging to the extracted area using a classification method, the pixels having color information obtained from the visible image and elevation information obtained from the digital elevation model, and extracts ridge areas from the classified pixel group. According to this aspect, it is possible to easily extract not only water-filled areas in the visible image, but also ridge areas in the visible image.

In the above information processing device, the ridge area extraction unit may expand the area of the partition information extracted by the area extraction unit according to a predetermined condition, and classify pixels belonging to the expanded area of the partition information using a classification method. According to this aspect, by using the pixels belonging to the expanded area, it is possible to effectively extract the ridge area even from some areas in the partition information that do not include the ridge area.

The information processing device may further include a non-paddy field removal unit that performs histogram analysis of the visible image corresponding to the water-filled area extracted by the water-filled area extraction unit, and removes areas determined to be non-paddy fields based on the characteristics of the analysis results from the water-filled area. According to this aspect, even when using plot information that does not necessarily match the actual information, it is possible to reliably remove non-paddy fields and match the actual information.

In the above information processing device, the features may include the kurtosis and smoothness of the curve. According to this aspect, the kurtosis and smoothness of the curve of the histogram analysis can be used to easily determine non-paddy field areas.

The information processing device may further include a learning data generating unit that generates learning data labeled with the water-filled areas and ridge areas in the visible image. According to this aspect, by using the visible image, the plot information, and the digital elevation model, learning data labeled with the water-filled areas and ridge areas in the visible image can be easily generated.

In the above information processing device, the first point may be a center of gravity. According to this aspect, the water-filled region can be extracted using a point that can be reliably distinguished from ridge regions and the like that may exist around the water-filled region.

A method according to another aspect of the present invention includes the steps of acquiring a visible image, parcel information corresponding to the visible image, and a digital elevation model, extracting corresponding regions in the visible image, parcel information, and digital elevation model, and extracting a water-covered region from the parcel information region and the digital elevation model region, by plotting a first point of the parcel information region on the digital elevation model region, and extracting, as the water-covered region, a set of pixels whose elevation difference from the elevation of the plotted point is within a predetermined range.

A program according to another aspect of the present invention causes one or more computers to execute the following processes: acquiring a visible image, partition information and a digital elevation model corresponding to the visible image; extracting corresponding areas in the visible image, partition information and digital elevation model; and extracting a water-covered area from the area of the partition information and the area of the digital elevation model, in which a first point of the area of the partition information is plotted on the area of the digital elevation model, and a set of pixels whose elevation difference from the elevation of the plotted point is within a predetermined range is extracted as the water-covered area.

The present invention provides an information processing device, information processing method, and program that can easily and accurately extract water-filled areas in a visible image by using partition information that does not necessarily match the actual information.

FIG. 2 is a schematic diagram illustrating the processing of an information processing device according to an embodiment of the present invention. 1 is a block diagram of an information processing apparatus according to an embodiment of the present invention; FIG. 13 is a diagram illustrating an example of an extracted visible image, a brush polygon, and a corresponding area in a DEM. FIG. 13 is a diagram illustrating histograms of paddy fields and non-paddy fields. FIG. 1 illustrates a scatter plot of pixels of a region with color information obtained from a visible image and elevation information obtained from a DEM. 1 is a flowchart showing a learning data generation process according to an embodiment of the present invention. 10 is a flowchart showing a water-filled region extraction process according to an embodiment of the present invention.

The following embodiments of the present invention will be described with reference to the attached drawings. Note that the following embodiments are provided to facilitate understanding of the present invention and are not intended to limit the present invention. Furthermore, the present invention can be modified in various ways without departing from the gist of the invention. Furthermore, those skilled in the art will be able to adopt embodiments in which the elements described below are replaced with equivalent elements, and such embodiments are also within the scope of the present invention.

(System configuration)

An overview of the present invention will be described using FIG. 1. FIG. 1 is a schematic diagram illustrating the processing of an information processing device according to one embodiment of the present invention.

The information processing device 100 is a device that generates learning data for constructing a learning model for identifying water-filled areas and ridge areas from an image. As shown in FIG. 1, the information processing device 100 generates desired learning data using a visible image I, a brush polygon P, and a digital elevation model M. In one embodiment, the information processing device 100 generates learning data that includes a visible image and a mask image that is composed of water-filled areas and ridge areas in the visible image.

The brush polygon P is agricultural land division information created based on satellite images, etc. The digital elevation model E is a collection of elevation data in which the earth's surface is divided into equally spaced squares, with each square assigned an elevation value at its center.

The information processing device 100 acquires a visible image I of the area for which learning data is to be generated, a brush polygon P corresponding to this visible image, and a digital elevation model M, and by using the color information contained in the visible image I, the farmland division information contained in the brush polygon P, and the elevation information contained in the digital elevation model M, it is possible to generate highly accurate learning data for constructing a learning model for identifying water-filled areas and ridge areas from an image.

(Functional configuration)

Figure 2 is a block diagram of an information processing device according to one embodiment of the present invention. Note that while Figure 2 assumes a single information processing device 100 and shows only the necessary functional configuration, the information processing device 100 can also be configured as part of a multi-functional distributed system consisting of multiple computer systems.

The information processing device 100 includes an input unit 110, a control unit 120, a memory unit 130, and a communication unit 140.

The input unit 110 is configured to accept operations from the administrator of the information processing device 100, and can be realized by a keyboard, mouse, touch panel, etc.

The control unit 120 includes an arithmetic processing unit 121 such as a CPU or MPU, and a memory 122 such as a RAM. The arithmetic processing unit 121 operates various functional units by executing a program recorded in the storage unit 130 based on various inputs. This program may be stored on a recording medium such as a CD-ROM, or distributed via a network and installed on the computer. The memory 122 is used to temporarily store various data required for calculations, etc. while the various programs are executing processes.

The memory unit 130 is configured with a storage device such as a hard disk, and records various programs necessary for executing the processes in the control unit 120, data necessary for executing the various programs, etc. In this embodiment, it is preferable that the memory unit 130 has an image memory unit 131, a brush polygon memory unit 132, a digital elevation model (DEM) memory unit 133, and a learning data memory unit 134.

The image storage unit 131 stores visible images including rice paddies.

The brush polygon memory unit 132 stores farmland division information created based on satellite images, etc. In one embodiment, brush polygons published by the Ministry of Agriculture, Forestry and Fisheries are used as farmland division information.

The DEM storage unit 133 stores a collection of elevation data in which the earth's surface is divided into equally spaced squares, and each square is assigned an elevation value at its center. In one embodiment, the digital elevation model in the basic map information published by the Geospatial Information Authority of Japan is used as the collection of elevation data.

The learning data storage unit 134 stores learning data that is labeled to indicate the water-filled regions and ridge regions in the visible image. In one embodiment, it is desirable for the learning data storage unit 134 to store learning data that includes the visible image and a mask image that is configured of the water-filled regions and ridge regions in the visible image.

The communication unit 140 is configured to connect the information processing device 100 to a network. For example, the communication unit 140 can be realized by a LAN card, an analog modem, an ISDN modem, etc., and an interface for connecting these to the processing unit via a transmission path such as a system bus.

Furthermore, as shown in FIG. 2, the calculation processing unit 121 has, as functional units, a data acquisition unit 123, a region extraction unit 124, a water-filled region extraction unit 125, a non-paddy field removal unit 126, a ridge region extraction unit 127, a learning data generation unit 128, and an output unit 129.

The data acquisition unit 123 acquires visible images of the area for which learning data is to be generated, and the brush polygons and DEMs corresponding to these visible images. In this embodiment, the data acquisition unit 123 acquires data from the image storage unit 131, the brush polygon storage unit 132, and the DEM storage unit 133.

The area extraction unit 124 extracts corresponding areas of a specified shape from the acquired visible image, brush polygon, and DEM. In this embodiment, the area extraction unit 124 overlays the visible image, brush polygon, and DEM to select one area in the brush polygon, and extracts an area from the visible image and DEM that sufficiently contains the selected area. Figure 3 shows corresponding areas in the visible image, brush polygon, and DEM extracted by the area extraction unit 124. Figure 3(a) shows an example of a brush polygon, Figure 3(b) shows a visible image, and Figure 3(c) shows a DEM.

The water-filled area extraction unit 125 extracts a water-filled area from the extracted brush polygon area and DEM area based on the elevation information. In this embodiment, the water-filled area extraction unit 125 plots the center of gravity of the brush polygon area on the DEM area, and extracts a set of pixels whose elevation difference from the elevation of the plotted point is within a predetermined range as the water-filled area. Specifically, the water-filled region extraction unit 125 first extracts, as the water-filled region, from among the 16 pixels surrounding the center-of-gravity pixel x _ij , namely x _{i +1j-1} , x _i+1j , x _i+ 1j+ _{1, x ij -1, x ij+1} , x _{i- 1j-1, x i-1j} , x _i-1j+1 , x _i+4j -4, x _i+4j , x _i+4j +4, x _ij-4 , x _ij+4 , x _i-4j-4 , x i-4j, x _i-4j+4 , those whose elevation difference from the center-of-gravity pixel x _ij is within ±0.1998 m. The water-filled region extraction unit 125 further extracts, as water-filled regions, adjacent pixels whose elevation difference from the pixels already extracted as water-filled regions is within ±0.1998 m, and repeats this process to extract water-filled regions.

Note that the range of elevation difference used to extract the water-filled region is merely an example, and other appropriate elevation difference ranges can be used to extract the water-filled region. Furthermore, the point (first point) used to extract the water-filled region is not limited to the center of gravity, but can be a center point or any point inside that can be distinguished from ridge regions that may exist around the water-filled region. The water-filled region extraction unit 125 may perform, for example, a morphological transformation to remove noise from waterways and the like. Note that the method for removing noise is not limited to morphological transformation, and any other method can be used.

The non-paddy field removal unit 126 removes non-paddy field areas based on the color information of the visible image. In this embodiment, the non-paddy field removal unit 126 performs histogram analysis on the visible image corresponding to the water-filled area extracted by the water-filled area extraction unit 125, and removes areas determined to be non-paddy fields based on the characteristics of the analysis results from the water-filled area.

As shown in the upper part of Figure 4, the histogram of paddy fields has a larger kurtosis and shows a smoother curve than the histogram of non-paddy fields in the lower part of the figure. The non-paddy field removal unit 126 determines the non-paddy field area based on the kurtosis and smoothness of the curve, that is, the number of times the curve passes through 0 on the Y axis when differentiated.

The ridge area extraction unit 127 classifies areas including water-filled areas and ridge areas based on color information of the visible image and altitude information of the DEM, and extracts the ridge area. In this embodiment, the ridge area extraction unit 127 expands the area of the brush polygon extracted by the area extraction unit 124 according to predetermined conditions, and classifies pixels belonging to the area of the expanded brush polygon having color information obtained from the visible image and altitude information obtained from the DEM using K-means. For example, the ridge area extraction unit 127 may expand the area of the brush polygon by performing a morphological transformation. In this embodiment, the morphological transformation is performed using a kernel of size 30 x 30, but any other kernel size can be used.

Next, the ridge region extraction unit 127 estimates whether the classified pixel group belongs to a water-filled region, a ridge region, or other region, and extracts the ridge region. For example, the ridge region extraction unit 127 can estimate whether the classified pixel group belongs to a water-filled region, a ridge region, or other region by using the area and color distribution of the classified pixel group.

Each point in the scatter plot shown in Figure 5 represents a pixel, the color of the point represents the altitude, and the position of the point represents the color information.

In this way, by expanding the area of the brush polygon, it is possible to effectively extract ridge areas even for some polygons that do not contain ridge areas in the brush polygon. Note that the method used for classification is not limited to K-means, and other classification methods can be used. Also, in this embodiment, color information in the Lab color space is used, but the color information used for classification is not limited to the Lab color space, and other color spaces can be used.

The learning data generating unit 128 generates learning data that is labeled to indicate the water-filled areas and ridge areas in the visible image. In this embodiment, the learning data generating unit 128 generates a mask image consisting of the water-filled areas extracted by the water-filled area extracting unit 125 and the ridge areas extracted by the ridge area extracting unit 127, and stores the generated mask image and the corresponding visible image in the learning data storage unit 134.

The output unit 129 outputs the learning data stored in the learning data storage unit 134.

(Learning data generation process)

The learning data generation process according to an embodiment of the present invention will be described in detail with reference to FIG. 6. In this embodiment, before the learning data generation process described in FIG. 6 is performed, each data is stored in the image storage unit 131, the brush polygon storage unit 132, and the DEM storage unit 133 under the management of the administrator of the information processing device 100.

In step S601, the data acquisition unit 123 of the information processing device 100 acquires a visible image of the area for which learning data is to be generated, and a brush polygon and DEM corresponding to this visible image. In this embodiment, the data acquisition unit 123 acquires data from the image storage unit 131, the brush polygon storage unit 132, and the DEM storage unit 133.

In step S602, the area extraction unit 124 of the information processing device 100 extracts corresponding areas of a specified shape from the acquired visible image, brush polygon, and DEM. In this embodiment, the area extraction unit 124 overlays the visible image, brush polygon, and DEM to select one area in the brush polygon, and extracts an area from the visible image and DEM that sufficiently contains the selected area. Figure 3 shows corresponding areas in the visible image, brush polygon, and DEM extracted by the area extraction unit 124. Figure 3(a) is an example of a brush polygon, Figure 3(b) is an example of a visible image, and Figure 3(c) is an example of a DEM.

In step S603, the water-filled area extraction unit 125 of the information processing device 100 extracts a water-filled area from the extracted brush polygon area and the DEM area based on the elevation information. Details will be described later with reference to FIG. 7.

In step S604, the non-paddy field removal unit 126 of the information processing device 100 removes non-paddy field areas based on the color information of the visible image. In this embodiment, the non-paddy field removal unit 126 performs histogram analysis on the visible image corresponding to the water-filled area extracted by the water-filled area extraction unit 125, and removes areas determined to be non-paddy fields based on the characteristics of the analysis results from the water-filled area.

As shown in the upper part of Figure 4, the histogram of paddy fields has a larger kurtosis and shows a smoother curve than the histogram of non-paddy fields in the lower part of the figure. The non-paddy field removal unit 126 determines the non-paddy field area based on the kurtosis and smoothness of the curve, that is, the number of times the curve passes through 0 on the Y axis when differentiated.

In step S605, the ridge area extraction unit 127 of the information processing device 100 classifies the area including the water-filled area and the ridge area based on the color information of the visible image and the elevation information of the DEM, and extracts the ridge area. In this embodiment, the ridge area extraction unit 127 performs a morphological transformation to expand the area of the brush polygon extracted by the area extraction unit 124, and classifies pixels belonging to the area of the expanded brush polygon having color information obtained from the visible image and elevation information obtained from the DEM using K-means. In this embodiment, the morphological transformation is performed using a kernel of size 30 x 30, but any other kernel size can be used.

Each point in the scatter plot shown in Figure 5 represents a pixel, the color of the point represents the altitude, and the position of the point represents color information. If the expanded area includes only water-filled areas and ridge areas, the ridge area extraction unit 127 classifies the pixels into two classes. On the other hand, if the expanded area includes areas other than water-filled areas and ridge areas, such as vinyl greenhouses and roads, the ridge area extraction unit 127 classifies the pixels into three classes.

Next, the ridge region extraction unit 127 estimates whether the classified pixel group belongs to a water-filled region, a ridge region, or other region, and extracts the ridge region. For example, the ridge region extraction unit 127 can estimate whether the classified pixel group belongs to a water-filled region, a ridge region, or other region by using the area and color distribution of the classified pixel group.

In this way, by expanding the area of the brush polygon, it is possible to effectively extract ridge areas even for some polygons that do not contain ridge areas in the brush polygon. Note that the method used for classification is not limited to K-means, and other classification methods can be used. Also, in this embodiment, color information in the Lab color space is used, but the color information used for classification is not limited to the Lab color space, and other color spaces can be used.

In step S606, the learning data generating unit 128 of the information processing device 100 generates learning data labeled with the water-filled areas and ridge areas in the visible image. In this embodiment, the learning data generating unit 128 generates a mask image consisting of the water-filled areas extracted by the water-filled area extracting unit 125 and the ridge areas extracted by the ridge area extracting unit 127, and stores the generated mask image and the corresponding visible image in the learning data storage unit 134.

As described above, according to this embodiment, the information processing device 100 can use brush polygons to generate learning data for constructing a learning model for identifying flooded areas and ridge areas from an image. By utilizing existing brush polygons, it is possible to reduce the time and effort required for preparing learning data, from data collection to labeling. This is particularly suitable when the area in which learning data is generated is flat and has rows of rectangular rice paddies.

(Water-filled area extraction process)

The water-filled region extraction process of step S603 will be described in detail with reference to FIG. 7. FIG. 7 is a flowchart showing the water-filled region extraction process according to one embodiment of the present invention.

In step S701, the water-filled area extraction unit 125 finds the center of gravity of the brush polygon area. Next, in step S702, the water-filled area extraction unit 125 plots the center of gravity of the brush polygon area on the DEM area, and in step S703, fills in pixels whose elevation difference from the elevation of the plotted point is within a predetermined range.

Specifically, the water-filled region extraction unit 125 first extracts, as the water-filled region, from among the 16 pixels surrounding the center-of-gravity pixel x _ij , namely x _{i +1j-1} , x _i+1j , x _i+ 1j+ _{1, x ij -1, x ij+1} , x _{i- 1j-1, x i-1j} , x _i-1j+1 , x _i+4j -4, x _i+4j , x _i+4j +4, x _ij-4 , x _ij+4 , x _i-4j-4 , x i-4j, x _i-4j+4 , those whose elevation difference from the center-of-gravity pixel x _ij is within ±0.1998 m. The water-filled region extraction unit 125 further extracts, as water-filled regions, adjacent pixels whose elevation difference from the pixels already extracted as water-filled regions is within ±0.1998 m, and repeats this process to extract water-filled regions.

The range of elevation difference used to extract the water-filled area is merely an example, and other appropriate elevation difference ranges can be used to extract the water-filled area. Furthermore, the point used to extract the water-filled area is not limited to the center of gravity, but can be the center point or any point inside that can be distinguished from ridge areas that may exist around the water-filled area.

In step S704, the water-filled region extraction unit 125 performs morphological transformation to remove noise such as water channels and extract the water-filled region. Note that the method for removing noise is not limited to morphological transformation, and any other method can be used.

100...information processing device, 110...input unit, 120...control unit, 121...arithmetic processing unit, 122...memory, 123...data acquisition unit, 124...area extraction unit, 125...water-filled area extraction unit, 126...non-paddy field removal unit, 127...ridge area extraction unit, 128...learning data generation unit, 129...output unit, 130...storage unit, 131...image storage unit, 132...brush polygon storage unit, 133...DEM storage unit, 134...learning data storage unit, 140...communication unit, I...visible image, P...brush polygon, M...elevation model

Claims (7)

a data acquisition unit for acquiring a visible image, section information corresponding to the visible image, and a digital elevation model;
an area extraction unit that extracts corresponding areas in the visible image, the block information, and the digital elevation model;
a water-filled region extraction unit that extracts a water-filled region from the region of the partition information and the region of the digital elevation model, the water-filled region extraction unit plotting a first point of the region of the partition information on the region of the digital elevation model, and extracting, as the water-filled region, a set of pixels whose elevation difference from the elevation of the plotted point is within a predetermined range ;
a non-paddy field removal unit that performs a histogram analysis of a visible image corresponding to the water-filled region extracted by the water-filled region extraction unit, and removes a region determined to be a non-paddy field based on a characteristic of the analysis result from the water-filled region;
A learning data generating unit that generates learning data labeled with water-filled regions and ridge regions in a visible image;
An information processing device comprising: The information processing device according to claim 1 further comprises a ridge area extraction unit that classifies pixels belonging to the extracted area using a classification method, the pixels having color information obtained from the visible image and elevation information obtained from the digital elevation model, and extracts a ridge area from the classified pixel group. The information processing device according to claim 2, wherein the ridge area extraction unit expands the area of the partition information extracted by the area extraction unit according to a predetermined condition, and classifies pixels belonging to the expanded area of the partition information using a classification method. The information processing device according to claim 1 , wherein the features include kurtosis and smoothness of the curve. The information processing device according to claim 1 , wherein the first point is a center of gravity. obtaining a visible image, parcel information corresponding to said visible image, and a digital elevation model;
extracting the visible imagery, the parcel information and corresponding areas in the digital elevation model;
a step of extracting a water-covered region from the region of the partition information and the region of the digital elevation model, in which a first point of the region of the partition information is plotted on the region of the digital elevation model, and a set of pixels whose elevation difference from the elevation of the plotted point is within a predetermined range is extracted as the water-covered region ;
a step of performing a histogram analysis on the visible image corresponding to the extracted water-filled region, and removing a region determined to be a non-paddy field based on characteristics of the analysis result from the water-filled region;
generating learning data labeled with labels indicating water-filled regions and ridge regions in a visible image;
The method includes: On one or more computers,
obtaining a visible image, parcel information corresponding to the visible image, and a digital elevation model;
extracting corresponding areas in the visible image, the parcel information, and the digital elevation model;
a process of extracting a water-covered region from the region of the partition information and the region of the digital elevation model, the process including plotting a first point of the region of the partition information on the region of the digital elevation model, and extracting a set of pixels whose elevation difference from the elevation of the plotted point is within a predetermined range as the water-covered region ;
a process of performing a histogram analysis on the visible image corresponding to the extracted water-filled region, and removing a region determined to be a non-paddy field based on characteristics of the analysis result from the water-filled region;
A process for generating learning data labeled with water-filled areas and ridge areas in visible images;
A program that executes the following.

Images