JP7445969B2 - Estimation device, estimation method, and program - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act (1) Publication date June 10, 2019 Publication JSTnews June 2019 issue (June 2019) P5-7 (Web version URL: https://www.jst. go.jp/pr/jst-news/backnumber/2019/201906/index.html https://www.jst.go.jp/pr/jst-news/backnumber/2019/201906/pdf/ 2019_06.pdf ) National Published by Japan Science and Technology Agency <Reference> JSTnews June 2019 issue Excerpt (2) Website publication date July 9, 2019 Website address https://www. jstage. jst. go. jp/browse/jstnews/-char/ja https://www. jstage. jst. go. jp/article/jstnews/2019/6/2019_5/_article/-char/ja/ <Data> Public website (J-Stage) printout (3) Publication date August 6, 2019 Publication National research Japan Science and Technology Agency Strategic Creative Research Promotion Project (PRESTO) “Creation of technological base to realize innovative crop cultivation methods through collaboration with information science” Research Area Area Activities/Evaluation Report - FY2018 Final Research Project - Published by the Japan Science and Technology Agency (Web version URL) https://www. jst. go. jp/kisoken/presto/evaluation/posteriori/h30/JST_1112075_2018_PEE. pdf <Materials> [Research report]

The present invention relates to an estimation device, an estimation method, and a program for estimating the yield of crops.

If the yield of crops in the field can be estimated before harvesting, this information will be useful for scheduling harvesting operations and subsequent processing steps. A method of estimating crop yield by combining crop images and crop growth models has been reported (for example, Non-Patent Documents 1-2).

Non-Patent Document 1-2 discloses a technique for estimating the yield of rice by integrating satellite images or aerial images taken by a drone with an existing crop growth model. The crop growth model used in Non-Patent Document 1-2 uses Leaf Area Index (LAI) as an observed variable. In Non-Patent Document 1-2, internal parameters of a crop growth model are determined using LAI estimated from images, and a growth simulation is performed using the model with determined internal parameters. The final yield is then calculated based on the results of the growth simulation.

High-accuracy prediction of rice growth and yield using remote sensing and crop models, [online], Grants-in-Aid for Scientific Research database, [searched on April 2, 2020], Internet <https://kaken.nii .ac.jp/ja/grant/KAKENHI-PROJECT-23780263/＞ Hama et al., 2018. Estimation of plant height and yield of paddy rice using UAV remote sensing and solar radiation, Journal of the Japanese Society of Hydrology and Water Resources 31(2), 68-82.

However, LAI cannot be directly measured from images. For this reason, it is necessary to determine a method for estimating LAI from images for each crop, and there is a problem in that the accuracy of estimating LAI varies depending on the crop. If LAI cannot be accurately estimated from images, the internal parameters of the model cannot be appropriately determined, making it difficult to accurately estimate the yield. Furthermore, LAI is a physical quantity that targets a crop community with a certain specific area. Therefore, LAI cannot be treated as information about individual crops. Although the methods described in Non-Patent Documents 1-2 can estimate the yield of crops on a field or community basis, there is a problem in that it is difficult to estimate the yield of each individual crop.

The present invention has been made in view of these circumstances, and provides an estimation device, an estimation method, and a program that can estimate the yield of each individual crop.

According to one aspect of the present invention, the estimation device includes an above-ground part volume calculating section that calculates an above-ground part volume of each crop individual at the target location based on three-dimensional information at the target location; a parameter determination unit that determines internal parameters of a crop growth model for each crop individual based on the above-ground volume of each crop individual at the target location calculated by; and a crop whose internal parameters are determined by the parameter determination unit. A yield estimating unit that estimates the yield of each individual crop at the target location using a growth model.

The three-dimensional information may be information restored based on a target image in which the target location is captured, or information measured by a distance measuring device.

The parameter determination unit adjusts the crop so that the above-ground volume of each crop individual obtained from the simulation result using the crop growth model approaches the above-ground volume of each crop individual in the field calculated by the above-ground volume calculation unit. The internal parameters of the growth model may also be determined.

The parameter determination unit is configured to calculate a time-series change in the above-ground volume of each crop individual obtained from a simulation result using a crop growth model as a time-series change in the above-ground volume of each crop individual in the field calculated by the above-ground volume calculation unit. The internal parameters of the crop growth model may be determined in accordance with series changes.

The parameter determining unit determines internal parameters of the crop growth model using the time-series changes in above-ground volume from germination or budding to harvest time for each crop individual in the field calculated by the above-ground volume calculation unit. You may also do so.

The yield estimating unit estimates the amount of dry matter of the individual crop at the harvest time obtained from the simulation result using the crop growth model whose internal parameters are determined by the parameter determining unit as the yield of the individual crop. It's okay.

The above-ground area volume calculation unit extracts a vegetation area where crops are vegetated based on the aerial photographed image, estimates the number of individual crop plants included in the extracted vegetation area, and uses the estimated number of individuals. Then, the above-ground volume of each individual crop in the field may be calculated.

According to one aspect of the present invention, in the estimation method, the above-ground volume calculation unit calculates the above-ground volume of each crop individual at the target location based on three-dimensional information at the target location, and the parameter determination unit: The internal parameters of the crop growth model are determined for each crop individual based on the above-ground volume of each crop individual at the target location calculated by the above-ground part volume calculation unit, and the yield estimation unit Using the crop growth model whose internal parameters have been determined, the yield of each individual crop at the target location is estimated.

According to one aspect of the present invention, the program causes a computer to calculate the above-ground volume of each crop individual at the target location based on three-dimensional information at the target location, and determining the internal parameters of a crop growth model for each crop individual based on the above-ground volume of each crop, and estimating the yield of each crop individual at the target location using the crop growth model with the determined internal parameters. This is a program for

According to the present invention, it is possible to estimate the yield for each individual crop.

FIG. 1 is a block diagram showing an example of the configuration of an estimation device 1 according to a first embodiment. FIG. 3 is a diagram illustrating processing performed by the vegetation portion extraction unit 12 according to the first embodiment. FIG. 2 is a diagram illustrating processing performed by the number of individuals estimating unit 13 according to the first embodiment. FIG. 3 is a diagram illustrating processing performed by the population estimation unit 13 according to the first embodiment. FIG. 2 is a diagram illustrating processing performed by the number of individuals estimating unit 13 according to the first embodiment. FIG. 3 is a diagram illustrating processing performed by the individual position determination unit 14 according to the first embodiment. FIG. 3 is a diagram illustrating processing performed by the individual position determination unit 14 according to the first embodiment. FIG. 3 is a diagram illustrating processing performed by the individual position determination unit 14 according to the first embodiment. FIG. 3 is a diagram illustrating processing performed by the inter-individual boundary determination unit 15 according to the first embodiment. It is a figure explaining the process performed by the individual growth area calculation part 16 based on 1st Embodiment. It is a figure explaining the process performed by the individual growth area calculation part 16 based on 1st Embodiment. It is a figure explaining the process performed by the individual growth area calculation part 16 based on 1st Embodiment. It is a figure explaining the process performed by the individual growth area calculation part 16 based on 1st Embodiment. It is a figure explaining the process performed by the individual growth area calculation part 16 based on 1st Embodiment. It is a figure explaining the process performed by the individual growth area calculation part 16 based on 1st Embodiment. It is a figure explaining the process performed by the individual growth area calculation part 16 based on 1st Embodiment. It is a flow diagram showing the flow of processing performed by the estimation device 1 according to the first embodiment. It is a block diagram showing an example of the composition of estimation device 100 concerning a 2nd embodiment. FIG. 7 is a diagram illustrating an example of an image acquired by the image acquisition unit 110 according to the second embodiment. FIG. 7 is a diagram illustrating an example of an image acquired by the image acquisition unit 110 according to the second embodiment. It is a figure which shows the relationship between the above-ground part volume and dry matter amount calculated by the above-ground part volume calculation part 112 based on 2nd Embodiment. FIG. 7 is a diagram illustrating a model in which internal parameters are determined by a parameter determination unit 113 according to the second embodiment. FIG. 7 is a diagram illustrating processing performed by a parameter determination unit 113 according to the second embodiment. FIG. 7 is a diagram illustrating an example in which the above-ground volume calculated by the above-ground volume calculation unit 112 according to the second embodiment is compared with the above-ground volume calculated from the simulation results using the model. FIG. 7 is a diagram illustrating an example of a simulation result using a model according to a second embodiment. FIG. 7 is a diagram illustrating an example of a simulation result using a model according to a second embodiment. FIG. 7 is a diagram illustrating an example of a simulation result using a model according to a second embodiment. FIG. 7 is a diagram illustrating an example of a simulation result using a model according to a second embodiment. It is a figure which shows the example of the result of comparing the harvest amount estimated using the 2nd model, and an actual harvest amount. It is a flow diagram showing the flow of processing performed by the estimation device 100 according to the second embodiment.

Embodiments of the present invention will be described below with reference to the drawings. The first embodiment is a preliminary embodiment for explaining the second embodiment. The second embodiment is an embodiment in which the yield of crops is estimated using the first embodiment.

(First embodiment)
The first embodiment will be described below. FIG. 1 is a block diagram showing an example of the configuration of an estimation device 1 according to the first embodiment. The estimation device 1 is a computer device that estimates the growth area of each crop (crop individual) growing in a field. The crop growth area is an area that can be used for crop growth. The estimation device 1 estimates the growth area of individual crops periodically (for example, every week) after the crops in the field germinate or sprout. This allows us to grasp the actual state of the growth area according to the growth situation of each individual crop.

The estimation device 1 includes, for example, an image acquisition unit 10, an image synthesis unit 11, a vegetation part extraction unit 12, an individual number estimation unit 13, an individual position determination unit 14, an inter-individual boundary determination unit 15, and an individual growth It includes an area calculation section 16 and a storage section 17.

The image acquisition unit 10 acquires aerial images. Aerial images are images of crops growing in a field as seen from above, and are captured using, for example, a drone equipped with an imaging device such as a digital camera. In this case, the drone continuously captures still images (aerial images) at regular intervals (for example, every 1 to 2 seconds) while flying back and forth over the field, dividing the drone into multiple images that capture the entire field. to be done. The image acquisition section 10 outputs the acquired aerial image to the image composition section 11.

The image composition unit 11 generates a composite image of the entire field (hereinafter simply referred to as a “composite image”) based on the aerial photographed image obtained by the image acquisition unit 10. The image synthesizing unit 11 generates an ortho image by joining images within a predetermined range from the center of a plurality of aerial images, and uses the generated ortho image as a composite image. In this case, the image synthesizing unit 11 may correct distortion occurring in the image based on height information of the soil and individual crops obtained from a plurality of aerial images.

In this embodiment, the composite image preferably has a spatial resolution that satisfies the accuracy required for image processing, which will be described later. The image processing described later is image processing performed by the image synthesis section 11, the vegetation part extraction section 12, the number of individuals estimation section 13, the individual position determination section 14, and the individual growth area calculation section 16. The spatial resolution required for the composite image is, for example, 1 [cm/pixel], that is, a resolution such that a range of about 1 [cm] in the actual space is imaged in 1 [pixel] of the image. In this case, for example, the spatial resolution of the aerial image is determined according to the spatial resolution required for the composite image, and the aerial image is captured using the determined spatial resolution.

Hereinafter, a case in which the estimation device 1 estimates a growth region using a composite image will be described as an example. However, it is not limited to this. The estimating device 1 only needs to be able to estimate the growing area using at least an image of the crop whose growing area is to be estimated and its surrounding area. For example, the estimation device 1 may use an image in which only a part of the field is captured to estimate the growth area of the individual crop imaged in the image. Further, from the viewpoint of reducing distortion that may occur in aerial images, it is preferable that the estimation device 1 estimates the growth region using an orthogonal image, but the estimation device 1 is not limited thereto. The estimation device 1 may estimate the growth area of the individual crop using the aerial photographed image before composition.

The vegetation portion extraction unit 12 extracts vegetation portions from the composite image. For example, the vegetation portion extraction unit 12 separates the vegetation portion and the soil portion in the composite image based on the difference in color between the two. In this case, the vegetation portion extraction unit 12 binarizes color information (for example, RGB values) for each pixel based on the color characteristics of the vegetation portion and the soil portion.

For example, the vegetation part extraction unit 12 selects a standard color assumed to be a vegetation part (for example, green: RGB values (0, 128, 0)) and a color whose distance in the color space is within a predetermined range. Converting a pixel to a particular first color (eg, white). The vegetation portion extraction unit 12 converts pixels of a standard color in the vegetation portion and a color whose distance in the color space is outside a predetermined range to a specific second color (for example, black). The vegetation portion extracting unit 12 extracts the portion converted to the first color (for example, white) from the composite image as a vegetation portion. The vegetation portion extraction unit 12 outputs information indicating the vegetation portion (for example, pixel coordinate values) in the composite image to the population estimation unit 13.

Here, the vegetation part extracted from the composite image is not only one extracted as a vegetation part consisting of one crop individual (see vegetation part R21 in FIG. 2), but also a state in which two or more crop individuals are connected. (See vegetation part R22 in FIG. 2). The estimation device 1 of this embodiment estimates the growth area of an individual crop. Therefore, in the estimation device 1, when a vegetation part is extracted from a composite image in a state where two or more crop individuals are connected, the extracted vegetation part is separated for each vegetation part corresponding to one crop individual. There is a need.

As a countermeasure for this, in the estimation device 1, the population estimation unit 13 estimates the number of plants included in the vegetation area. Then, the estimation device 1 estimates the growth area of each crop individual using a method suitable for each number of crops, depending on the number of crops included in the vegetation area. . As a result, the estimation device 1 can estimate the growth area of each crop individual even if a vegetation part is extracted from the composite image in a state where two or more crop individuals are connected. becomes possible.

The population estimation unit 13 estimates the number of crops included in the vegetation area extracted by the vegetation area extraction unit 12. Generally, it is estimated that the larger the area of a vegetation part, the greater the number of crops contained in that vegetation part. In addition, the vegetation part consisting of one crop individual has a substantially circular shape, and when the vegetation parts of multiple crop individuals are connected to form one vegetation part, the peripheral edge of the approximately circular shape of each crop individual It is estimated that the larger the overlapped shape and area, the larger the number of crops included in that vegetation area.

The population estimation unit 13 utilizes this property to estimate the number of crops included in the vegetation area based on the size and shape of each vegetation area. For example, the population estimation unit 13 uses the classification result by the support vector machine to separate the population of crops included in the vegetation area into two classes. In this case, the explanatory variables input to the support vector machine are features related to the area and shape of the vegetation portion. The shape-related feature may be any feature, but it is particularly preferable that it is a shape feature that corresponds to the number of crop individuals included in the vegetation portion. Thereby, by inputting features related to the shape, it is possible to make the results obtained from the support vector machine into results separated according to the number of crop individuals included in the vegetation portion.

For example, the population estimation unit 13 uses, as characteristics related to the shape of the vegetation part, the circumference of the vegetation part, circularity, long side length and short side length of the minimum circumscribed rectangle, RMS (Root Mean Square) error from the regression circle, and the radius of the regression circle. The perimeter is the length of the perimeter of the vegetation portion, and is derived, for example, by extracting pixels corresponding to the contour (edge) of the vegetation portion and depending on the number of extracted pixels. Circularity is an index indicating the circularity of the shape of a vegetation part. For example, the degree of circularity is derived according to the relationship between the area and perimeter of the actual vegetation portion, compared to the relationship between the area and perimeter when the shape of the vegetation portion is circular. The minimum circumscribed rectangle is the smallest rectangle that circumscribes the vegetation portion. A regression circle is a circle that can be fitted to the shape of a vegetation section.

First, the population estimation unit 13 inputs features related to the area and shape of all the vegetation parts extracted from the composite image to the support vector machine. Thereby, the support vector machine separates the vegetation part into a vegetation part made up of one crop individual and a vegetation part made up of multiple (two or more) crop individuals based on the input explanatory variables. (See Figure 3).

Next, the population estimation unit 13 uses a support vector machine to calculate features related to the area and shape of the vegetation part, which are classified into vegetation parts consisting of a plurality of (two or more) crop individuals, using a support vector machine. Enter into the machine. As a result, the support vector machine separates the vegetation part into a vegetation part made up of two crop individuals and a vegetation part made up of three or more crop individuals, for example, based on the input explanatory variables (Figure 4 reference).

Furthermore, the population estimation unit 13 inputs characteristics related to the area and shape of the vegetation part into the support vector machine, targeting the vegetation part classified into a vegetation part consisting of three or more crop individuals by the support vector machine. . Based on the input explanatory variables, the support vector machine can, for example, separate the vegetation part into a vegetation part made up of three crop individuals and a vegetation part made up of more (four or more) crop individuals. (See Figure 5).

In this way, for example, the population estimation unit 13 estimates the number of crop individuals included in the vegetation area using the results obtained from the support vector machine while narrowing down the targets to be separated.

Note that if the number of crops included in a vegetation area increases, the accuracy of separation using a support vector machine may deteriorate. For example, it is conceivable that a vegetation part made up of four crop individuals and a vegetation part made up of more (five or more) crop individuals have similar characteristics regarding area and shape. If no features are found that can separate vegetation parts by number of individuals, the accuracy of separation using a support vector machine may deteriorate.

As a countermeasure against this, the number of individuals estimating unit 13 may set in advance an upper limit of the number of individuals to be separated by the support vector machine. For example, the number of individuals estimating unit 13 estimates the number of individuals using a support vector machine within a range that can be accurately separated by the support vector machine. For example, the population estimation unit 13 uses the support vector machine to separate a vegetation part whose population is known, based on whether or not the separation matches the actual population number when the support vector machine separates the vegetation part. Determine the upper limit on the number of individuals that will undergo this process.

In addition, although the case where the population estimation part 13 performs classification using a support vector machine was explained above as an example, it is not limited to this. The number of individuals estimating unit 13 may estimate the number of individuals using at least the results of classification based on the area and shape of the vegetation portion. For example, the population estimation unit 13 estimates the number of crop individuals included in the vegetation area by referring to a table in which the combination of area and shape of the vegetation area is associated with the number of individuals in that vegetation area. You can do it like that.

The individual position determination unit 14 determines the individual position of the crop individual in the vegetation area. The individual position is the position of the individual crop used when estimating the growing area. The individual position determination unit 14 determines whether the crop individuals included in the vegetation area are
(Case 1) When there is one individual. (Case 2) When there are two or three individuals. (Case 3) When there are four or more individuals. The individual positions are determined using different methods depending on each case.

First, a method will be described in which the individual position determination unit 14 determines the individual position of the crop individual in the vegetation area in the case (Case 1) in which there is only one crop individual included in the vegetation area. When the number of crop individuals included in the vegetation portion is one individual, the individual position determining unit 14 sets the center of gravity position of the vegetation portion as the individual position of the crop individual in the vegetation portion (see FIG. 6).

Next, a method for the individual position determination unit 14 to determine the individual positions of the crop individuals in the vegetation area in the case where there are two or three crop individuals included in the vegetation area (case 2) will be described. First, a case will be described in which there are two crop individuals included in the vegetation area. The individual position determination unit 14 defines the main axis in the vegetation portion. The main axis here is a straight line passing through points assumed to be the individual positions of the two crop individuals included in the vegetation portion (see main axis SG in FIG. 7).

The vegetation area tends to spread concentrically around the individual position of the crop. If the vegetation area contains two individual crops, two approximately circular shapes corresponding to the two individual crops overlap at the adjacent periphery, and the central part of the ellipse is depressed. It will have a shape like this. A straight line passing through the centers of the two circles in this case corresponds to the main axis in this embodiment.

The principal axis defined as above can be regarded as the axis that maximizes the variation in pixel distribution in the vegetation area. Utilizing this property, the individual position determination unit 14 determines the principal axis using, for example, principal component analysis. In this case, the first principal component axis obtained by principal component analysis of variations in pixel distribution in the vegetation portion corresponds to the principal axis of the vegetation portion in this embodiment.

Specifically, the individual position determining unit 14 first defines a coordinate system (hereinafter referred to as an xy coordinate system) whose origin is the average coordinate of the position coordinates of pixels forming the outline of the vegetation portion. The individual position determining unit 14 expresses the position coordinates of each pixel forming the outline of the vegetation portion using an xy coordinate system. The individual position determining unit 14 defines a straight line TS (ax+by=0) passing through the origin of the xy coordinate system. a and b are arbitrary real numbers. The individual position determination unit 14 sets the shortest distance from the xy coordinates (x _i , y _i ) of pixels forming the contour of the vegetation portion to the straight line TS as an error e _i . However, i is any natural number from 1 to N, where N is the total number of pixels forming the outline of the vegetation portion. The error e _i is derived by the following (Formula 1).

e _i = | ax _i , by _i |/√(a ² + b ² )...(Formula 1)

The individual position determining unit 14 derives a combination of a and b that satisfies the condition that the sum of errors e _i in all pixels forming the vegetation portion is the minimum. A combination of a and b that satisfies this condition is expressed by the following (Formula 2). α and β in (Equation 2) are a combination of a and b that satisfy the condition that the sum of errors e _i is the minimum. Further, the range to be added by Σ in (Equation 2) is from (i=1) to (i=N). N is the total number of pixels forming the outline of the vegetation area. When (Equation 2) is modified, it becomes a problem to find the eigenvectors (α, β) of variance and covariance for the original data (pixels forming the outline of the vegetation area).

α, β=argmin {Σe _i ² } … (Formula 2)

The individual position determination unit 14 uses a straight line TS (αx+βy=0) defined by α and β that satisfies (Equation 2) as its main axis.

Alternatively, the individual position determination unit 14 may use a straight line passing through the vegetation part that has a long distance as the main axis. In this case, the individual position determination unit 14 sets the main axis to the straight line connecting any two points extracted from the pixels forming the outline of the vegetation portion, the straight line having the maximum distance.

Alternatively, it is conceivable that the individual position determination unit 14 uses a regression line of pixels forming the contour of the vegetation portion as the main axis. This is because the principal axis in the above-mentioned principal component analysis and the regression line are common in that they seek a straight line TS that minimizes the sum of distances to the straight line.

However, it is difficult to use a regression line as a method for determining the principal axis of a vegetation area. This is because the method of calculating the error e _i is different between the principal axis of the principal component analysis and the regression line. In principal component analysis, the shortest distance from a pixel to a straight line is defined as an error. On the other hand, in the regression line, the distance from the pixel to the straight line in the y-axis direction is defined as an error. Therefore, if the vegetation part has a vertically long shape in the y-axis direction, the slope of the straight line that minimizes the sum of errors (absolute value of a/b) tends to become large, and an unintended straight line It may be derived as For this reason, in this embodiment, when the vegetation part has a vertically elongated shape in the y-axis direction, it is not preferable to use the regression line in the vegetation part as the main axis.

The individual position determination unit 14 creates a histogram (see histogram HG in FIG. 7) in which the number of pixels of the vegetation portion is projected in a direction horizontal to the main axis (see direction SH in FIG. 7). The inter-individual boundary determination unit 15 performs discriminant analysis to divide the distribution into two in the created histogram. For example, the individual position determining unit 14 determines, in the histogram, a location corresponding to a position where the distribution number of pixels changes from decreasing to increasing as a boundary for dividing the distribution into two by discriminant analysis (see boundary threshold KS in FIG. 7). .

The individual position determination unit 14 determines two points (endpoints EP1 and EP2 in FIG. 7) where the contour of the vegetation portion intersects with the main axis as endpoints in the vegetation portion. The individual position determination unit 14 locates one of the two crop individuals included in the vegetation portion by the midpoint (point GP1 in FIG. 7) of a straight line connecting the intersection of the principal axis and the boundary threshold value KS from the end point EP1 on the principal axis. Let be the individual position of the crop individual. In addition, the individual position determination unit 14 locates the midpoint (point GP2 in FIG. 7) of the straight line connecting the intersection of the principal axis and the boundary threshold value KS from the end point EP2 on the principal axis to one of the two crop individuals included in the vegetation area. , is the individual position of the other crop individual.

Even when the number of crop individuals included in the vegetation part is three, the individual positions of the crop individuals can be determined by applying the same method as when the number of crop individuals included in the vegetation part is two. In this case, the individual position determination unit 14 performs discriminant analysis to divide the distribution into three in the created histogram.

Next, a method will be described in which (Case 3) the individual position determination unit 14 determines the individual positions of the crop individuals in the vegetation area when the number of crop individuals included in the vegetation area is four or more.

When the number of crop individuals included in the vegetation part is four or more, it becomes difficult to distinguish three or more depressed parts in the long elliptical shape of the vegetation part (see region R51 in FIG. 8). Therefore, it is difficult to determine the individual position by applying a method similar to (Case 2).

Generally, when the growth of crops has progressed to a certain extent, four or more crops will appear to be connected to each other. The individual position determination unit 14 utilizes this property and determines the current position of the crop individual using the position of the crop individual in the previous composite image.

Specifically, the individual position determination unit 14 determines the individual position of each individual crop based on a composite image based on aerial images captured before four or more crops are connected. In this case, since the vegetation part includes less than four individual crops, the individual positions can be determined using (Case 1) or (Case 2). The individual position determining unit 14 stores the determined individual position in association with, for example, a composite image.

If it is estimated that four or more crops are included in the vegetation part extracted in the current composite image, the individual position determination unit 14 does not determine the individual position using the vegetation part. Instead, the individual position determination unit 14 determines the individual position in the area corresponding to the vegetation part for which the individual position has not been determined this time (region R31 or R41 in FIG. 8) in the previous composite image. Let it be the individual position.

The inter-individual boundary determination unit 15 determines the boundary between individuals (hereinafter also referred to as inter-individual boundary). The inter-individual boundary is the boundary between the growth areas of a crop individual and its neighboring crop individuals. In general, it is considered that a crop individual with a large area of vegetation has a wider growing area than an individual crop with a small area of vegetation. In order to reflect this property, the inter-individual boundary determination unit 15 determines the inter-individual boundary based on the individual position of the crop individual determined by the individual position determination unit 14 and the covered area (area of the vegetation part) of the individual crop. Determine boundaries.

The inter-individual boundary determining unit 15 focuses on two crop individuals (vegetation portions SB3 and SB4) whose inter-individual boundaries are to be determined. The inter-individual boundary determining unit 15 defines a rectangular coordinate system in which the origin is the midpoint of the line segment connecting each of the two individual positions, and the x-axis is the straight line connecting each of the two individual positions (Fig. 9 reference).

The inter-individual boundary determining unit 15 sets the hyperbola (x ² /A ² -y ² /B ² =1) with the two individual positions as focal points as the inter-individual boundary (boundary RK34 in FIG. 9), and calculates the coefficients A and B. is determined by the following method.

First, the inter-individual boundary determination unit 15 calculates the covered area (area of the vegetation part) of the two crop individuals, and sets the areas to w1 and w2, respectively. Furthermore, the inter-individual boundary determining unit 15 sets the xy coordinates of the positions of the two crop individuals to (-L, 0) and (L, 0), respectively. Furthermore, the inter-individual boundary determination unit 15 divides the distance 2L between the two individual positions at a ratio of area w1:w2, and sets the distances to p and q (see FIG. 9). In this case, p and q can be expressed by the following (Formula 3).

p={2L/(w1+w2)}w1,
q={2L/(w1+w2)}w2...(Formula 3)

The inter-individual boundary determination unit 15 uses p and q shown in (Formula 3) to express the following (Formula 4) so that the xy coordinates of the vertex of the hyperbola become (p-L, 0). Determine A and B as follows.

A=(p-q)/2,
B=√(pq)
However, if w1>w2, x>0
If w1<w2, x<0
When w1=w2, the inter-individual boundary coincides with the y-axis...(Equation 4)

The individual growth area calculation unit 16 calculates the growth area of an individual crop. The individual growth area calculation unit 16 focuses on the individual crop whose growth area is to be calculated (vegetation portion SB5 in FIG. 10). The individual growth area calculation unit 16 obtains the inter-individual boundaries between the focused crop individual and each of the adjacent crop individuals (vegetation parts SB7 to SB10 in FIG. 10). The inter-individual boundary here is determined by the inter-individual boundary determination unit 15. The individual growth area calculation unit 16 sets the area surrounded by the acquired boundary (region SR in FIG. 10) as the growth area of the individual crop.

The functional units of the estimation device 1 described above (image acquisition unit 10, image synthesis unit 11, vegetation part extraction unit 12, number of individuals estimation unit 13, individual position determination unit 14, inter-individual boundary determination unit 15, and individual growth area calculation The unit 16) is realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components are hardware (circuit parts) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit). (including circuitry), or may be realized by collaboration between software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as an HDD (Hard Disk Drive) or flash memory, or may be stored in a removable storage device such as a DVD or CD-ROM. It is stored in a medium (non-transitory storage medium), and may be installed by loading the storage medium into a drive device.

The storage unit 17 includes, for example, an HDD (Hard Disk Drive), a flash memory, a RAM (Random Access Memory), and the like. The storage unit 17 stores, for example, image information. For example, the storage unit 17 stores image information of aerial images. In this case, the image acquisition unit 10 causes the 17 to store information on the acquired aerial image. The storage unit 17 may store image information of the composite image. In this case, the image composition section 11 causes the storage section 17 to store image information of the generated composite image.

The storage unit 17 may store image information of a composite image showing the vegetation portion extracted by the image composition unit 11. The storage unit 17 may store information indicating the number of individuals included in the vegetation portion estimated by the population estimation unit 13. The storage unit 17 may store information indicating the individual position determined by the individual position determination unit 14. The storage unit 17 may store image information of a composite image showing the inter-individual boundary determined by the inter-individual boundary determination unit 15. The storage unit 17 may store image information of a composite image showing the growth area of the individual crop calculated by the individual growth area calculation unit 16. The storage unit 17 also stores other information, such as a program to be executed by the CPU of the estimation device 1 and various parameters used when the program is executed.

FIG. 2 is a diagram illustrating processing performed by the vegetation portion extraction unit 12 according to the first embodiment. On the left side of FIG. 2, an image G1 showing a part of the composite image is shown. On the right side of FIG. 2, a binary image G2 is shown in which the vegetation part extracted from the image G1 is shown in white, and the part different from the vegetation part is shown in black. In this manner, the vegetation portion extraction unit 12 extracts the vegetation portion by binarizing the color of each pixel in the image G1 based on the color of the vegetation portion. Note that in the region R20 in the image G2, an example of a vegetation portion including two individual crops is shown. Further, in the region R21 in the image G2, an example of a vegetation portion consisting of one individual crop is shown.

FIGS. 3 to 5 are diagrams illustrating the processing performed by the population estimation unit 13 according to the first embodiment. 3 to 5 illustrate multidimensional vector spaces generated by a support vector machine based on features such as the shape of vegetation sections. In each vector space, each feature of the vegetation portion extracted from the composite image is indicated by positional coordinates according to the feature axis in the vector space.

FIG. 3 shows how the vegetation is divided into a class C1 in which the number of individuals included in the vegetation is one, and a class C2 in which the number of individuals included in the vegetation is two or more. The support vector machine presents a separation surface for classes C1 and C2 as shown in FIG. 3, for example, based on explanatory variables input for all the vegetation parts extracted from the composite image.

Figure 4 shows how class C2, in which the number of individuals included in the vegetation part is two or more, is separated into class C21, in which the number of individuals is two, and class C22, in which the number of individuals is three or more. There is. The support vector machine presents a separation surface between classes C21 and C22 as shown in FIG. 4, for example, based on explanatory variables input for the vegetation portion classified into class C2.

FIG. 5 shows how class C22, in which the number of individuals included in the vegetation part is three or more, is separated into class C221, in which the number of individuals is three, and class C222, in which the number of individuals is four or more. There is. The support vector machine presents a separation surface between classes C221 and C222, as shown in FIG. 5, for example, based on explanatory variables input for the vegetation portion classified into class C22.

6 to 8 are diagrams illustrating the processing performed by the individual position determination unit 14 according to the first embodiment. FIG. 6 shows the individual position when the number of crops included in the vegetation part is one, that is, the individual position determined by the method of (Case 1). Image G2 in FIG. 6 is a composite image in which the vegetation portion is converted to white, similar to image G2 in FIG. Image G20 is an enlarged version of the vegetation portion (vegetation portion including one crop individual) in image G2. In this case, the individual position determination unit 14 sets the position of the center of gravity GP1 of the vegetation portion SB1 as the individual position based on the method of (Case 1).

FIG. 7 shows the individual positions when the number of crops included in the vegetation part is two or three, that is, the individual positions determined by the method of (Case 2). Image G2 in FIG. 7 is a composite image in which the vegetation portion is converted to white, similar to image G2 in FIG. Image G21 is an enlarged version of the vegetation portion (vegetation portion including two individual crops) in image G2. In this case, the individual position determining unit 14 determines the main axis SG of the vegetation portion SB2 based on the method of (Case 2). The individual position determination unit 14 creates a histogram HG of pixels forming the vegetation portion SB along the direction SH perpendicular to the determined main axis SG. The individual position determining unit 14 determines the position of the boundary threshold KS based on the created histogram. The individual position determination unit 14 sets a point that equally divides the distance from the end point EP1 of the vegetation portion SB2 to the intersection of the main axis and the boundary threshold KS as one individual position GP1. The individual position determining unit 14 sets the point that equally divides the distance from the end point EP2 of the vegetation portion SB2 to the intersection of the main axis and the boundary threshold KS as the other individual position GP2.

FIG. 8 shows the individual position when the number of crops included in the vegetation part is four or more, that is, the individual position determined by the method of (Case 3). FIG. 8 shows images G3 to G5, which are three composite images captured on different days. Images G3 to G5 all show the same area in the field. Among images G3 to G5, image G3 is the earliest date (for example, June 7th), image G4 is the runner-up date (for example, June 9th), and image G5 is the latest date (for example, June 11th). The image was taken on the following day.

When a vegetation part in which four or more crops are connected is observed, as in region R51 in FIG. 8, the individual position determining unit 14 cannot determine the individual position in region R51 as it is. Therefore, the individual position determination unit 14 converts the individual position determined in region R41 of image G4 captured two days ago or region R31 of image G3 captured four days ago into the individual position determined in region R51 of image G5. shall be.

In order to more stably determine the individual position in (Case 3), it is conceivable to take aerial images more frequently (for example, every day or every few days). The crops are fixed and do not move during the growing process. For this reason, it is considered that if the individual position is determined at least once during the growing period of the crop, there is no need to determine the individual position from the composite image on subsequent days.

However, by determining the individual position each time, it is possible to improve the accuracy of the individual position. For example, for an individual position determined each time, if the individual positions determined multiple times (for example, three times) are all within a predetermined range, it is possible to consider that the individual position indicates an accurate position. .

Using this, for example, the individual position determination unit 14 determines the likelihood (likelihood) of the individual position based on the degree of variation in the individual position in multiple times (for example, three times) for the individual position determined each time. may be determined.

FIG. 9 is a diagram illustrating processing performed by the inter-individual boundary determination unit 15 according to the first embodiment. FIG. 9 shows a boundary RK34 that is an inter-individual boundary between two vegetation parts SB3 and SB4. As shown in FIG. 9, the inter-individual boundary determining unit 15 focuses on the individual positions (-L, 0) and (L, 0) of the two vegetation parts SB3 and SB4, and the distance 2L between the vegetation parts SB3 and SB4. A hyperbola whose apex is the point (p-L, 0) that divides the area at a ratio (w1:w2) according to the areas w1 and w2 of the vegetation parts SB3 and SB4 is defined as the inter-individual boundary between the vegetation parts SB3 and SB4. Assume that a certain boundary is RK34.

FIGS. 10 to 16 are diagrams illustrating processing performed by the individual growth area calculation unit 16 according to the first embodiment. FIG. 10 shows the growth region SR of the vegetation portion SB5 of the individual crop. As shown in FIG. 10, the individual growth area calculation unit 16 obtains the boundaries between the vegetation portion SB5 and each of the vegetation portions SB7 to SB10 of adjacent crop individuals from the inter-individual boundary determination unit 15. The boundary between the vegetation parts SB5 and SB6 is the boundary RK56. The boundary between the vegetation parts SB5 and SB7 is the boundary RK57. The boundary between the vegetation parts SB5 and SB8 is the boundary RK58. The inter-individual boundary between the vegetation parts SB5 and SB9 is the boundary RK59. The individual boundary between the vegetation parts SB5 and SB10 is the boundary RK510. The individual growth area calculation unit 16 calculates the area surrounded by the boundaries (boundaries RK56, RK57, RK58, RK59, and RK510) between the vegetation portion SB5 and each of the vegetation portions SB7 to SB10 of adjacent crop individuals. Let it be the growth region SR.

FIG. 11 shows the growth area of each individual crop included in each of the vegetation parts extracted in the image G10. As shown in FIG. 11, for crops planted at the edge of a field, the boundary between the field and the outside of the field may be applied as the boundary on the side where there are no adjacent crop individuals.

FIG. 12 shows how the growing area of a crop changes as the crop grows. If the areas of the vegetation parts of adjacent crops are the same, the boundary will be a line that passes through the center of the straight line connecting the individual positions and is almost perpendicular to the straight line connecting the individual positions (see photo date 1). As the growth of crops progresses and a difference appears in the area of the vegetation parts of adjacent crops, the boundary between the individual crops is determined by dividing the straight line connecting the individual positions into the area by the ratio (the side with the smaller area). It becomes a hyperbola with the apex at (position according to) (see photo date 2). When the difference in area between the vegetation parts of adjacent crops increases, the boundary between the individuals becomes a hyperbolic curvature that is further increased (see photographing date 3). When the vegetation parts of adjacent crops are connected, the boundary determined before the vegetation parts are connected is adopted as the final boundary (see shooting date 4).

13 to 16 show interindividual boundaries determined at each stage from the stage when crops begin to germinate in the field until the vegetation parts are connected. At the stage when crops begin to germinate, a new individual position is detected every time an aerial image is taken, so a new growth area is determined based on the newly detected individual position (Fig. 13, and Figure 14). Since the inter-individual boundary is determined using the area of the vegetation part as a weight, the shape of the growing region changes every time an image is taken during the growing stage of the crop (see FIGS. 14 and 15). The shape of the growing area is updated until the vegetation parts of adjacent crops are connected (see Figures 15 and 16).

FIG. 17 is a flow diagram showing the flow of processing performed by the estimation device 1 according to the first embodiment. The estimation device 1 acquires an aerial image of a field taken from above by a drone or the like (step S1). The estimation device 1 connects and synthesizes the acquired aerial images to generate a composite image (orthoimage) of the entire field (step S2). The estimation device 1 extracts the vegetation portion in the composite image by, for example, binarizing the colors of the composite image (step S3). The estimation device 1 estimates the number of crop individuals included in each of the extracted vegetation parts by using a support vector machine or the like (step S4). The estimation device 1 determines the individual positions of the crop plants by using principal component analysis or the like according to the estimated number of plants (step S5). The estimation device 1 uses the determined individual position to determine a boundary (inter-individual boundary) weighted by the covered area (area of the vegetation part) of adjacent crop individuals (step S6). The estimation device 1 calculates the area surrounded by the boundary determined between the individual crop and the adjacent crop as the growth area of the individual crop (step S7).

(Second embodiment)
A second embodiment will be described. FIG. 18 is a block diagram showing an example of the configuration of an estimation device 100 according to the second embodiment. The estimation device 100 is a computer device that estimates the yield of each crop (individual crop) grown in a field. The estimation device 100 includes, for example, an image acquisition section 110, a three-dimensional information restoration section 111, an above ground volume calculation section 112, a parameter determination section 113, a harvest amount estimation section 114, and a storage section 117.

The image acquisition unit 110 acquires aerial images. Aerial images are images of crops growing in a field as seen from above, and are captured using, for example, a drone equipped with an imaging device such as a digital camera. In this case, the drone continuously captures still images (aerial images) at regular intervals (for example, every 1 to 2 seconds) while flying back and forth over the field, dividing the drone into multiple images that capture the entire field. to be done.

In this embodiment, after the crops in the field germinate or sprout, aerial photography is performed periodically (for example, every week). As a result, aerial images showing the growth process of crops are captured in chronological order. The image acquisition unit 110 acquires an aerial image every time an image is captured, and outputs the acquired aerial image to the image composition unit 11.

The three-dimensional information restoration unit 111 restores three-dimensional information on the surface of the field based on the aerial image obtained by the image acquisition unit 110. The three-dimensional information here is information in which the depth (depth information) of each pixel is added to the position coordinates of each pixel of an aerial photographed image, or three-dimensional point group information. For example, the three-dimensional information restoration unit 111 calculates the depth of each pixel in the image using three-dimensional reconstruction technology that applies the principle of a stereo camera to a plurality of aerial images taken from different imaging positions. . Alternatively, a three-dimensional point group on the field surface may be obtained by measuring with a distance sensor such as LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging). A distance sensor such as LiDAR here is an example of a "measuring device that measures distance." The three-dimensional information restoration unit 111 outputs three-dimensional information to the above ground volume calculation unit 112.

The above-ground part volume calculating unit 112 calculates the volume of the above-ground part of each crop in the field (hereinafter also referred to as above-ground part volume) based on the three-dimensional information restored by the three-dimensional information restoring unit 111. The above-ground part here refers to the part of the crop that is above the ground, that is, the leaf part of the crop.

For example, the above-ground volume calculation unit 112 first extracts three-dimensional information on crops from an aerial photographic image by subtracting three-dimensional information on soil from three-dimensional information on a field. The aboveground volume calculation unit 112 acquires three-dimensional information (see FIG. 19) of the field in a state where there are no crops (bare land). A bare field is, for example, an aerial image of a field taken before the crops germinate. The aboveground volume calculation unit 112 acquires three-dimensional information (see FIG. 20) of the field in a state where there are crops. The field with crops is an aerial image of the field for which the above-ground volume of individual crops is calculated.

The above-ground volume of an individual crop can be expressed by the following (Equation 1). In (Formula 1), Vn is the above-ground volume of the n-th crop individual. Rn is the area of the nth crop individual. g(x, y, z) is three-dimensional information of a bare field. f(x, y, z) is three-dimensional information of a field with crops.

Vn=∫∫∫ _Rn {f(x, y, z)-g(x, y, z)}dxdydz...(Formula 1)

The above-ground volume calculation unit 112 subtracts the three-dimensional information of the bare field from the three-dimensional information of the field with crops, using (Formula 1) to obtain three-dimensional information of only the crops in the field. Extract. The above-ground part volume calculation unit 112 calculates the above-ground part volume of the individual crop by spatially integrating the extracted three-dimensional information of the crop over the area of the individual crop. The aboveground volume calculation unit 112 outputs the calculated aboveground volume of the crop individual to the parameter determination unit 113.

The aboveground volume calculation unit 112 needs to know the area of each crop when performing spatial integration. As the area of the individual crop, for example, the growing area of the individual crop in the first embodiment is applied.

In this embodiment, a crop growth model is used to estimate the yield of an individual crop. A crop growth model is a numerical model that derives the growth status of crops through simulation. The crop growth conditions derived by the crop growth model are, for example, the above-ground dry mass amount and the underground dry mass amount. Above-ground dry mass is the amount of dry matter in the above-ground part (leaf) of a crop. The amount of underground dry matter is the amount of dry matter in the underground part (tuber part) of a crop. The dry mass is the weight of a crop, excluding the amount of water contained in the crop. In the case of root vegetables such as potatoes, the amount of underground dry matter corresponds to the yield.

Generally, it is known that the above-ground volume of an individual crop has a linear relationship with the above-ground dry mass of the crop individual (see FIG. 21). Here, it is thought that the amount of dry matter above ground has a correlation with the volume of above ground. In this embodiment, the internal parameters of the crop growth model are determined using the relationship between the above-ground volume and the above-ground dry mass amount of the individual crop.

Specifically, the parameter determination unit 113 adjusts the crop growth model so that the volume of the individual crop obtained as a result of executing the simulation of the crop growth model approaches the volume of the individual crop calculated by the above-ground volume calculation unit 112. Determine internal parameters. This allows the crop growth model to reflect the actual crop growth situation. Therefore, crop yield can be estimated with high accuracy by simulating crop growth models.

Further, the parameter determining unit 113 determines that the time-series change in the volume of the individual crop obtained as a result of executing the simulation of the crop growth model approaches the time-series change in the volume of the crop individual calculated by the above-ground volume calculation unit 112. Determine the internal parameters of the crop growth model. This allows the crop growth model to accurately predict future crop growth conditions. Therefore, crop yield can be estimated with high accuracy by simulating crop growth models. Note that the internal parameters of the crop growth model will be explained in detail later.

The yield estimating unit 114 estimates the yield of an individual crop using a crop growth model in which the internal parameters determined by the parameter determining unit 113 are set. For example, the yield estimating unit 114 inputs input variables (for example, meteorological data, etc.) to the crop growth model, and causes the crop growth model to output information indicating a time-series change in the amount of underground dry matter of the crop. The yield estimating unit 114 estimates the amount of underground dry matter corresponding to the harvest time, out of the time-series changes in the amount of underground dry matter, as the yield of the individual crop.

The functional units of the estimation device 100 (the image acquisition unit 110, the three-dimensional information restoration unit 111, the above-ground volume calculation unit 112, the parameter determination unit 113, and the harvest amount estimation unit 114) described above include, for example, a CPU (Central Processing Unit). ) is realized by executing a program (software). Some or all of these components are hardware (circuit parts) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit). (including circuitry), or may be realized by collaboration between software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as an HDD (Hard Disk Drive) or flash memory, or may be stored in a removable storage device such as a DVD or CD-ROM. It is stored in a medium (non-transitory storage medium), and may be installed by loading the storage medium into a drive device.

The storage unit 117 includes, for example, an HDD (Hard Disk Drive), a flash memory, a RAM (Random Access Memory), and the like. The storage unit 117 stores, for example, image information. For example, the storage unit 17 stores image information of aerial images. In this case, the image acquisition unit 110 causes the storage unit 117 to store information on the acquired aerial image. The storage unit 117 may store the three-dimensional information restored by the three-dimensional information restoration unit 111. The storage unit 17 may store information indicating the above-ground volume calculated by the above-ground volume calculation unit 112. The storage unit 117 may store the internal parameters determined by the parameter determination unit 113 in association with individual crops. The storage unit 117 may store the yield estimated by the yield estimation unit 114 in association with each crop. The storage unit 117 also stores other information, such as a program to be executed by the CPU of the estimation device 100 and various parameters used when the program is executed.

19 and 20 are diagrams showing examples of images acquired by the image acquisition unit 110 according to the second embodiment. FIG. 19 shows an example of three-dimensional information of a bare field. FIG. 20 shows an example of three-dimensional information of a field with crops. In this way, the three-dimensional information may be information that is associated with pixels in an aerial photographed image and indicates the depth of the image using shading or the like.

FIG. 21 is a diagram showing the relationship between the above ground volume and the dry matter amount calculated by the above ground volume calculation unit 112 according to the second embodiment. In Fig. 4, the horizontal axis shows the above-ground volume, and the vertical axis shows the above-ground dry mass amount. As shown in FIG. 21, the volume of the above-ground part and the amount of dry matter of the above-ground part are correlated, and as the volume of the above-ground part increases, the amount of dry matter of the above-ground part also tends to increase.

Here, the internal parameters of the crop growth model will be explained using FIG. 22. FIG. 22 is a diagram illustrating a model in which internal parameters are determined by the parameter determination unit 113 according to the second embodiment.

As shown in the example of FIG. 22, the crop growth model uses, for example, weather data _Tk as an input variable. k is the number of days after germination. The weather data _Tk is expressed as a set of weather data _tk from the germination date to the present (k days have passed). The weather data _tk is information indicating the weather of the day, such as minimum temperature, maximum temperature, average temperature, solar radiation energy (PAR, Photosynthetically Active Radiation), and day length.

The crop growth model outputs growth conditions according to the growing season. For example, as shown in the example in Figure 5, at the time of germination (k = 0) in late May and the time of tuber formation in mid-June, the amount of aboveground dry matter increases, while the amount of underground dry matter increases. It doesn't increase much. During the growth peak in mid-July and the yellowing season in mid-August, both the above-ground dry matter amount and the underground dry matter amount increase. Then, in late August, when the leaves are completely withered, the aboveground dry mass decreases, and the underground dry mass does not increase much.

The internal parameters of the crop growth model are set in accordance with the growing season and the like. For example, the daily increase amount ΔW _k in the amount of above-ground dry matter is expressed as the increase amount f (T _k ) of above-ground dry matter minus the decrease amount g (T _k ). The growth rate in the increase amount f(T _k ) can be set according to the respective periods immediately after germination, during the initial growth period, and when the aboveground cover is saturated, using the internal parameters a ₁ , a ₂ , and a ₃ . The period from leaf formation to withering in the amount of decrease g (T _k ) can be set by the internal parameter _a4 .

The above-ground dry matter amount W _k+1 is expressed as the above-ground dry matter amount W _k of the previous day plus the above-ground dry matter increase amount ΔW _k on that day.

The above-ground volume V _k is expressed as the above-ground dry mass W _k multiplied by a function h(k). The behavior of the function h(k) can be set by internal parameters _{a5 and a6} .

The amount of increase ΔY _k in the amount of underground dry matter is expressed as the above-ground received light energy multiplied by the conversion ratio L(T _k ) to the amount of underground dry matter. The above-ground part received light energy is expressed as the solar radiation energy PAR _k multiplied by the above-ground part volume V _k . The conversion ratio L( _Tk ) can be set according to the product type, etc. using the internal parameter _a7 .

The underground dry matter amount Y _k+1 is expressed as the sum of the underground dry matter amount Y _k of the previous day and the underground dry matter increase amount ΔY _k on that day.

Here, the method by which the parameter determination unit 113 determines internal parameters will be explained using FIG. 23. FIG. 23 is a diagram illustrating processing performed by the parameter determination unit 113 according to the second embodiment. In the upper left, lower left, and right graphs of FIG. 23, the horizontal axes indicate the time period, and the vertical axes indicate the aboveground volume.

As shown in the upper left graph of FIG. 23, there are countless patterns of time-series changes in aboveground volume depending on the internal parameters of the crop growth model. As mentioned above, there are many internal parameters in crop growth models. This is because the behavior of the crop growth model differs depending on the combination of values set for the internal parameters.

On the other hand, as shown in the graph at the bottom left of Figure 23, the time-series changes in ground volume derived from aerial images (denoted as drone images in Figure 23) are unique each time because they are calculated from the images. determined.

The parameter determining unit 113 performs data assimilation by determining internal parameters so that the time-series change in the above-ground volume based on the crop growth model approaches the time-series change in the above-ground volume derived from the aerial photographic image. Data assimilation may be performed using any method, such as the Markov chain Monte Carlo method (MCMC), the least squares method, or a particle filter.

As shown in the graph on the right of Figure 23, the time-series changes in the above-ground volume output from the crop growth model whose internal parameters have been determined by data assimilation are the same as the time-series changes in the above-ground volume derived from the aerial images. It will be a similar pattern. For example, the parameter determining unit 113 extracts, from among the simulation results based on the crop growth model, those that closely resemble the time-series changes in above-ground volume derived from aerial images. The parameter determination unit 113 sets the internal parameters set in the extracted crop growth model subjected to the simulation as the internal parameters of the model for estimating the yield.

FIG. 24 is a diagram illustrating an example in which the above-ground volume derived from the aerial image according to the second embodiment is compared with the above-ground volume shown by the simulation results. FIG. 24 shows the time-series changes in the aboveground volume of each of the 16 individual crops. In each graph in FIG. 24, the horizontal axis indicates the time period, and the vertical axis indicates the above-ground volume.

As shown in FIG. 24, by appropriately setting internal parameters to match the actually observed time-series changes in above-ground volume, it is possible to faithfully simulate the growth status of individual crops. In this way, by using a model with appropriately set internal parameters, it is possible to estimate the yield more accurately.

25 to 28 are diagrams showing examples of simulation results using the model according to the second embodiment. FIG. 25 shows the time-series change in the above-ground volume V _k together with the observed value (the above-ground volume derived from the aerial image). FIG. 26 shows a time-series change in the above-ground dry matter increase amount ΔW _k of a model in which the same internal parameters as in FIG. 25 are set. FIG. 27 shows a time-series change in the underground dry matter increase amount ΔY _k of a model in which the same internal parameters as in FIG. 25 are set. FIG. 28 shows a time-series change in the integral value of the underground dry matter increase amount ΔY _k shown in FIG. 27 as the underground dry matter amount Y _k . If the time-series change in above-ground volume closely resembles the time-series change in the actual situation (above-ground volume derived from aerial images), it can be said that the model accurately represents the actual crop growth situation. I can guess. In this way, a model that accurately represents the actual growth status of crops can produce changes that closely resemble actual changes in the time-series changes in the above-ground dry matter increase ΔW _k and the below-ground dry matter increase ΔY _k . It is thought that Therefore, it can be expected that the yield estimated using this model accurately represents the actual yield.

FIG. 29 is a diagram illustrating an example of a comparison result between the harvest amount estimated using the model according to the second embodiment and the actual harvest amount. The horizontal axis of FIG. 12 shows the actual harvest amount, and the vertical axis shows the estimated harvest amount. As shown in FIG. 12, there is a correlation between the estimated harvest amount and the actual harvest amount, indicating that the model can accurately predict the harvest amount.

FIG. 30 is a flow diagram showing the flow of processing performed by the estimation device 100 according to the second embodiment. The estimation device 100 acquires an aerial image of a field taken from above by a drone or the like (step S11). The estimation device 100 uses the acquired aerial images to restore three-dimensional information of the field by three-dimensional reconstruction technology (step S12). The estimation device 100 calculates the above-ground volume of the individual crop (step S13). For example, the estimation device 100 subtracts the three-dimensional information of the bare field from the three-dimensional information of the field, and spatially integrates the three-dimensional information after the subtraction with respect to the area of the individual crop, thereby calculating the above-ground volume of the individual crop. calculate. The estimation device 100 acquires internal parameters (step S14). The estimation device 100 sets internal parameters such that the time-series change in the above-ground volume obtained from the simulation results of the model approaches the above-ground volume of the individual crop derived from the aerial photographed image. The estimation device 100 estimates the yield obtained from the individual crop (step S15). The estimation device 100 estimates the harvest amount based on the simulation results obtained from the model in which the internal parameters determined in step S4 are set.

As explained above, the estimation device 100 of the second embodiment includes the aboveground volume calculation section 112, the parameter determination section 113, and the harvest amount estimation section 114. The above-ground part volume calculation unit 112 calculates the above-ground part volume of each crop individual in the field based on three-dimensional information in the field. A field is an example of a "target location." An aerial photographed image is an example of a "target image." The parameter determination unit 113 determines internal parameters of the crop growth model for each crop individual based on the above-ground volume of each crop individual in the field calculated by the above-ground volume calculation unit 112. The yield estimating unit 114 estimates the yield of each individual crop in the field using the crop growth model whose internal parameters have been determined by the parameter determining unit 113. As a result, the estimation device 100 of the second embodiment estimates the yield for each crop individual, for which the internal parameters of the model are individually set for each crop individual, based on the above-ground volume of the crop individual. Therefore, it is possible to estimate the yield for each individual crop.

Further, in the estimation device 100 of the second embodiment, the three-dimensional information is information indicating the three-dimensional shape of the field restored using an aerial image by the three-dimensional information restoration unit 111, or information using a distance sensor such as LiDAR. This is the information measured using Thereby, the estimating device 100 of the second embodiment produces effects similar to those described above.

Further, in the estimation device 100 of the second embodiment, the parameter determining unit 113 determines that the above-ground volume of the individual crop obtained from the simulation result using the crop growth model is based on the actual situation (calculated by the above-ground volume calculating unit 112). The internal parameters of the crop growth model are determined so as to approximate the above-ground volume of each individual crop in the field. As a result, the estimation device 100 of the second embodiment can determine internal parameters in accordance with the actual situation, and can create a highly accurate model that can perform simulations in accordance with the actual situation. Therefore, using a highly accurate model, it is possible to estimate the harvest amount with high accuracy.

Furthermore, in the estimation device 100 of the second embodiment, the parameter determination unit 113 determines that the time-series change in the above-ground volume of an individual crop obtained from the simulation results using the crop growth model is based on the actual situation (the above-ground volume calculation unit). The internal parameters of the crop growth model are determined in accordance with the calculated time-series changes in the above-ground volume of each crop individual in the field. As a result, the estimation device 100 of the second embodiment can determine internal parameters in accordance with the actual situation, and can create a highly accurate model that can perform simulations in accordance with the actual situation. Therefore, the same effects as those described above are achieved.

Furthermore, in the estimation device 100 of the second embodiment, the parameter determination unit 113 uses the time-series changes in the above-ground volume from germination to harvest time for each individual crop in the field calculated by the above-ground volume calculation unit. Then, determine the internal parameters of the crop growth model. As a result, the estimation device 100 of the second embodiment can determine the internal parameters to match the actual situation regarding the time-series change in the above-ground volume from germination to the harvest time, and execute a simulation that matches the actual situation. It is possible to create a highly accurate model. Therefore, the same effects as those described above are achieved.

Further, in the estimation device 100 of the second embodiment, the yield estimation unit 114 calculates the underground portion of the individual crop at the harvest time obtained from the simulation result using the crop growth model whose internal parameters are determined by the parameter determination unit 113. The amount of dry matter is estimated as the yield of the individual crop. Thereby, the estimation device 100 of the second embodiment can estimate the amount of underground dry matter of an individual crop at the harvest time as the harvest amount. Therefore, it is possible to estimate the harvest amount according to the harvest time.

Furthermore, in the estimation device 100 of the second embodiment, the above-ground volume calculation unit 112 extracts the vegetation portion where crops are vegetated based on the aerial photographed image, and calculates the individual crop portions included in the extracted vegetation portion. The above-ground volume of each individual crop in the field is calculated using the estimated number of individuals. As a result, in the estimation device 100 of the second embodiment, even if a plurality of crops appear connected in an image, each individual crop can be extracted and its area can be specified. Therefore, the above ground volume calculation unit 112 can accurately calculate the above ground volume. Therefore, internal parameters can be determined in accordance with more accurate actual conditions, and yields can be estimated with high accuracy using a highly accurate model.

Moreover, in the embodiment described above, the case where the yield of an individual crop is estimated using an aerial photographed image has been described as an example. However, it is not limited to aerial images. The estimation device 1 may estimate the yield of an individual crop using any digital image as long as it is a digital image captured to the extent that at least the position and size of the crop growing in the field can be seen.

Moreover, in the embodiment described above, the case where the yield of individual crops growing in a field is estimated is explained as an example. However, it is not limited to fields. The estimation device 1 may estimate the yield of an individual crop at any location as long as it is at least a location where crops grow.

Further, in the above-described embodiment, the case where the yield of an individual crop is estimated using the underground dry matter amount has been described as an example. However, it is not limited to underground dry mass. The yield of an individual crop may be estimated simply by using the dry matter amount without being limited to the aboveground dry matter amount or the underground dry matter amount. In this case, for example, the mass obtained by multiplying the dry mass of the individual crop at the harvest time by a predetermined ratio may be estimated as the harvest amount.

Note that a program for realizing all or part of the processing (calculations and control) performed by the functional units of the estimation device 100 is recorded on a computer-readable recording medium, and the program recorded on this recording medium is readable by the computer system. Each part may be processed by loading and executing the program. Note that the "computer system" herein includes hardware such as an OS and peripheral devices.
Furthermore, the term "computer system" includes the homepage providing environment (or display environment) if a WWW system is used.
Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. Furthermore, a "computer-readable recording medium" refers to a storage medium that dynamically stores a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It also includes devices that retain programs for a certain period of time, such as volatile memory inside a computer system that serves as a server or client. Further, the program may be one for realizing a part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

Although the embodiments of the present invention have been described above in detail with reference to the drawings, the specific configuration is not limited to these embodiments, and design changes may be made within the scope of the gist of the present invention. .

100 Estimation device 110 Image acquisition unit 111 Three-dimensional information restoration unit 112 Above ground volume calculation unit 113 Parameter determination unit 114 Harvest amount estimation unit 117 Storage unit

Claims (9)

an aboveground volume calculation unit that calculates the aboveground volume of each crop individual at the target location based on three-dimensional information at the target location;
a parameter determination unit that determines internal parameters of a crop growth model for each crop individual based on the above-ground volume of each crop individual at the target location calculated by the above-ground volume calculation unit;
a yield estimation unit that estimates the yield of each crop individual at the target location using the crop growth model whose internal parameters have been determined by the parameter determination unit;
An estimation device comprising: The three-dimensional information is information indicating a three-dimensional shape of the target location restored based on a target image in which the target location was captured, or information measured by a measuring device that measures distance.
The estimation device according to claim 1. The parameter determination unit is configured to make the above-ground volume of each crop individual obtained from a simulation result using a crop growth model approach the above-ground volume of each crop individual at the target location, which is calculated by the above-ground volume calculation unit. to determine the internal parameters of the crop growth model;
The estimation device according to claim 1 or claim 2. The parameter determination unit is configured to calculate a time-series change in the above-ground volume of each crop individual obtained from a simulation result using a crop growth model as a time-series change in the above-ground volume of each crop individual in the field calculated by the above-ground volume calculation unit. Determine the internal parameters of the crop growth model to follow the series changes,
The estimation device according to any one of claims 1 to 3. The parameter determination unit calculates the internal structure of the crop growth model using the time-series change in the above-ground volume from germination or budding to harvest time for each crop individual at the target location, which is calculated by the above-ground volume calculation unit. determine the parameters,
The estimation device according to any one of claims 1 to 4. The yield estimating unit estimates the dry mass of the individual crop at the harvest time obtained from the simulation result using the crop growth model whose internal parameters are determined by the parameter determining unit, as the yield of the individual crop.
The estimation device according to any one of claims 1 to 5. The above-ground area volume calculation unit extracts a vegetation area where crops are vegetated based on the target image in which the target location is captured, and estimates and estimates the number of crop individuals included in the extracted vegetation area. Calculating the above-ground volume of each crop individual at the target location using the number of individuals obtained,
The estimation device according to any one of claims 1 to 6. the above-ground part volume calculation unit calculates the above-ground part volume of each crop individual at the target location based on three-dimensional information at the target location;
a parameter determination unit determines internal parameters of a crop growth model for each crop individual based on the above-ground volume of each crop individual at the target location calculated by the above-ground volume calculation unit;
a yield estimation unit estimates the yield of each individual crop at the target location using the crop growth model whose internal parameters have been determined by the parameter determination unit;
Estimation method. to the computer,
Calculating the above-ground volume of each individual crop at the target location based on three-dimensional information at the target location;
determining internal parameters of a crop growth model for each crop individual based on the calculated above-ground volume of each crop individual at the target location;
using the crop growth model for which the internal parameters have been determined, estimating the yield of each individual crop at the target location;
program for.

Images