JP7216162B2 - Turf health condition determination system for golf course, lawn health condition determination method and program for golf course - Google Patents

Description

The present invention relates to a turf health condition determination system for a golf course, a turf health condition determination method and program for a golf course.

Patent Literature 1 describes a growth management system that calculates a growth index of a crop based on an image of the crop captured by a sensor camera.

JP 2014-183788 A

The technique described in Patent Document 1 can be used, for example, in determining the health condition of turf such as fairways and greens in golf courses.

However, in a golf course, it is necessary to pinpoint and quickly identify areas where unhealthy turf exists at high frequency, such as every day, in order to maintain unhealthy turf as soon as possible. The technology described in Patent Document 1 cannot do such a thing.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and one of its objects is to provide a turf health condition determination system for a golf course that can pinpoint and pinpoint at high speed an area where unhealthy turf exists on a golf course. An object of the present invention is to provide a lawn health condition determination method and program in a field.

A turf health condition determination system for a golf course according to the present invention comprises: golf course image acquisition means for acquiring a golf course image of the golf course; a lawn area extracting means for extracting an area; and a first judgment for judging the health condition of the lawn reflected in the first judgment object image based on the first judgment object image which is an image of at least a part of the lawn area. means for each of a plurality of second determination target images obtained by dividing the first determination target image, and including the first determination target image in which the grass in the image is determined to be unhealthy in the second determination target image; and a second determination means for determining the health condition of the grass reflected in the second determination target image based on variations in pixel values of the pixels.

In one aspect of the present invention, the first determination means determines, for each of the plurality of first determination target images obtained by dividing the image of the grass area, the first determination target image based on the first determination target image. Determine the health condition of the grass reflected in the image.

Further, in one aspect of the present invention, the first determination means, when the first determination target image having a predetermined number of pixels is input to the trained model, determines the first image based on the output from the trained model. The health condition of the lawn reflected in the determination target image is determined.

Further, in one aspect of the present invention, the second determination means determines whether the grass reflected in the second determination target image is based on a small standard deviation or variance of pixel values of pixels included in the second determination target image. determine the health status of

Further, in one aspect of the present invention, the second determination means determines the health condition of the grass reflected in the second determination target image based on an image obtained by applying a bandpass filter to the second determination target image.

Further, in one aspect of the present invention, the second determination means determines the health condition of the lawn reflected in the second determination target image based on an image obtained by performing smoothing processing on the second determination target image.

Further, a lawn health condition determination method for a golf course according to the present invention includes the step of acquiring a golf course image of a golf course, and determining a lawn area representing at least one of a fairway and a green in the golf course image. extracting; determining the health condition of the grass reflected in the first determination target image based on the first determination target image that is an image of at least a part of the grass region; For the first determination target image determined to be unhealthy, for each of a plurality of second determination target images obtained by dividing the first determination target image, variation in pixel values of pixels included in the second determination target image and determining the health condition of the grass reflected in the second determination target image based on the determination.

Further, a program according to the present invention includes a procedure for acquiring a photographed golf course image, a procedure for extracting a turf area indicating at least one of fairways and greens in the golf course image, Based on a first determination target image that is an image of at least a part of the area, a procedure for determining the health condition of the grass reflected in the first determination target image, the first With respect to the determination target image, for each of a plurality of second determination target images obtained by dividing the first determination target image, based on the variation in the pixel values of the pixels included in the second determination target image, A computer is caused to execute a procedure for judging the health condition of the photographed grass.

It is a figure showing an example of composition of a turf health condition judging system concerning one embodiment of the present invention. 1 is a functional block diagram showing an example of functions of a turf health condition determination system according to an embodiment of the present invention; FIG. It is a figure which shows an example of the RGB image expressed by gray scale. FIG. 10 is a diagram showing an example of a segmentation result image; It is a figure which shows an example of a binarized image. It is a figure which shows an example of a binarized image. FIG. 4 is a diagram showing an example of a grass area image; FIG. 4 is a diagram showing an example of a first determination target image; FIG. 11 shows an example of a heatmap of pixels noted by CNN; FIG. 4 is a diagram schematically showing an example of the relationship between a first determination target image and a second determination target image; It is a figure which shows an example of the turf health condition determination result image. FIG. 2 is a flow diagram showing an example of the flow of processing performed by the turf health condition determination system according to one embodiment of the present invention;

An embodiment of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a diagram showing an example of the configuration of a turf health condition determination system 10 according to this embodiment.
The lawn health condition determination system 10 according to this embodiment is, for example, a computer such as a personal computer or a server computer. As shown in FIG. 1 , the turf health condition determination system 10 includes, for example, a processor 12 , a storage unit 14 , a communication unit 16 , a display unit 18 and an operation unit 20 .

The processor 12 is, for example, a program-controlled device such as a CPU that operates according to a program installed in the turf health condition determination system 10 .

The storage unit 14 is, for example, a storage element such as ROM or RAM, a solid state drive, a hard disk drive, or the like. The storage unit 14 stores programs and the like executed by the processor 12 .

The communication unit 16 is, for example, a communication interface such as a network board.

The display unit 18 is a display device such as a liquid crystal display, and displays various images according to instructions from the processor 12 .

The operation unit 20 is a user interface such as a keyboard and a mouse, and receives user operation input and outputs a signal indicating the content of the input to the processor 12 .

The lawn health condition determination system 10 may include an optical disc drive for reading optical discs such as DVD-ROMs and Blu-ray (registered trademark) discs, a USB (Universal Serial Bus) port, and the like.

The lawn health condition determination system 10 according to the present embodiment is a combination of an image processing pipeline and a machine learning network. According to the turf health condition determination system 10 according to the present embodiment, it is possible to detect and identify potential problems in the health condition of turf on golf courses using high-resolution orthomosaic map images of golf courses. Become.

FIG. 2 is a functional block diagram showing an example of the functions of the turf health condition determination system 10 according to this embodiment. Note that the turf health condition determination system 10 according to the present embodiment does not need to implement all the functions shown in FIG. 2, and functions other than the functions shown in FIG. 2 may be installed.

As shown in FIG. 2, the turf health condition determination system 10 according to the present embodiment functionally includes, for example, a golf course image storage unit 30, a golf course image acquisition unit 32, a first processing unit 40, a second processing A unit 50 and a third processing unit 60 are included.

Then, as shown in FIG. 2, the first processing unit 40 includes a segmentation unit 42, a grass area extraction unit 44, and a gap filling noise removal unit 46. FIG. The second processing unit 50 also includes a first division unit 52 , a machine learning model 54 and a first determination unit 56 . The third processing section 60 also includes a second division section 62 , a second determination section 64 , and a display control section 66 .

The golf course image storage unit 30 is implemented mainly by the storage unit 14 . The golf course image acquiring unit 32, the grass area extracting unit 44, the gap filling noise removing unit 46, the first dividing unit 52, the first determining unit 56, the second dividing unit 62, and the second determining unit 64 are mainly implemented by the processor 12. be done. The segmentation unit 42 and machine learning model 54 are mainly implemented by the processor 12 and the storage unit 14 . The display control unit 66 is mainly implemented by the processor 12 and the display unit 18 .

The functions described above may be implemented by causing the turf health condition determination system 10, which is a computer, to execute a program including instructions corresponding to the above functions, which is installed in the turf health condition determination system 10. Also, this program may be supplied to the turf health condition determination system 10 via a computer-readable information storage medium such as an optical disk, magnetic disk, magnetic tape, or magneto-optical disk, or via the Internet. good.

In the present embodiment, the golf course image storage unit 30 stores, for example, a golf course image obtained by photographing the golf course. A golf course image according to the present embodiment is, for example, an image of a golf course captured by a camera mounted on an unmanned aerial vehicle (UAV) such as a drone.

The UAV according to this embodiment is equipped with, for example, an RGB camera, a near-infrared (NIR) camera, and a thermal camera. In this embodiment, an image is captured by each of the RGB camera, the near-infrared camera, and the thermal camera.

Image processing is then performed on these captured images to generate an RGB image, a near-infrared image, and a thermal image, which are orthomosaic map images covering the entire area of the golf course. be done. Then, the generated three types of orthomosaic map images are stored in the golf course image storage unit 30 as golf course images (RGB image, near-infrared image, and thermal image) according to this embodiment. In this embodiment, these orthomosaic map images are, for example, tif format images. FIG. 3 is a diagram showing an example of an RGB image according to this embodiment. Although the RGB image shown in FIG. 3 is expressed in grayscale, the actual RGB image is a full-color image. In addition, geographic information such as the latitude and longitude of the place where the image was taken is associated with the golf course image as a geographic reference.

In the present embodiment, the golf course image acquiring unit 32 acquires, for example, a golf course image obtained by photographing the golf course. For example, the golf course image acquisition unit 32 acquires golf course images stored in the golf course image storage unit 30 .

In this embodiment, for example, the segmentation unit 42 segments the golf course image into different types of regions. Here, for example, an area appearing in a golf course image (for example, an RGB image) is an area indicating sand such as a bunker, an area indicating a cart path, an area indicating a building or clutter, an area indicating rough, and an area indicating trees. , a fairway area, a green area, a tee box area, a water area such as a water hazard, and a background area.

The segmentation unit 42 according to this embodiment may include an existing trained Unet neural network. Then, the segmentation unit 42 may perform the above-described segmentation using a trained Unet neural network.

The trained Unet neural network outputs, for example, the segmentation result image illustrated in FIG. 4 in response to the input of the golf course image. Here, for example, part or all of the RGB image, the near-infrared image, or the thermal image acquired by the golf course image acquisition unit 32 is input to the learned Unet neural network.

In the present embodiment, for example, mutually different pixel values associated with each of the ten types of regions described above are determined in advance. In this embodiment, for example, the number of vertical and horizontal pixels is the same as that of the golf course image, and each pixel is associated with the type of area that is the classification result of the pixels in the golf course image associated with the pixel. A segmentation result image is generated that is a grayscale image with values set. Also, the segmentation result image according to the present embodiment is, for example, a png format image.

In this embodiment, for example, the turf area extraction unit 44 extracts a turf area indicating at least one of fairways and greens in the golf course image.

In this embodiment, for example, the turf area extracting unit 44 extracts pixels corresponding to pixels for which pixel values corresponding to the fairway or green are set in the segmentation result image from the above-described RGB image.

For example, the turf area extracting unit 44 masks the RGB image with a mask image, which is an image obtained by extracting pixels corresponding to fairways or greens from the segmentation result image shown in FIG. may be extracted. For example, pixels corresponding to the pixels shown in the mask image may be extracted from the RGB image.

For example, the turf area extraction unit 44 may specify the coordinate values of fairway or green pixels in the mask image. Then, the turf area extraction unit 44 may cross-reference the specified coordinate values with the coordinate values of the pixels included in the RGB image to cut out areas other than the fairway or green area from the RGB image. As a result, in the RGB image, the area other than the green or fairway area has a predetermined pixel value. Here, for example, in the RGB image, areas other than the green or fairway areas are assumed to be black pixels. Hereinafter, the RGB image from which the area other than the area representing the fairway or green is clipped in this manner will be referred to as an extracted RGB image.

In the present embodiment, the gap-filling noise removal unit 46 executes, for example, a process of filling gaps and a process of removing noise from the extracted RGB image.

Segmentation of high-resolution images with the Unet neural network described above may result in holes and gaps in the mask image described above for the entire golf course. Such holes and gaps are often classified as areas that indicate background or clutter, and such pixels are missing from the mask image. Pixels corresponding to holes and gaps in the mask image often contain important information in the final classification. Based on this fact, in the present embodiment, processing for filling holes and gaps is executed.

The gap-filling noise removal unit 46, for example, first binarizes the extracted RGB image. FIG. 5 is a diagram showing an example of an extracted RGB image binarized in this way. This image is hereinafter referred to as a binarized image. In the binarized image shown in FIG. 5, the fairway or green area is expressed in white, and the other area is expressed in black. Note that a binarized image may be generated by binarizing a mask image instead of the extracted RGB image.

Then, in this embodiment, the gap-filling noise removal unit 46 removes background clutter spots by, for example, performing several morphological transformations on the binarized image.

Specifically, for example, opening transformation is performed on the binarized image, and then closing transformation is performed to fill the gaps and remove clutter.

The opening transformation, for example, performs an erosion to reduce the edges of the white areas of the image, followed by a dilation to enlarge the white areas of the image. The opening transformation removes additional noise in the background.

The closing transformation is the opposite of the opening transformation. That is, the closing transformation performs dilation and then erosion. Closing transformations fill gaps in the foreground.

FIG. 6 is a diagram showing an example of a binarized image that has undergone the above processing. The gap-filling noise removal unit 46 generates a final extracted RGB image by extracting pixels corresponding to all white pixels in the binarized image subjected to the above processing from the RGB image. Hereinafter, the image of the turf area generated in this way will be referred to as a turf area image.

FIG. 7 is a diagram showing an example of a grass area image. As in FIG. 3, in FIG. 7, the turf area image is expressed in grayscale, but the actual turf area image is a full-color image.

In the present embodiment, for example, the first dividing unit 52 generates the first determination target image by dividing the grass area image described above. For example, the first determination target image is generated by dividing the grass area image into areas of the first predetermined number of pixels×the second predetermined number of pixels. Here, it is assumed that the first determination target image is generated by dividing the grass area image into areas of 1024 pixels×1024 pixels. FIG. 8 is a diagram showing an example of the first determination target image. The first determination target image according to this embodiment is, for example, a tile image of 1024 pixels×1024 pixels. The generation of the first determination target image can be easily performed by, for example, a window approach in which tiles are obtained while the window is slid by 1024 pixels.

The background of the grass area image is all black. Based on this, the first dividing unit 52 calculates the proportion of black pixels among the pixels included in the first determination target image. Here, the first determination target image in which the proportion of black pixels exceeds 50% is excluded from subsequent processing targets. By doing so, the processing time can be shortened. Also, most of the first determination target images excluded in this way do not show unhealthy grass.

The machine learning model 54 is, for example, a learned machine learning model (learned model) that outputs a heat map image according to the input of the first determination target image in this embodiment.

In this embodiment, for example, learning of the machine learning model 54 using a plurality of training data may be performed in advance. Training data may include, for example, learning input images and teacher data. The training input image may be an image of grass that is input to the machine learning model 54 during training of the machine learning model 54 . The teacher data included in the training data may be, for example, an image representing whether the lawn represented by the pixel is healthy or unhealthy for each pixel included in the learning input image included in the training data. . For example, an image that is training data may be, for example, an image that is black for pixels representing healthy turf and white for pixels representing unhealthy turf.

Then, for example, an output when a learning input image included in training data is input to the machine learning model 54 may be specified. Then, an error (comparison result) between the output and teacher data included in the training data may be specified. Here, for example, the softmax entropy error may be determined. Based on the identified error, the values of the parameters of the machine learning model 54 may then be updated, for example, by error backpropagation.

Then, learning of the machine learning model 54 may be performed by performing the above processing on a plurality of pieces of training data.

Due to the relatively small amount of training data available (approximately 5000), the inventors initially extracted features using a pre-trained InceptionResnetV2 model and used a small sequential convolutional neural network (CNN). performed the learning of The CNN was then added to the InceptionResnetV2 model.

Next, training was performed on the 150 epoch dataset preprocessed by the stochastic gradient descent algorithm at slow learning speed for only the top four layers of the base model.

Also, the inventors used transfer learning rather than running the model from scratch. We also performed Imagenet's weight-based learning to exploit universal low-level features such as general shape identification.

The inventor also performed training with a similar training procedure for the Densenet, VGG19, and VGG16 models. Then, in the final stage of training, an ensemble learning of uncorrelated CNNs voted by class labels that take either healthy or unhealthy values was performed. By doing so, the bias is reduced and the classification accuracy is improved.

We also used false positives and false negatives as metrics to monitor and tune model performance during training. In order to find as many first determination target images that show truly unhealthy turf as possible and not to lose confidence in the user of the turf health condition determination system 10, the first determination target image that is erroneously classified as unhealthy is used. 1 This was done to reduce the amount of images to be judged.

In the present embodiment, for example, the first determination unit 56 determines the health condition of the turf shown in the first determination target image based on the first determination target image, which is an image of at least a part of the turf region. Here, for example, it is determined whether the grass reflected in the first determination target image is healthy or unhealthy. Here, the first determination target image may be an image obtained by dividing the grass area image as described above. Then, for each of the plurality of first determination target images obtained by dividing the turf region image, the first determination unit 56 determines the health condition of the grass reflected in the first determination target image based on the first determination target image. You may

In this embodiment, for example, the first determination unit 56 inputs the first determination target image to the learned machine learning model 54 .

Here, the first determination unit 56 may perform preprocessing using Gaussian blur on the first determination target image that is not excluded in order to suppress measurement noise and abnormal spike noise in pixel values. The first determination unit 56 may also perform morphological operations (erosion and dilation) on the first determination target image. The morphological operations are performed to further clean the first image of interest by closing boundary discontinuities in shape and filling holes in convex objects in the first image of interest. Then, the first determination unit 56 may input the first determination target image on which such processing has been performed to the learned machine learning model 54 .

Then, as described above, the trained machine learning model 54 outputs a heat map image according to the input of the first determination target image. FIG. 9 is a diagram showing an example of a heat map image output from the learned machine learning model 54 in response to the input of the first determination target image. The pixel value of each pixel included in the heat map image shown in FIG. 9 corresponds to the probability that the lawn represented by the pixel is unhealthy.

Then, based on the heat map image, the first determination unit 56 may specify the probability that the grass reflected in the first determination target image input to the learned machine learning model 54 is unhealthy. Then, if the specified probability is higher than a predetermined threshold (for example, 85%), the first determination unit 56 may determine that the grass reflected in the first determination target image is unhealthy.

As described above, when the first determination target image having a predetermined number of pixels is input to the trained model, the first determination unit 56 determines the first determination target image based on the output from the trained model. You may judge the health condition of the grass reflected in .

However, the determination here does not accurately determine the area in which unhealthy grass appears in the first determination target image of 1024 pixels×1024 pixels, and the possibility of unhealthy grass in the first determination target image is not determined accurately. It just shows that the area where some grass is captured is included.

In the present embodiment, for example, the second dividing unit 62 divides each of the first determination target images in which it is determined that the grass in the image is unhealthy by dividing the first determination target image by the third predetermined number of pixels×the fourth pixel number. A second determination target image is generated by dividing the image into regions each having a predetermined number of pixels. Here, for example, as shown in FIG. 10, the second determination target image I2 is generated by dividing the first determination target image I1 into regions of 256 pixels×256 pixels. In this example, 16 second determination target images I2 are generated from one first determination target image I1.

Here, the second determination target image in which all pixels are black pixels is excluded from subsequent processing. This helps pinpoint exactly where unhealthy turf exists. Also, irrelevant areas within the golf course where there are as many areas with unhealthy turf as there are may not be shown.

In the present embodiment, for example, the second determination unit 64 divides the first determination target image into a plurality of second determination target images for each of the first determination target images in which the grass in the image is determined to be unhealthy. , the health condition of the lawn reflected in the second determination target image is determined based on variations in pixel values of pixels included in the second determination target image.

In the present embodiment, the second determination unit 64 performs an additional determination process of searching for an area with few outliers and uniform pixel values in order to exclude false positives. Here, an area with little or no variation in color distribution, such that it is not clear that the grass reflected in the area is unhealthy or sick, is searched.

Here, the second determination unit 64 may first apply a bandpass filter to the second determination target image. The band-pass filter is, for example, a filter that removes colors corresponding to areas indicating sand such as bunkers, areas indicating passages, black areas, or areas that are not grass.

Then, the second determination unit 64 may perform smoothing processing on the second determination target image in order to suppress noise. For example, the second determination unit 64 may additionally perform smoothing processing using coarse Gaussian blur on the second determination target image. By doing so, it is possible to determine the health condition of the grass more accurately.

Then, the second determination unit 64 determines the health condition of the grass reflected in the second determination target image based on the standard deviation or small variance of the pixel values of the pixels included in the second determination target image. good. Here, for example, it is determined whether the grass reflected in the second determination target image is healthy or unhealthy. For example, the second determination unit 64 identifies a value (for example, standard deviation or variance) that indicates variation in pixel values of pixels included in the second determination target image. Then, if the difference between the value indicating the pixel value variation and the predetermined value (assumed to be a small value) is equal to or less than the threshold, the second determination unit 64 determines that the grass reflected in the second determination target image is healthy. It is determined that there is (that is, it is a false positive).

Alternatively, if the value (for example, standard deviation or variance) indicating variation in pixel values of pixels included in the second determination target image is less than a predetermined threshold, the second determination unit 64 determines that the second determination target image may be determined to be healthy (that is, false positive).

The area corresponding to the second determination target image determined as false positive in this manner is excluded from the area determined as unhealthy turf.

Here, the second determination unit 64 may determine the health condition of the lawn reflected in the second determination target image based on the image obtained by applying the band-pass filter to the second determination target image as described above. .

Further, the second determination unit 64 may determine the health condition of the grass reflected in the second determination target image based on the image obtained by performing the smoothing process on the second determination target image as described above.

The inventor ranked a plurality of sample second determination target images based on the number of outliers present in the histogram of pixel values. Here, it was found that in the second determination target image in which the number of outliers is in the top 25%, the pixel value variation can be analyzed based on the standard deviation of the pixel value histogram. From this, the inventor discovered that the standard deviation of the pixel values of the second determination target image, in which uniform and healthy grass is captured, tends to be tight or small.

Based on this, in the present embodiment, as described above, it is determined whether or not the second determination target image is false-positive based on the standard deviation or variance of the pixel values.

In this embodiment, for example, the display control unit 66 generates a turf health condition determination result image illustrated in FIG. 18.

For example, the display control unit 66 displays a healthy turf area, which is a geographical area corresponding to fairways or greens where turf is healthy, and a geographical area corresponding to fairways or greens, where turf is unhealthy. You may identify an unhealthy turf area where . For example, the display control unit 66 may specify, as a healthy turf area, a geographical area associated with the first determination target image in which the first determination unit 56 has determined that the turf in the image is healthy. In addition, the display control unit 66 divides the first determination target image in which the grass in the first determination unit 56 has been determined to be unhealthy, and which is the second determination target image that has been captured in the second determination unit 64 . A geographic region associated with the second determination target image in which the turf on the ground is determined to be healthy may also be identified as the healthy turf region.

Then, the display control unit 66 divides the first determination target image in which the grass in the image is determined to be unhealthy by the first determination unit 56, and divides the second determination target image. A geographical area associated with the second determination target image in which the turf shown is determined to be unhealthy may be specified as the unhealthy turf area.

Then, the display control unit 66 may generate a turf health condition determination result image in which the healthy turf area and the unhealthy turf area are expressed so as to be identifiable. For example, a turf health condition determination result image may be generated in which healthy turf areas and unhealthy turf areas are expressed in different colors.

In the present embodiment, for example, after the determination by the second determination unit 64 ends, the determination result is converted into a georeference based on the geographic information associated with the second determination target image.

Here, for example, the georeference of the second determination target image is converted into polygons. They are then grouped using the cascaded union algorithm in the open source python library. This grouping groups all adjacent polygons to produce a series of newly grouped polygons.

The coordinates of the polygon representing the unhealthy turf area are used to generate an API request to generate a turf health alert marker. Turf health alert markers are saved in geojson format files and visualized as translucent polygons on top of a given orthomosaic map. An example of an image visualized in this way is the turf health condition determination result image illustrated in FIG.

Here, an example of the flow of processing performed by the turf health condition determination system 10 according to the present embodiment will be described with reference to the flowchart illustrated in FIG. 12 .

In this processing example, three golf course images (RGB image, near-infrared image, and thermal image) of a given golf course are pre-stored in the golf course image storage unit 30 .

First, the golf course image acquisition unit 32 acquires three golf course images stored in the golf course image storage unit 30 (S101).

Then, the segmentation unit 42 generates a segmentation result image based on the golf course image acquired in the process shown in S101 (S102).

Then, the turf area extraction unit 44 generates an extracted RGB image by extracting a turf area representing at least one of the fairway and the green in the RGB image acquired in the processing shown in S101 (S103). .

Then, the gap-filling noise removal unit 46 generates a binarized image based on the extracted RGB image generated by the processing shown in S103 (S104).

Then, the gap-filling noise removal unit 46 performs the above-described morphological transformation on the binarized image generated by the processing shown in S104 (S105).

Then, the gap-filling noise removal unit 46 generates a grass area image based on the binarized image subjected to the morphological transformation in the process shown in S105 and the RGB image acquired in the process shown in S101 ( S106).

Then, the first dividing unit 52 divides the grass area image generated by the processing shown in S106 to generate a plurality of first determination target images (S107).

Then, the first dividing unit 52 selects a first determination target image to be subjected to subsequent processing from among the plurality of first determination target images generated in the processing shown in S107 (S108).

Then, the first determination unit 56 performs predetermined processing such as preprocessing by Gaussian blur and morphology operation on each of the first determination target images selected in the processing shown in S108 (S109).

Then, the first determination unit 56 determines, for each of the first determination target images processed in the process shown in S109, whether the grass reflected in the first determination target image is healthy or unhealthy (S110). .

Then, the second dividing unit 62 specifies the first determination target image in which the grass in the image is determined to be unhealthy in the processing shown in S110 (S111).

Then, the second dividing unit 62 divides each of the first determination target images specified in the process shown in S111, thereby dividing the first determination target image into a plurality of images associated with the first determination target image. A second determination target image is generated (S112).

Then, the second dividing unit 62 selects a second determination target image to be subjected to subsequent processing from among the plurality of second determination target images generated in the processing shown in S112 (S113).

Then, the second determination unit 64 applies a given bandpass filter to each of the second determination target images selected in the process shown in S113 (S114).

Then, the second determination unit 64 performs smoothing processing on each of the second determination target images to which the bandpass filter has been applied in the processing shown in S114 (S115).

Then, the second determination unit 64 determines the second determination target image based on the standard deviation or small variance of the pixel values of the pixels included in the second determination target image that has been smoothed in the process shown in S115. It is determined whether the grass in the image is healthy or unhealthy (S116).

Then, the display control unit 66 generates a turf health condition determination result image based on the results of the processing shown in S110 and the processing shown in S116 (S117).

Then, the display control unit 66 causes the display unit 18 to display the turf health condition determination result image generated in the processing shown in S117 (S118), and the processing shown in this processing example ends.

In order to maintain unhealthy turf as early as possible, golf courses need to pinpoint and quickly identify areas where unhealthy turf exists, as often as every day.

When the machine learning model 54 is used to determine the health of the turf for the entire area of the golf course, the more the number of images input to the machine learning model 54, the more generally the health of the turf. It takes time to determine the status.

Therefore, it is conceivable to increase the number of pixels included in the first determination target image (increase the data size) so that the geographical area corresponding to the first determination target image input to the machine learning model 54 becomes wider. . By doing so, it is possible to reduce the number of first determination target images that need to be input to the machine learning model 54 in order to perform the determination for the entire area of the golf course. It is possible to shorten the time.

However, if this is done, the granularity of the geographic region for which the health condition of the lawn is determined by the determination becomes coarser, and determination cannot be made in a pinpoint region.

Therefore, in the present embodiment, determination based on the first determination target image is once performed, and as a result, the first determination target image in which it is determined that the grass in the image is unhealthy is divided, and the corresponding geographical area is divided into the first determination target image. A second determination target image narrower than the first determination target image is generated.

Then, determination based on variation in pixel values, which generally requires less processing load than determination using the machine learning model 54, is performed on the second determination target image.

In this manner, according to the present embodiment, by performing multiple determinations, it is possible to pinpoint, at high speed, an area in which unhealthy turf exists on a golf course.

It should be noted that the present invention is not limited to the above-described embodiments.

For example, the second determination unit 64 does not have to execute the processing shown in S114 described above. Also, the second determination unit 64 does not have to execute the process shown in S115 described above.

Alternatively, the first determination unit 56 may determine the health condition of the lawn shown in the first determination target image using another existing technique without using the machine learning model 54 .

Also, it is not necessary to determine whether the turf is healthy or unhealthy for both the area representing the fairway and the area representing the green. It may be determined whether the turf is healthy or unhealthy for either the area representing the fairway or the area representing the green.

Also, the second determination unit 64 determines whether the grass reflected in the second determination target image is healthy or unhealthy, based on the standard deviation or variance of the pixel values of the pixels included in the second determination target image. do not have to. For example, out of the pixels included in the second determination target image, pixels whose pixel values are outliers may be specified according to a predetermined standard. Then, based on the number of pixels specified in this way, it may be determined whether the grass reflected in the second determination target image is healthy or unhealthy.

10 turf health condition determination system 12 processor 14 storage unit 16 communication unit 18 display unit 20 operation unit 30 golf course image storage unit 32 golf course image acquisition unit 40 first processing unit 42 segmentation unit 44 lawn area extraction unit 46 gap filling noise removal unit 50 second processing unit 52 first division unit 54 machine learning model 56 first determination unit 60 third processing unit 62 second division unit 64 second 2 determination unit, 66 display control unit.

Claims (8)

a golf course image obtaining means for obtaining a golf course image obtained by photographing the golf course;
turf area extracting means for extracting a turf area representing at least one of a fairway and a green in the golf course image;
a first determination means for determining the health condition of the grass reflected in the first determination target image based on the first determination target image, which is an image of at least a part of the turf region;
For each of a plurality of second determination target images obtained by dividing the first determination target image, the number of pixels included in the second determination target image is determined for each of the first determination target images in which the grass in the image is determined to be unhealthy. a second determination means for determining the health condition of the grass reflected in the second determination target image based on variations in pixel values;
A turf health condition determination system in a golf course, comprising: For each of the plurality of first determination target images obtained by dividing the image of the turf region, the first determination means determines the health condition of the grass reflected in the first determination target image based on the first determination target image. judge,
The turf health condition determination system for a golf course according to claim 1, characterized in that: The first determination means determines the health condition of the lawn reflected in the first determination target image based on an output from the learned model when the first determination target image having a predetermined number of pixels is input to the learned model. determine the
The turf health condition determination system for a golf course according to claim 1 or 2, characterized in that: The second determination means determines the health condition of the grass reflected in the second determination target image based on the standard deviation or small variance of the pixel values of the pixels included in the second determination target image.
The turf health condition determination system for a golf course according to any one of claims 1 to 3, characterized in that: The second determination means determines the health condition of the grass reflected in the second determination target image based on an image obtained by applying a band-pass filter to the second determination target image.
The turf health condition determination system for a golf course according to any one of claims 1 to 4, characterized in that: The second determination means determines the health condition of the grass reflected in the second determination target image based on an image obtained by performing smoothing processing on the second determination target image.
The turf health condition determination system for a golf course according to any one of claims 1 to 5, characterized in that: a golf course image acquiring means acquiring a golf course image of the golf course;
a turf area extraction means extracting a turf area indicating at least one of a fairway and a green in the golf course image;
a step in which a first determination means determines the health condition of the turf reflected in the first determination target image based on the first determination target image, which is an image of at least a part of the turf region;
A second determination means, for each of a plurality of second determination target images obtained by dividing the first determination target image, for each of the first determination target images in which the grass in the image is determined to be unhealthy. determining the health condition of the grass reflected in the second determination target image based on variations in pixel values of pixels included in the image;
A lawn health condition determination method on a golf course, comprising: a procedure for acquiring a golf course image of a golf course;
A procedure for extracting a turf area representing at least one of a fairway and a green in the golf course image;
a procedure for determining the health condition of the turf reflected in the first determination target image based on the first determination target image, which is an image of at least a part of the turf region;
For each of a plurality of second determination target images obtained by dividing the first determination target image, the number of pixels included in the second determination target image is determined for each of the first determination target images in which the grass in the image is determined to be unhealthy. a procedure for determining the health condition of the grass reflected in the second determination target image based on variations in pixel values;
A program characterized by causing a computer to execute

Images