JP7828072B2 - Strawberry plant information acquisition system - Google Patents

Description

The present invention relates to a strawberry plant information acquisition system.

Image measurement is known as an effective method for biometric measurements of plants, as it is non-destructive, allows for wide-area measurements, and is labor-saving. When the plant being biometrically measured is a strawberry, measurement targets include petiole length and leaf area. These measurement targets are important factors in growth diagnosis and yield prediction, so it is desirable to be able to measure them accurately.

Note that a conventional technology is known in which a photographing unit captures an image of a plant while a fan is blowing air onto the plant, and the image is then processed to determine the plant's growth status (stem thickness) (see, for example, Patent Document 1). Also known is a technology for monitoring the plant's growth status (such as harvest time) based on the chronological progression of images (see, for example, Patent Document 2).

JP 2016-29940 A Japanese Patent Application Laid-Open No. 2019-37225

However, the above-mentioned petiole length and leaf area cannot be measured even using the above-mentioned Patent Documents 1 and 2. For example, since the petiole is located below the leaf blade (leaflet) and is difficult to see from the outside, it is difficult to measure the petiole length using an image. Furthermore, since leaf blades often overlap each other, it is also difficult to measure the leaf area with high accuracy.

Therefore, the present invention aims to provide a strawberry plant information acquisition system that can accurately calculate strawberry plant information.

The strawberry plant information acquisition system of the present invention comprises a blowing device that blows wind onto strawberry plants from above, a photographing device that takes images of the plants from above, a control device that controls the photographing device to take a first image of the plants from above in a first state in which no wind is blown onto the plants from the blowing device or in which wind is blown onto the plants from the blowing device at a first speed, and a control device that controls the photographing device to take a second image of the plants from above in a second state in which wind is blown onto the plants from the blowing device at a second speed, and a control device that determines the petiole length of a predetermined leaf of the strawberry plant from the images taken by the photographing device. and a calculation device that calculates a value, where the second speed is a speed at which the movement amount of the specified leaf becomes equal to or less than a predetermined value when the speed of the wind blown from the blowing device is increased, and the calculation device determines the difference between a first distance from the center of the plant to the petiole tip of the specified leaf in the first image and a second distance from the center of the plant to the petiole tip of the specified leaf in the second image, and calculates the petiole length value of the specified leaf based on the petiole angle value of the specified leaf in each of the first and second states and the difference between the first and second distances.

The strawberry plant information acquisition system of the present invention has the effect of being able to accurately calculate strawberry plant information.

1 is a diagram schematically illustrating a configuration of an information acquisition system according to a first embodiment. FIG. 2 is a diagram illustrating a hardware configuration of a control device and an information processing device. FIG. 2 is a functional block diagram of a control device and an information processing device according to the first embodiment. FIG. 1 is a diagram showing an experimental setup. Figure 5(a) is an image of a strawberry plant taken by a camera in the experimental apparatus of Figure 4 when the blower was not operating and the wind speed was 0 m, and Figure 5(b) is an image of a strawberry plant taken by a camera when wind was blown from the blower at a predetermined wind speed. FIG. 6(a) is a diagram schematically showing the state of the petiole and leaf blade in the first and second states, and FIG. 6(b) is a diagram for explaining the position of the tip of the petiole. Figures 7(a) to 7(e) are graphs showing the results of actual measurements of the horizontal projection movement _Mp of the petiole of each of the first to fifth leaves of a strawberry plant against the wind speed v, obtained using the experimental apparatus of Figure 4. 8(a) to 8(e) are diagrams showing examples of approximations of the actual measurement values of FIGS. 7(a) to 7(e). FIG. 1 is a schematic diagram (part 1) for explaining an approximation formula. FIG. 10 is a second schematic diagram for explaining an approximation formula. 11(a) to 11(c) are diagrams for explaining a method for detecting the center of a stock. 4 is a flowchart showing processing by the control device according to the first embodiment. 4 is a flowchart showing processing performed by the information processing apparatus according to the first embodiment. 14A to 14C are diagrams for explaining the processing of the pre-processing unit. 15(a) to 15(e) are diagrams for explaining an example of determining the petiole length L and the leaf area A using approximate formulas. FIG. 10 is a functional block diagram of a control device and an information processing device according to a second embodiment. 10 is a flowchart showing a process of a control device according to a second embodiment. 10 is a flowchart illustrating a process performed by an information processing apparatus according to a second embodiment.

First Embodiment
The strawberry plant information acquisition system according to the first embodiment will be described in detail below.

Figure 1 shows a schematic configuration of an information acquisition system 100 according to a first embodiment. In this embodiment, the information acquisition system 100 is a system that calculates and records the petiole length and leaf area of each strawberry plant grown in an elevated position in a cultivation bed 12 as shown in Figure 1. The distance (height) from the ground to the top surface of the cultivation bed 12 is H1 (H1 is, for example, 1000 mm). Note that the strawberry plants do not have to be grown in the cultivation bed 12; they may be planted in the ground in a ridge, for example.

The information acquisition system 100 includes a monitoring device 10 and an information processing device 90 as a calculation device. The monitoring device 10 moves near the cultivation bed 12 and acquires images of strawberry plants taken from above. The information processing device 90 is communicatively connected to the monitoring device 10 and calculates the petiole length and leaf area of each strawberry plant planted in the cultivation bed 12 based on the images and position information acquired from the monitoring device 10. The information processing device 90 and the monitoring device 10 may be connected via a wired LAN (Local Area Network) or the like, or wirelessly via Wi-Fi or the like. The information processing device 90 may also be mounted on the monitoring device 10. In this embodiment, the extension direction of the cultivation bed 12 is defined as the X-axis direction, the direction perpendicular to the X-axis direction in a horizontal plane is defined as the Y-axis direction, and the vertical direction is defined as the Z-axis direction.

As shown in Figure 1, the monitoring device 10 is installed on rails 14 laid along the X-axis direction below the cultivation bed 12 (on the ground). The monitoring device 10 is movable in the X-axis direction along the rails 14.

The monitoring device 10 comprises a housing 20, wheels 22, a camera 24 as a photographing device, a blower 26 as a spraying device, a motor 28, a position detection device 29, and a control device 30. The housing 20 has a generally rectangular frame shape. The ceiling panel 20a of the housing 20 extends above the cultivation beds 12 and strawberry plants, with the underside (-Z surface) of the ceiling panel 20a facing the strawberry plants from above.

The wheels 22 are driven to rotate on the rails 14 by a motor 28. The rotation of the motor 28 is controlled by a control device 30. That is, in this embodiment, the movement mechanism for moving the monitoring device 10 includes the wheels 22 that run on the rails 14 and the motor 28. However, this is not limited to this, and the movement mechanism may also include a running unit such as a wheel or crawler that runs on the ground, and a drive unit (such as a motor) that drives the running unit.

The camera 24 and blower 26 are mounted on the underside of the ceiling panel 20a of the housing 20. The distance (height) of the camera 24 from the cultivation bed 12 is H2 (H2 is, for example, 1000 mm). The camera 24 photographs the strawberry plants from above. The blower 26 has the function of blowing air downward onto the strawberry plants. The speed (wind speed) of the air blown from the blower 26 is adjustable and controlled by the control device 30. The control device 30 photographs the plants from above using the camera 24 with or without air being blown onto the plants by the blower 26. The images captured by the camera 24 are transmitted to the control device 30, which then transmits the captured images to the information processing device 90. The control device 30 also transmits information on the speed (wind speed) of the air blown onto the plants from the blower 26 during photography to the information processing device 90.

The position detection device 29 may be, for example, a position detection device having an RFID reader capable of communicating with multiple RFID (radio frequency identifier) tags installed near the cultivation bed 12, a position detection device having a camera capable of photographing markers installed near the cultivation bed 12, a position detection device that detects position from the amount of rotation of the wheels 22 or motor 28, or an RTK-GNSS (Real Time Kinematic-Global Navigation Satellite System). The position detection device 29 detects the XY position of the camera 24. From the XY position of the camera 24, the XY position of each point within the captured image (capture range) can be determined. The detection results of the position detection device 29 are transmitted to the control device 30.

The control device 30 controls the motor 28 based on the detection results of the position detection device 29 to adjust the position of the monitoring device 10 (the shooting location of the camera 24). The control device 30 also transmits to the information processing device 90 the image captured by the camera 24, its position information (XY position), and information on the speed of the wind blown by the blower 26 when the image was captured. Figure 2 shows the hardware configuration of the control device 30. As shown in Figure 2, the control device 30 includes a CPU (Central Processing Unit) 190, a ROM (Read Only Memory) 192, a RAM (Random Access Memory) 194, a storage unit (such as an HDD (Hard Disk Drive) or SSD (Solid State Drive)) 196, a network interface 197, and a portable storage medium drive 199. These components of the control device 30 are connected to a bus 198. In the control device 30, the functions of the units shown in FIG. 3 are realized by the CPU 190 executing a program stored in the ROM 192 or storage unit 196, or a program read from the portable storage medium 191 by the portable storage medium drive 199. Note that the functions of the units shown in FIG. 3 may also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

The information processing device 90 uses the location information transmitted from the control device 30 to identify the plant photographed in the image. The information processing device 90 also uses the image and wind speed information to calculate the petiole length and leaf area of each plant. The information processing device 90 has the same hardware configuration as the control device 30 (see Figure 2). In the information processing device 90, the CPU 190 executes a program to realize the functions of each unit shown in Figure 3.

(Calculation formula for petiole length and leaf area)
Here, we will explain the mathematical formulas used by the information processing device 90 (calculation unit 56 in FIG. 3, which will be described later) when calculating the petiole length and leaf area. Note that in this embodiment, the formulas (approximation formulas) described later are used, but this is not limiting and other formulas can also be used.

Figure 4 shows a schematic diagram of the experimental apparatus 200 used to derive the formula. As shown in Figure 4, the experimental apparatus 200 is equipped with a cultivation pot 202 containing planted strawberry plants. The cultivation pot 202 is installed so that its upper end is at a height H0 (e.g., 500 mm) above the ground. The experimental apparatus 200 also includes a hollow cylinder 204 with a height H2 (e.g., 1000 mm) that covers the strawberry plants. The diameter of this hollow cylinder 204 is, for example, 500 mm. In the experimental apparatus shown in Figure 4, the height H0 of the cultivation pot 202 is set equal to the diameter of the hollow cylinder 204 so that air passes through the hollow cylinder 204 (to achieve rectification). Furthermore, the experimental apparatus 200 is equipped with a camera 24 and a blower 26 at the upper end of the hollow cylinder 204. The camera 24 and blower 26 in Figure 4 are the same as the camera 24 and blower 26 in Figure 1.

Figure 5(a) is an image of a strawberry plant taken by camera 24 when blower 26 is not operating and the wind speed is 0 m. Figure 5(b) is an image of a strawberry plant taken by camera 24 when wind is blown from blower 26 at a predetermined wind speed. As shown in Figure 5(b), when wind is blown from above, the leaf blades are subjected to a force (wind load), and as can be seen by comparing it with Figure 5(a), the leaf stalks fall outward around the base (center of the plant (crown)). The leaf blades also fall outward, and the entire leaf moves outward.

In Fig. 6(a), the state of the entire leaf in a state where a wind speed of 0 m or a weak wind (wind at a first speed described later) is blowing against the leaf blade (first state) as shown in Fig. 5(a) is shown by a thick solid line. Also, in Fig. 6(a), the state of the entire leaf in a state where a moderate or strong wind (wind at a second speed described later) is blowing against the leaf blade (second state) as shown in Fig. 5(b) is shown by a thick dashed line. In Fig. 6(a), the horizontal difference M _P between the position of the petiole tip in the first state and the position of the petiole tip in the second state is referred to as the horizontal projection movement amount M _p of the petiole. Note that the position of the petiole tip shown in Fig. 6(a) is actually the base (see symbol S) of the petiole (the thin petiole connecting the petiole to the leaf blade) shown in Fig. 6(b).

In addition, Figures 7(a) to 7(e) show the results of actual measurements of the horizontal projection movement _Mp of the petiole of each of the first to fifth leaves of a strawberry plant in response to wind speed v, obtained using the experimental device 200.

(About the approximate formula)
This approximation formula is an example of approximating the relationship between wind speed v and horizontally projected displacement M _P as shown by the solid lines in FIGS. 8(a) to 8(e) based on the actual measurement values in FIGS. 7(a) to 7(e).

9 and 10 are schematic diagrams for explaining the approximation formula. In FIGS. 9 and 10, the state of the entire leaf when exposed to a wind of 0 m/s or a first speed (weak wind) is shown by a thick solid line, and the state of the entire leaf when exposed to a second speed (moderate or strong wind) is shown by a thick dashed line. The second speed (second wind speed) is the speed at which the leaf movement amount becomes less than a predetermined amount (i.e., the speed at which the leaf hardly moves) when the wind speed blowing against the leaf blade is increased. In FIG. 9, the petiole angle (the angle between the petiole extension direction and the horizontal plane) in the first state is θ (rad), and the petiole angle in the second state is δ (rad). θ can be said to be the value of the petiole angle when the leaf is attached. The actual movement amount of the petiole tip between each state is M (mm), and the horizontally projected movement amount is M _P (mm).

Based on the knowledge that the wind load on the leaf blade is proportional to the square of the wind speed (v) and the pressure-receiving area (A), we decided to use the following formula (1), where l is the proportionality coefficient.

In addition, this approximate formula expresses how the angle δ returns from the state shown by the dashed line due to bending (flexion) of the petiole, as shown in Figure 10.

Specifically, we decided to use the following formula (2):

Here, p is an advance angle coefficient, and (v ⁴ /p) represents the return of the angle due to bending of the petiole.

Furthermore, in this approximation formula, the formula for converting the length of petiole inclination into a projection value is defined as follows (3):

In this embodiment, the petiole length L and leaf area A are calculated using the above approximate formulas (formulas (1) to (3)).

(Functions of the control device 30 and the information processing device 90)
Returning to FIG. 3, the functions of the control device 30 and the information processing device 90 will be described.

(Functions of the control device 30)
As shown in FIG. 3, the control device 30 has the functions of a position control unit 40, a wind speed control unit 41, an imaging control unit 42, and an image acquisition and transmission unit 44.

The position control unit 40 controls the motor 28 based on the detection results of the position detection device 29 to adjust the position of the monitoring device 10 (the shooting location of the camera 24). For example, the position control unit 40 moves the monitoring device 10 so that each plant is successively within the shooting range of the camera 24.

When a single plant is within the shooting range of the camera 24, the wind speed control unit 41 controls the blower 26 to blow air at a predetermined speed onto the plant.

The photography control unit 42 uses the camera 24 to photograph the plants from above. The photography control unit 42 photographs plants that are not being exposed to the wind from the blower 26 and plants that are being exposed to the wind from the blower 26. When the photography control unit 42 has completed photographing a plant, it notifies the position control unit 40 of this fact. Upon receiving this notification, the position control unit 40 adjusts the position of the monitoring device 10 so that the next plant falls within the photography range of the camera 24.

The image acquisition/transmission unit 44 acquires the image captured by the camera 24 and transmits the acquired image to the information processing device 90 together with the position information (XY position) at the time of image capture obtained from the position control unit 40 and the wind speed information at the time of image capture obtained from the wind speed control unit 41.

(Functions of the information processing device 90)
As shown in FIG. 3 , the information processing device 90 includes an image acquisition unit 52 , an image analysis unit 54 , a calculation unit 56 , a data management unit 58 , an output unit 60 , and a pre-processing unit 62 .

The image acquisition unit 52 acquires images of the stocks taken by the camera 24, information about the location where the images were taken, and information about the wind speed at the time the images were taken.

The image analysis unit 54 analyzes the images acquired by the image acquisition unit 52 and passes the analysis results to the calculation unit 56.

The calculation unit 56 calculates the petiole length and leaf area of the strawberry based on the analysis results of the image analysis unit 54, predetermined coefficients, and information entered by the user.

The data management unit 58 acquires the calculation results (petiole length and leaf area) from the calculation unit 56 and manages them in association with the plant's location information.

The output unit 60 outputs information managed by the data management unit 58.

The pre-processing unit 62 uses the images acquired by the image acquisition unit 52 and information on the wind speed at the time of image capture to execute a process for determining the wind speed (second wind speed) to be set when calculating the petiole length and leaf area of strawberries. The method for determining the second wind speed is explained below. As mentioned above, the second wind speed is the speed at which the amount of leaf movement becomes less than a predetermined amount when the speed of the wind blowing against the leaf blade is increased (i.e., the speed at which the leaf hardly moves at all).

(Second method for determining wind speed)
The pre-processing unit 62 notifies the control device 30 that pre-processing will be performed. When the control device 30 receives this notification, the position control unit 40 moves the monitoring device 10 to a predetermined position (e.g., a position where one of the plants grown in the cultivation bed 12 can be photographed). The wind speed control unit 41 and the photography control unit 42 photograph the plant from above multiple times using the camera 24 while changing the wind speed applied to the selected plant. For example, the wind speed control unit 41 gradually increases the wind speed from 0 m/s or a weak wind, and the photography control unit 42 photographs the plant from above at predetermined time intervals. The image acquisition/transmission unit 44 then transmits the multiple images obtained by the photography and information on the wind speed at which each image was taken to the pre-processing unit 62 via the image acquisition unit 52.

The pre-processing unit 62 uses the multiple acquired images and information on the wind speed at the time each image was taken to determine the wind speed (second wind speed) to be set when calculating the petiole length and leaf area of strawberries as follows:

(1) The pre-processing unit 62 first detects the center of a plant from each of the multiple images. For example, the pre-processing unit 62 can perform deep learning using a large number of images showing the shape and center of a plant as training data, and then use the resulting training model to detect the center of the plant present in each image. The pre-processing unit 62 can also detect the center of a plant using methods other than deep learning. For example, as shown in FIG. 11(a), the image analysis unit 54 can detect multiple petioles from an image and detect the intersection of their extensions as the center of the plant. By using this method, the center of the plant can be accurately detected even if it is hidden by leaves from other plants, as shown in FIG. 11(b). Furthermore, as shown in FIG. 11(c), strawberries typically have three leaflets connected to one petiole. Therefore, the pre-processing unit 62 can, for example, perform a process of drawing a straight line (see the dashed line in FIG. 11(c)) multiple times in the longitudinal direction of the middle leaflet of the three leaflets, and determine the intersection of these straight lines as the center of the plant. The pre-processing unit 62 may prioritize detecting the center of the plant using deep learning, and if the center of the plant cannot be detected using deep learning, may detect the center of the plant using the method shown in Figure 11(a), Figure 11(b), or Figure 11(c).

(2) Next, the pre-processing unit 62 detects petioles from each image. Here, the pre-processing unit 62 detects the leaf blades in order from the first leaf, second leaf, third leaf, etc. from each image, and detects the petioles of each leaf. Detecting the petioles of each leaf in this manner is for determining the second wind speed of each leaf. The reason for detecting the center of the plant in (1) and the petioles of each leaf in (2) is that even if the leaves and petioles are crowded together, if the petioles of each leaf and the center of the plant can be detected, the distances Dn and Dm, described below, can be calculated. For example, the pre-processing unit 62 performs deep learning using a large amount of learning data that associates leaf images with the degree of unexpandedness to obtain a learning model. The pre-processing unit 62 then uses the obtained learning model to detect unexpanded leaves of the plant from within the image, and designates the detected unexpanded leaf as the first leaf. Alternatively, the pre-processing unit 62 may detect leaves from the image and designate the smallest detected leaf as the first leaf. The pre-processing unit 62 may also detect leaves from the image and determine the lightest colored leaf among the detected leaves as the first leaf. Furthermore, the pre-processing unit 62 may also detect leaves from the image and determine the leaf that is closest to the center of the plant among the detected leaves as the first leaf. The pre-processing unit 62 may preferentially apply a method for identifying the first leaf using deep learning, and if the first leaf cannot be identified using this method, may identify the first leaf using one of the other methods described above. The pre-processing unit 62 may also identify the first leaf using multiple of the methods described above and combine the multiple identification results to identify the first leaf. Furthermore, the pre-processing unit 62 may detect the angle of the line connecting the first leaf and the center of the plant relative to a reference line, and identify the leaf located approximately 144° from the center of the plant relative to the direction of growth of the first leaf as the second leaf, the leaf located approximately 144° from the center of the plant relative to the direction of growth of the second leaf (approximately 288° from the direction of growth of the first leaf) as the third leaf, and so on. The pre-processing unit 62 then detects the petioles connected to the identified first leaf, second leaf, etc. Note that deep learning can be used as a method for detecting petioles. However, this is not limiting, and for example, the petiole and one leaf blade can be detected and the connection point between them can be considered to be the petiole. Alternatively, the position where the longitudinal directions of three leaf blades intersect can be considered to be the petiole.

(3) Next, the pre-processing unit 62 measures the distance from the center of the plant to the petiole of each leaf in each image (see Dn and Dm in Figure 6(a)).

(4) Next, the pre-processing unit 62 calculates the horizontal projection movement amount M _P from each image. Specifically, the horizontal projection movement amount M _P is calculated using the following equation (4).
M _P =Dm-Dn...(4)

Note that Dn is the distance from the center of the plant to the petiole of each leaf when the wind speed is 0 m or weak, and Dm is the distance from the center of the plant to the petiole of each leaf when the wind speed is gradually increased from 0 m or weak.

(5) Next, the pre-processing unit 62 determines, for each leaf, the wind speed at which the horizontal projection movement amount M _P changes little even when the wind speed is increased, as the second wind speed. In other words, the pre-processing unit 62 determines, for each leaf, the wind speed at which the change in the horizontal projection movement amount M _P is equal to or less than a predetermined threshold, as the second wind speed. Note that the second wind speed determined as described above can be applied to plants whose planting date is the same as or close to that of the plant photographed in the above process.

(Regarding processing by the control device 30)
FIG. 12 is a flowchart showing the processing of the control device 30. It is assumed that the plants to be photographed in the following processing are plants whose planting date is the same as or close to that of the plants photographed when the pre-processing unit 62 determines the second wind speed. In this embodiment, as an example, processing for measuring the petiole length and leaf area of a predetermined leaf (e.g., the fourth leaf) will be described. Therefore, in the processing of FIG. 12 , the control device 30 uses the second wind speed of the predetermined leaf from the second wind speeds for each leaf determined by the pre-processing unit 62.

When the processing in Figure 12 begins, first in step S10, the position control unit 40 controls the motor 28 based on the detection results of the position detection device 29, and moves the monitoring device 10 to a position where the camera 24 can photograph the strawberry plants. Note that the approximate position of each plant on the cultivation bed 12 is assumed to be known in advance.

Next, in step S12, the wind speed control unit 41 turns off the blower 26 or controls the wind speed from the blower 26 to a first wind speed (weak wind) (i.e., so that the stumps are in a first state). Note that here, the first state is assumed to be a windless state. Note that if the blower 26 is in an OFF state when step S12 is performed, the wind speed control unit 41 does not control the blower 26 and maintains the state as is.

Next, in step S14, the photography control unit 42 uses the camera 24 to photograph the strawberry plants from above.

In step S16, the image acquisition/transmission unit 44 acquires the image (first image) captured in step S14 from the camera 24 and transmits the first image, information about the position at which the first image was captured obtained from the position control unit 40, and information about the wind speed of the blower 26 at which the first image was captured obtained from the wind speed control unit 41 to the information processing device 90.

Next, in step S18, the wind speed control unit 41 controls the wind speed from the blower 26 to the second wind speed (medium or strong) determined by the pre-processing unit 62 (i.e., so that the stalks are in the second state).

Next, in step S20, the photography control unit 42 uses the camera 24 to photograph the strawberry plants from above.

Next, in step S22, the image acquisition/transmission unit 44 acquires the image (second image) captured in step S20 from the camera 24 and transmits the second image, information about the position at which the second image was captured, obtained from the position control unit 40, and information about the wind speed of the blower 26 at which the second image was captured, obtained from the wind speed control unit 41, to the information processing device 90.

Then, the process returns to step S10. Note that the wind speed control unit 41 may stop the operation of the blower 26 before returning to step S10. When the process returns to step S10, the position control unit 40 moves to photograph the next plant. Then, in steps S12 to S22, the control device 30 performs the process of photographing the first and second images of the next plant.

12, the monitoring device 10 is moved (S10), a first image is captured at a first wind speed (S12, S14), a second image is captured at a second wind speed (S18, S20), and the monitoring device 10 is moved again (S10). However, this is not a limitation. For example, images of multiple plants (first images) may be captured while the monitoring device 10 is moved with the blower 26 turned off or maintained at a first wind speed, and then images of multiple plants (second images) may be captured while the monitoring device 10 is moved with the blower 26 maintained at a second wind speed. The image acquisition and transmission unit 44 may then associate the first and second images captured at the same location (i.e., the first and second images of the same plant) and transmit them to the information processing device 90 along with location information and wind speed information.

(Regarding processing by the information processing device 90)
Next, the processing of the information processing device 90 will be described with reference to the flowchart of FIG.

When the processing in Figure 13 starts, first in step S30, the image acquisition unit 52 acquires image, position information, and wind speed information transmitted from the image acquisition/transmission unit 44 of the control device 30.

Next, in step S32, the image analysis unit 54 detects the center of the plant from either the image in the first state or the image in the second state. The method for detecting the center of the plant in step S32 is similar to the method described in the processing of the pre-processing unit 62 (for example, the method using deep learning or the method shown in Figures 11(a) to 11(c)).

Next, in step S34, the image analysis unit 54 detects petioles from the image in the first state. The processing in step S34 is similar to the processing in the pre-processing unit 62.

Next, in step S36, the image analysis unit 54 uses the first image to measure the distance Dn (see Figure 6(a)) from the center of the plant to the petiole. Note that the distance Dn is measured only for the specified leaves that require measurement.

Next, in step S38, the image analysis unit 54 detects petioles from the image in the second state. The processing in step S38 is the same as that in step S34 described above.

Next, in step S40, the image analysis unit 54 uses the second image to measure the distance Dm (see Figure 6(a)) from the center of the plant to the petiole. Note that the distance Dm is measured only for the specified leaves that require measurement.

Next, in step S42, the calculation unit 56 calculates the horizontal projection movement amount M _P based on the above equation (4).

Next, in step S44, the calculation unit 56 calculates the petiole length L and leaf area A of the specified leaf using the above-mentioned approximation formulas (formulas (1) to (3) above). Details of the processing of step S44 will be described later.

Next, in step S46, the data management unit 58 manages the petiole length L and leaf area A in association with the plant position.

Next, in step S48, the output unit 60 determines whether an output request has been input. If the determination in step S48 is negative, the process returns to step S30; if the determination is positive, the process proceeds to step S50.

When the process proceeds to step S50, the output unit 60 outputs the data for which an output request has been made. Then, the process returns to step S30 and the above process is repeated.

In the above explanation, an example was described in which an image was taken using a second wind speed corresponding to a specified leaf and the petiole length L and leaf area A of the specified leaf were calculated. However, this is not limited to this, and the petiole length L and leaf area A of leaves at multiple leaf positions may also be calculated. In this case, it is necessary to photograph each plant using a second wind speed corresponding to each of the multiple leaf positions. Furthermore, the petiole length L and leaf area A of each leaf are calculated using an image (second image) taken at the second wind speed corresponding to each leaf. In this case, the petiole length L and leaf area A are managed in association with each leaf position.

(Example)
Next, the processing of the pre-processing unit 62 and the processing of step S44 in FIG. 13 will be described using an example.

14(a) and 14(b) show information on wind speed when multiple images were taken by the pre-processing unit 62, and the value of horizontal projection movement M _P obtained from the multiple images. Also, FIG. 14(c) shows the relationship between wind speed and horizontal projection movement M _P on a coordinate system (horizontal axis: wind speed, vertical axis: horizontal projection movement M _P ). This example shows the horizontal projection movement M _P of the fourth leaf of a strawberry plant.

From Figure 14(c), a wind speed of 4.503 m/s is identified as the wind speed at which the amount of leaf movement is below a predetermined level (the wind speed at which leaves hardly move at all), so the pre-processing unit 62 determines this wind speed of 4.503 m/s as the second wind speed.

Figures 15(a) to 15(e) are diagrams illustrating an example of calculating petiole length L and leaf area A using the above formulas (1) to (3). Note that this example shows values for the fourth leaf of a strawberry plant.

Figure 15(a) shows an example of the coefficients l and p (advance angle coefficients) that must be calculated in advance when using approximate formula 1. l is the coefficient of formula (1), and p is the coefficient of formula (2).

Figure 15(b) also shows an example of the petiole angle δ and petiole angle θ (see Figure 10) when pressure is applied, which are values entered by the user. Note that δ and θ may be measured in advance, or if the relationship between leaf position and δ and θ is known in advance, δ and θ may be calculated using that relationship.

Figure 15(c) shows an example of the wind speed (first wind speed) v1 in the first state and the wind speed (second wind speed) v2 in the second state. The difference between this second wind speed v2 and the first wind speed v1 is v in equations (1) and (2).

The calculation unit 56 calculates the petiole length L by substituting the value in FIG. 15(d) for M _P in Equation (3) and the values in FIG. 15(b) for δ and θ. As a result, the petiole length L is calculated to be 170.8311 mm, as shown in FIG. 15(e). Note that the true value of the petiole length L in this case was 165 mm, indicating that the above method can derive a value for the petiole length L close to the true value.

The calculation unit 56 also calculates the petiole movement amount M (see Figure 10) by substituting the petiole length L calculated as above, δ and θ in Figure 15(b), and p and v = 4.503 m/s in Figure 15(a) into the above equation (3). As a result, the value shown in Figure 15(e) (M = 48.37061 mm) is obtained as the petiole movement amount M.

Furthermore, the calculation unit 56 calculates the leaf area A by substituting the petiole movement amount M calculated as above, v = 4.503 m/s, and the value of l in Figure 15(a) into the above formula (1). As a result, the leaf area A is calculated to be 20388.81 ^mm2 , as shown in Figure 15(e). Note that the true value of the leaf area A at this time was 20240.78 ^mm2 , so it can be seen that the above method can derive a value for the leaf area A that is close to the true value.

As described above in detail, according to the first embodiment, the control device 30 controls the camera 24 to capture a first image of the plant from above in a first state in which no wind is blown onto the plant from the blower 26 or in which wind is blown onto the plant from the blower 26 at a first speed, and controls the camera 24 to capture a second image of the plant from above in a second state in which wind is blown onto the plant from the blower 26 at a second speed ( FIG. 12 ). The second speed (wind speed) is set to a speed at which the amount of leaf movement becomes less than a predetermined value (i.e., the leaf hardly moves) as the wind speed from the blower 26 increases. The information processing device 90 also calculates the difference (horizontal projection movement amount M P ) between the distance Dn from the center of the plant to the tip of the petiole of a given leaf in the first image and the second distance Dm from the center of the plant to the tip of the petiole of a given leaf in _the second image. Furthermore, the information processing device 90 calculates the petiole length L using the petiole angle θ in the first state, the petiole angle δ in the second state, and the horizontal projection shift amount M _P according to equation (3). Thus, even if the petiole is located below the leaf blade (leaflet) and is difficult to see from the outside, the petiole length L can be calculated accurately by using the first and second images (see FIG. 15( e)).

In addition, in this embodiment, the information processing device 90 calculates the value of the leaf area A based on the value of the petiole length L, the difference v in the speed of the air blown onto the plant from the blower 26 in the first state and the second state, and the petiole angles δ and θ. This allows the leaf area to be calculated with high accuracy even when leaf blades are overlapping (see Figure 15(e)).

The wind speed distribution of the blower 26 is strongest directly below the blower 26 and weaker at positions further away from the blower 26. Therefore, if the monitoring device 10 is moved while the blower 26 is maintained in a constant state, the plants transition from a state in which they are exposed to weak wind (first state) to a state in which they are exposed to strong wind (second state). Therefore, in the above embodiment, images may be captured when the plants are located away from directly below the blower 26 (first state) and when they are located close to directly below the blower 26 (second state), and the processing shown in FIG. 13 may be performed using these images. In this case, the monitoring device 10 may have a first camera positioned away from the blower 26 in the X-axis direction to capture images of the plants in the first state, and a second camera positioned near the blower 26 to capture images of the plants in the second state. Alternatively, a single camera may be used to photograph the stock in the first state and the stock in the second state at different angles, and the images may be corrected to appear as if they were photographed from the same angle, before being used in the processing shown in Figure 13 above.

Second Embodiment
Next, a second embodiment will be described. Fig. 16 shows a functional block diagram of a control device 30 and an information processing device 90 according to the second embodiment. As can be seen by comparing Fig. 16 with Fig. 3 (the first embodiment), the information processing device 90 according to the second embodiment does not include a pre-processing unit 62, and as a result, the processing of the control device 30 and the information processing device 90 differs from that of the first embodiment. Specifically, in the second embodiment, the second speed is not determined in advance, but is determined (specified) for each plant based on the photographing results of each plant.

(Regarding processing by the control device 30)
FIG. 17 is a flowchart showing the processing of the control device 30 according to the second embodiment.

As shown in FIG. 17, the control device 30 executes steps S10 to S16 in the same manner as in the first embodiment. That is, in steps S10 to S16, photographs of the plants are taken with a wind speed of 0 m/s or with wind at a first wind speed. Next, in step S18', the wind speed control unit 41 controls the wind speed from the blower 26 to the next wind speed. Note that in this embodiment, the wind speed values to be set in stages in step S18' are predefined (for example, wind speeds v1 to v7 in FIG. 14(a)). Therefore, each time step S18' is performed, the wind speed control unit 41 increases the wind speed in stages, such as wind speed v1, v2, etc.

After step S18', steps S20 and S22 are executed in the same manner as in the first embodiment. Then, in the next step S24, it is determined whether or not shooting has been completed at all wind speeds (e.g., wind speeds v1 to v7). If the determination in step S24 is negative, the process returns to step S18'; if the determination is positive, the process returns to step S10.

In this way, in this second embodiment, when photographing stocks, images are obtained of stocks exposed to wind at multiple wind speeds (e.g., v0 to v7).

(Regarding processing by the information processing device 90)
18 is a flowchart showing the processing of the information processing device 90 according to the second embodiment. The second embodiment is characterized in that the processing of step S43 is executed between steps S42 and S43 of the processing of the first embodiment (FIG. 13).

When the processing of FIG. 18 is started, steps S30 to S42 are executed in the same manner as in the first embodiment. However, as the second state images, multiple (e.g., seven) images obtained at multiple wind speeds (e.g., v1 to v7) are obtained. Therefore, in step S42, multiple (e.g., seven) horizontal projection displacements M _P are obtained. Note that the method for identifying this second wind speed is the same as the method for determining the second wind speed by the pre-processing unit 62 in the first embodiment. That is, in step S30 of FIG. 18, data such as that shown in FIG. 14(a) is obtained, and in step S42, data such as that shown in FIG. 14(b) is obtained. Therefore, in step S43, these data are used to identify the second wind speed (see FIG. 14(c)).

Thereafter, the processes from step S43 onward are performed in the same manner as in the first embodiment, using the second wind speed identified in step S43 and the horizontal projection movement amount M _P corresponding to the second wind speed.

As with the first embodiment, this allows for accurate calculation of the petiole length L even when the petiole is located below the leaf blade (leaflet) and is difficult to see from the outside. Furthermore, even when the leaf blades overlap, the leaf area A can be calculated accurately.

Furthermore, in this second embodiment, the second speed is specified for each plant, so the petiole length L and leaf area A of each plant can be calculated with high accuracy.

In the second embodiment, the change in the horizontal projection movement amount M _P may be calculated each time the processes of steps S18', S20, and S22 in Fig. 17 are executed, and the wind speed at which the change in the horizontal projection movement amount M _P becomes equal to or smaller than a predetermined value may be identified as the second wind speed. In this case, the determination in step S24 may be made positive once the second wind speed is identified.

In the above embodiment, the case where the wind speed values to be set in stages in step S18' are determined in advance is described. However, this is not limited to this. The wind speed values to be set in stages may also be determined based on the second wind speed determined in a previous process (e.g., the previous process).

It is also possible to perform the processing of the second embodiment (processing in Figures 17 and 18) on the first plant, and then perform the processing of the first embodiment (processing in Figures 12 and 13) on the second and subsequent plants. In other words, the second wind speed identified in processing the first plant may be used in processing the second and subsequent plants.

In the first and second embodiments, the monitoring device 10 moves on the ground, but this is not limited to this. For example, an unmanned aerial vehicle (drone or multicopter) equipped with a camera 24, blower 26, position detection device 29, and control device 30 may also be used as the monitoring device. In this case, the strength of the wind hitting the plants can be adjusted by adjusting the height of the unmanned aerial vehicle and the relative positions of the unmanned aerial vehicle and the observation target. In this case, the blower 26 may be omitted. Note that adjusting the height of the unmanned aerial vehicle and the relative positions of the unmanned aerial vehicle and the observation target changes the relative positions of the camera 24 and the observation target, which changes the shooting distance and shooting angle. However, the above processing can be performed taking into account the relative positions of the unmanned aerial vehicle and the observation target.

The above-described embodiment is a preferred example of the present invention. However, it is not limited to this, and various modifications are possible within the scope of the gist of the present invention.

24 Camera (photography device)
26 Blower (spraying device)
30 Control device 90 Information processing device (calculation device)
100 Information Acquisition System

Claims (4)

A blowing device that blows air onto strawberry plants from above,
a photographing device for photographing an image of the stock from above;
a control device that controls the photographing device to photograph a first image of the plant from above in a first state where no wind is blown from the blowing device to the plant or where wind is blown from the blowing device at a first speed, and that controls the photographing device to photograph a second image of the plant from above in a second state where wind is blown from the blowing device at a second speed;
a calculation device that calculates the value of the petiole length of a predetermined leaf of the strawberry plant from the image captured by the photographing device;
Equipped with
the second speed is a speed at which the movement amount of the predetermined leaf becomes equal to or less than a predetermined amount when the speed of the wind blown from the blowing device is increased,
The calculation device
determining a difference between a first distance from the center of the plant to the tip of the petiole of the specified leaf in the first image and a second distance from the center of the plant to the tip of the petiole of the specified leaf in the second image;
calculating a value of the petiole length of the specified leaf based on values of the petiole angle of the specified leaf in each of the first state and the second state and a difference between the first distance and the second distance;
A strawberry plant information acquisition system characterized by: the control device captures the second image a plurality of times while varying the second speed;
The strawberry plant information acquisition system described in claim 1, characterized in that the calculation device determines the second speed and second image to be used when calculating the petiole length value from among the multiple second speeds and multiple second images based on the multiple second images. The strawberry plant information acquisition system described in claim 1, characterized in that the values of the petiole angle of the specified leaf in each of the first and second states are pre-entered values. The calculation device
In the calculation process, a value of the leaf area of the specified leaf is calculated based on the value of the petiole length of the specified leaf, the difference in speed of the wind blown from the blowing device to the plant in the first state and the second state, and the value of the petiole angle of the specified leaf in each of the first state and the second state.
4. The strawberry plant information acquisition system according to claim 1, wherein the strawberry plant information acquisition system is a system for acquiring information on strawberry plants.