JP7784698B2 - Information processing device, flying creature extermination system, pest capture system, information processing method, and program - Google Patents

Description

The present invention relates to an information processing device, a flying creature extermination system, a pest capture system, an information processing method, and a program.

Technologies for measuring the positions of moving objects such as insects and animals are known. For example, Patent Document 1 describes collecting stereo images associated with the 3-D movement-related behavior of a laboratory animal, and processing the collected image frames to obtain the 3-D movement parameters of the laboratory animal.

Special Publication No. 2009-500042

The technology described in Patent Document 1 measures the position of a moving object, but does not predict the position of the moving object. In particular, when the moving object is a fast-moving pest or the like, the measured position may differ significantly from the actual position due to the time lag caused by reading images captured by a camera and performing calculations. Therefore, there is a need for technology that can predict the position of a moving object even when the object is moving at high speed.

The present invention was made in consideration of these circumstances, and one of its objectives is to provide an information processing device, flying creature extermination system, pest capture system, information processing method, and program that can predict the position of a moving object even when the moving object is moving at high speed.

An information processing device according to one aspect of the present invention includes a derivation unit that derives time-series state information of a moving object, including at least a position, based on point cloud information representing the moving object extracted from each of a plurality of images captured in time series of the moving object, and a prediction unit that predicts the future position of the moving object based on the time-series state information and information about the surrounding environment of the moving object.

The prediction unit may predict the future position by repeating a process of correcting a previously estimated state value of the moving object using the observed values of the time-series state information to obtain a retrospectively estimated state value of the moving object.

The prediction unit may predict the future position by inputting the time-series state information and the surrounding environment information into a trained model that receives the time-series state information and surrounding environment information of the moving object as input and is trained to output the future position of the moving object.

The surrounding environment information may include at least one of information regarding wind direction, wind speed, air temperature, temperature, precipitation, light source position, light source spectral intensity, season, and time of day in the environment in which the moving object is located.

the moving object is a flying pest,
The time-series state information may further include at least one of the roll angle, pitch angle, wing flapping frequency, wing direction, wing angle, abdomen direction, abdomen twist, abdomen bending, head direction, head twist, antenna direction, difference in movement of both wings, leg direction, and abdomen size of the pest.

the moving object is an animal belonging to the avian family,
The time-series state information may further include at least one of the animal's wing flapping frequency, wing orientation, wing angle, abdomen orientation, head orientation, leg orientation, and abdomen size.

the moving object is a quadrupedal animal,
The time-series state information may further include at least one of the orientation of the animal's abdomen, the orientation of its head, the orientation of its legs, and the size of its abdomen.

According to the present invention, the position of a moving object can be predicted even when the object is moving at high speed. Furthermore, the position of a moving object can be predicted even when the object's speed changes frequently or the object moves in a non-linear manner.

1 is a diagram illustrating an example of a usage environment and configuration of an information processing apparatus 100 according to a first embodiment. 10 is a diagram showing an example of a three-dimensional point cloud extracted by a point cloud extraction unit 120. FIG. 10 is a diagram for explaining a method by which the state derivation unit 130 extracts state information related to wings of flying pests. FIG. 10 is a diagram for explaining a method by which the state derivation unit 130 extracts state information related to the abdomen of a flying pest. FIG. 10 is a diagram for explaining a method by which the state derivation unit 130 extracts state information relating to the head of a flying pest. FIG. 10 is a diagram for explaining a method by which the state derivation unit 130 extracts state information related to the antennae of flying pests. FIG. 10 is a diagram for explaining a method by which the state derivation unit 130 extracts state information relating to the legs of flying pests. FIG. FIG. 1 is a diagram for explaining an outline of a method for predicting future positions of flying pests by Kalman filter processing. This is a diagram for explaining a method for predicting the future positions of flying pests using a deep neural network. The flow of processing executed by the information processing apparatus 100 according to the first embodiment will be described. FIG. 10 is a diagram illustrating an example of a usage environment and configuration of an information processing apparatus 100 according to a second embodiment. The flow of processing executed by the information processing apparatus 100 according to the second embodiment will be described.

[First embodiment]
An information processing device 100 according to a first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing an example of a usage environment and configuration of the information processing device 100 according to the first embodiment. The information processing device 100 operates in cooperation with, for example, a camera 10 and a laser irradiation device 50. The combination of the information processing device 100 and the laser irradiation device 50 is an example of a "flying creature extermination system."

The camera 10 is a 3D camera, such as a stereo camera, that captures images of an object from different directions, enabling it to acquire depth information of the object in addition to two-dimensional vertical and horizontal information. The camera 10 may also be a 3D laser scanner, such as a LIDAR (light detection and ranging) scanner. The camera 10 is installed in an area inhabited by flying pests, such as common cutworms, facing in any direction (e.g., horizontal or vertical relative to the ground). The camera 10 is communicatively connected to the information processing device 100 via a wired cable or a wireless network. In response to instructions from the information processing device 100 or periodically, the camera 10 captures images of the flying pests in chronological order and transmits the captured 3D images (hereinafter sometimes simply referred to as "images") to the information processing device 100. Figure 1 shows an example in which the camera 10 is installed horizontally relative to the ground, and the 3D image is recognized with the vertical direction, horizontal direction, and height direction of the camera 10 as the X-axis, Y-axis, and Z-axis, respectively. Flying pests are an example of a "moving object."

The laser irradiation device 50 is, for example, a high-power laser device that outputs a laser capable of killing flying pests, and is communicatively connected to the information processing device 100 via a wired cable or wireless network. More specifically, when the laser irradiation device 50 receives the future position of a flying pest predicted by the position prediction unit 140 of the information processing device 100 (described below), it controls the built-in galvanometer mirror to emit a laser toward the future position, and outputs the laser. This makes it possible to exterminate flying pests.

The information processing device 100 is a computing device such as a personal computer or tablet terminal. The information processing device 100 includes, for example, an image acquisition unit 110, a point cloud extraction unit 120, a state derivation unit 130, a position prediction unit 140, and a memory unit 150. Each of the image acquisition unit 110, the point cloud extraction unit 120, the state derivation unit 130, and the position prediction unit 140 is implemented by a hardware processor, such as a CPU (Central Processing Unit), executing a program (software). Some or all of these components may be implemented by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be implemented by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as a hard disk drive (HDD) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device. The storage unit 150 is realized by a storage device such as a HDD, flash memory, or RAM (random access memory).

The image acquisition unit 110 acquires images of flying pests captured in time series by the camera 10 from the camera 10 and stores them in the memory unit 150.

The point cloud extraction unit 120 extracts a three-dimensional point cloud representing a flying pest as point cloud information from each of multiple images of the flying pest captured in time series. Figure 2 is a diagram showing an example of a three-dimensional point cloud extracted by the point cloud extraction unit 120. As shown on the left side of Figure 2, when the point cloud extraction unit 120 extracts a three-dimensional point cloud representing the flying pest from the image, the state derivation unit 130 calculates the position P of the flying pest by taking the average value of each point that makes up the three-dimensional point cloud.

As shown on the right side of Figure 2, the state derivation unit 130 further calculates a regression plane RP that approximates the three-dimensional point cloud by, for example, applying the least squares method to the three-dimensional point cloud. The state derivation unit 130 calculates the attitude angles (roll angle and pitch angle) of the flying pest based on the tilt of the regression plane RP in XYZ space. Alternatively, the state derivation unit 130 may use a template matching technique to determine the attitude angles of the flying pest by comparing the three-dimensional point cloud with shape data and angle data of the flying pest pre-stored in the memory unit 150.

The state derivation unit 130 further derives state information such as the wing flapping frequency, wing direction, wing angle, abdomen direction, abdomen twist, abdomen curvature, head direction, head twist, antenna direction, difference in wing movement, leg direction, and abdomen size of the flying pest in chronological order based on the point cloud information of each image.

Figure 3 is a diagram illustrating how the state derivation unit 130 extracts state information related to the wings of a flying pest. The state derivation unit 130 first applies techniques such as clustering and pattern matching to the 3D point cloud to extract a partial point cloud corresponding to the wings of the flying pest as a point cloud of interest from the 3D point cloud. Next, the state derivation unit 130 calculates a representative vector RV representing the wings of the flying pest by averaging vectors whose starting point is the position P of the flying pest and whose ending point is each point in the point cloud of interest. For example, as shown in the lower right of Figure 3, the state derivation unit 130 can calculate the wing flapping frequency by measuring the temporal progression of the representative vector RV and counting the moment when the representative vector RV matches the vector at the initial point as one flapping.

The state derivation unit 130 further calculates the wing orientation and wing angle based on the focus point cloud representing the wings of the flying pest. More specifically, for example, the state derivation unit 130 first applies a method such as clustering or pattern matching to the 3D point cloud to extract a partial point cloud corresponding to the abdomen of the flying pest from the 3D point cloud. Next, the state derivation unit 130 calculates a regression plane RP that approximates the abdomen of the flying pest by applying, for example, the least squares method to the partial point cloud corresponding to the abdomen of the flying pest. The state derivation unit 130 calculates the angle between the representative vector RV representing the wings of the flying pest and the regression plane RP that approximates the abdomen, and determines this as the wing angle.

The state derivation unit 130 further calculates a regression surface RS that approximates the wings of the flying pest, for example, by applying the least squares method to the partial point cloud corresponding to the wings of the flying pest. The state derivation unit 130 determines the normal vector NV of the calculated regression surface RS as the orientation of the wing.

The state derivation unit 130 further determines the differences in the movement of both wings by, for example, calculating the difference vector between the representative vector RV representing the left wing of the flying pest and the representative vector RV representing the right wing of the flying pest. Through the above processing, the flapping frequency, wing direction, wing angle, and differences in the movement of both wings are determined.

Figure 4 is a diagram illustrating how the state derivation unit 130 extracts state information related to the abdomen of a flying pest. By applying techniques such as clustering and pattern matching to the 3D point cloud, the state derivation unit 130 extracts a partial point cloud corresponding to the abdomen of the flying pest as a point cloud of interest, and derives state information such as the orientation, twist, curvature, and size of the abdomen. More specifically, for example, the state derivation unit 130 first calculates an approximated surface RS that approximates the abdomen of the flying pest by applying a technique such as the least squares method to the partial point cloud corresponding to the abdomen of the flying pest. For example, the state derivation unit 130 calculates the normal vector NV of the approximated surface RS as the orientation of the abdomen, calculates the curvature of the approximated surface RS as the curvature of the abdomen, and calculates the torsion rate (torsion tensor) of the approximated surface RS as the torsion of the abdomen.

Furthermore, the state derivation unit 130 calculates, for example, a closed surface that approximates the partial point cloud corresponding to the abdomen of the flying pest using a technique such as fitting, and calculates the volume of the area enclosed by this closed surface as the size of the abdomen. Through the above processing, the orientation, twisting, curvature, and size of the abdomen are determined.

Figure 5 is a diagram illustrating how the state derivation unit 130 extracts state information related to the head of a flying pest. The state derivation unit 130 applies techniques such as clustering and pattern matching to the 3D point cloud to extract a partial point cloud corresponding to the head of the flying pest as a point cloud of interest, and derives state information such as the twist and orientation of the head. More specifically, for example, the state derivation unit 130 calculates an approximated surface RS that approximates the head of the flying pest by applying a technique such as the least squares method to the partial point cloud corresponding to the head of the flying pest, as in the case of the abdomen. The state derivation unit 130, for example, calculates the torsion rate (torsion tensor) of the approximated surface RS as the twist of the head, and calculates the normal vector NV of the approximated surface RS as the orientation of the head. The twist and orientation of the head are determined through the above processing.

Figure 6 is a diagram illustrating how the state derivation unit 130 extracts state information related to the antennae of flying pests. The state derivation unit 130 applies techniques such as clustering and pattern matching to the three-dimensional point cloud to extract partial point clouds corresponding to the antennae of the flying pest as a point cloud of interest, and derives state information such as the direction of the antennae. More specifically, for example, the state derivation unit 130 applies a technique such as the least squares method to the partial point cloud corresponding to the antennae of the flying pest to calculate regression lines RL1 and RL2 that approximate the left and right antennae of the flying pest. For example, the state derivation unit 130 calculates directional vectors V1 and V2 of the regression lines RL1 and RL2 as the directions of the left and right antennae. The direction of the antennae is determined through the above processing.

Figure 7 is a diagram illustrating how the state derivation unit 130 extracts state information related to the legs of flying pests. The state derivation unit 130 applies techniques such as clustering and pattern matching to the three-dimensional point cloud to extract partial point clouds corresponding to the legs of the flying pest as a point cloud of interest, and derives state information such as the direction of the legs. More specifically, for example, the state derivation unit 130 applies a technique such as the least squares method to the partial point clouds corresponding to the legs of the flying pest to calculate a regression line RC that approximates the legs of the flying pest for each leg. The state derivation unit 130 calculates, for example, the direction vector V of the regression line RL as the direction of the leg. The leg direction is determined by the above process.

The above describes a method by which the state derivation unit 130 determines state information such as the wing flapping frequency, wing direction, wing angle, abdomen direction, abdomen twist, abdomen curvature, head direction, head twist, antenna direction, difference in wing movement, leg direction, and abdomen size of a flying pest. However, the above-described method is merely an example, and any method may be used to calculate these values.

The position prediction unit 140 predicts the future position of flying pests based on the state information derived in time series by the state derivation unit 130 and pre-set surrounding environment information. Here, surrounding environment information refers to information that represents the surrounding conditions of the environment in which the camera 10, laser irradiation device 50, and information processing device 100 are installed, and includes, for example, at least one of the following information: wind direction, wind speed, air temperature, temperature, precipitation, host plant location, host plant type, host plant growth stage, host plant odor components, light source (e.g., street lamp), light source spectral intensity, moon position, moon spectral intensity, bat location, presence or absence of bat ultrasound, location of female flying pests (if the flying pest is male), season, and time of day.

For example, when the start time of observation of the status information is represented by 0 and the current time by t, the position prediction unit 140 predicts the future position of the flying pest at time t+n, n steps ahead (n is any positive integer), according to equation (1).

In equation (1), Y _k (k=0, ..., t-2, t-1, t) is state information observed during the period from time 0 to time t, θ _t is surrounding environment information at pre-set time t, Y^ _t+n is flying pest state information at time t+n, and f _θt is a linear or nonlinear function that takes surrounding environment information θ _t as an internal parameter, inputs Y _k (k=0, ..., t-2, t-1, t), and outputs Y^ _t+n . More specifically, Y _t and θ _t are vectors represented by equations (2) and (3).

In equation (2), _xt , _yt , and _zt are the x-, y-, and z-components of the position P of the flying pest, respectively. _φt is the roll angle of the flying pest, _θt is the pitch angle of the flying pest, and _ωt is the wing flapping frequency. Although not shown in equation (2), _Yt may further include at least some of the following: wing orientation, wing angle, abdomen orientation, abdomen twist, abdomen curvature, head orientation, head twist, antenna orientation, difference in movement of both wings, leg orientation, and abdomen size. More generally, _Yt may include at least the positions _xt , _yt , and _zt of the flying pest.

In equation (3), a _t , b _t , c _t , d _t , e _t ... include at least some of the information related to wind direction, wind speed, air temperature, temperature, precipitation, location of host plant, type of host plant, growth stage of host plant, odor components of host plant, location of light source (e.g., street light), spectral intensity of light source, position of the moon, spectral intensity of the moon, location of bats, presence or absence of bat ultrasound, location of female flying pests (if flying pests are male), season, time of day, etc.

Under the above assumptions, the position prediction unit 140, for example, constructs a state space model including a system model and observation model for flying pests, and predicts the future position of the flying pest at time t+n by performing Kalman filter processing. More specifically, the system model and observation model for flying pests are defined by equations (4) and (5), respectively.

In equation (4), X _t is the state variable of the flying pest at time t, and is defined by equation (6).

In equation (6), _xt , _yt , and _zt are the x, y, and z components of the position P of the flying pest, respectively; x ^· _t , y ^· _t , and z ^· _t are the velocities obtained by differentiating the x, y, and z components of the position P of the flying pest, respectively; _φt is the roll angle of the flying pest; _θt is the pitch angle of the flying pest; and _ωt is the wing flapping frequency.

Returning to the explanation of equation (4), _Aθ is a system matrix with the surrounding environment information as an internal parameter, _Vt is system noise that follows a normal distribution with mean 0 and variance Q, and b is a constant vector. In equation (5), _Yt is the observed value of flying pests at time t, and is defined by equation (7).

In equation (7), _xt , _yt , and _zt are the x, y, and z components of the position P of the flying pest, respectively, _φt is the roll angle of the flying pest, _θt is the pitch angle of the flying pest, and _ωt is the wing flapping frequency.

Returning to the explanation of equation (5), C is a transformation matrix that converts the state variable _Xt of the flying pest into the observation value _Yt , and _Wt is the observation noise that follows a normal distribution with mean 0 and variance R. The position prediction unit 140 predicts the future position of the flying pest at time t+n by performing Kalman filter processing on the state space model constructed as described above.

8 is a diagram for explaining an overview of a method for predicting a future position using Kalman filter processing. Fig. 8 shows a flow for updating the Kalman gain G _t and obtaining a model for calculating a predicted value Y^ _{t+n n} steps ahead by sequentially inputting the observed values Y _k (k=0, ..., t-2, t-1, t) into a Kalman filter composed of a prediction step and a filtering step. More specifically, the position prediction unit 140 uses the observed values Y _k (k=0, ..., t-2, t-1, t) to correct the pre-estimated state value x^ ^- _t of the flying pest to obtain a post-estimated state value x^ _t , thereby updating the Kalman gain G _t and obtaining a model for calculating the predicted value Y^ _{t+n n} steps ahead.

Figure 9 is a diagram illustrating how deep neural networks can be used to predict the future positions of flying pests. As shown in FIG. 9 , the information processing device 100 uses as input data some or all of the following information: the three-dimensional position, three-dimensional speed, roll angle, pitch angle, wing flapping frequency, wing direction, wing angle, abdomen direction, abdomen twist, abdomen bending, head direction, head twist, antennae direction, difference in wing movement, leg direction, and abdomen size of the flying pest at times 0, 1, ..., t-2, t-1, and t, as well as the wind direction, wind speed, temperature, humidity, precipitation, host plant position, host plant type, host plant growth stage, host plant odor components, light source position, light source spectral intensity, moon position, moon spectral intensity, bat position, presence or absence of bat ultrasound, female bat position (for males), season, and time of day. A trained model trained using, for example, a deep neural network is generated in advance so as to output the future position of the flying pest at time t+n. By inputting the above data at times 0, 1, ..., t-2, t-1, and t into the trained model, the position prediction unit 140 can predict the future position of flying pests at time t+n.

Next, the flow of processing executed by the information processing device 100 according to the first embodiment will be described with reference to FIG. 10. FIG. 10 is a flowchart showing an example of the flow of processing executed by the information processing device 100 according to the first embodiment.

First, the image acquisition unit 110 acquires images of flying pests captured in chronological order by the camera 10 (step S100). Next, the point cloud extraction unit 120 extracts point cloud information representing the flying pests in chronological order from each acquired image (step S101). Next, the state derivation unit 130 derives state information of the flying pests in chronological order based on the point cloud information extracted in chronological order (step S102).

Next, the position prediction unit 140 predicts the future position of the flying pest using Kalman filter processing or a trained model generated using a deep neural network, based on the flying pest status information derived in time series by the state derivation unit 130 and the surrounding environment information (step S103). Next, the information processing device 100 transmits the predicted future position to the laser irradiation device 50, and the laser irradiation device 50 outputs a high-power laser toward the predicted future position (step S104). This completes the processing of this flowchart.

According to the first embodiment described above, the future positions of flying pests are predicted based on the status information of the flying pests derived in time series and information about the surrounding environment. This makes it possible to predict the position of a moving object even if the object is moving at high speed. Furthermore, it is possible to predict the position of a moving object even if the object's speed changes frequently or if it moves in a non-linear manner.

In the first embodiment, an example was described in which the future position of flying pests was predicted and a high-power laser was output toward the predicted future position. However, the configuration of the first embodiment can also be applied to avian animals, such as crows. In this case, the state information derived in time series may include, in addition to position information, at least one of the animal's wing flapping frequency, wing direction, wing angle, abdomen direction, head direction, leg direction, and abdomen size. Similar to pests, extermination is performed by outputting a high-power laser toward the predicted future position.

[Second embodiment]
In the first embodiment, the present invention is applied to a case where the moving object whose future position is predicted is a flying pest. However, the present invention is not limited to such a configuration and can be applied to predicting the future position of any moving object. In the second embodiment, as another example, the present invention is applied to predicting the future position of a deer. Below, only the configuration that differs from the first embodiment will be described, and a description of the common configuration will be omitted.

Figure 11 is a diagram showing an example of the usage environment and configuration of the information processing device 100 according to the second embodiment. The information processing device 100 operates in cooperation with, for example, a camera 10 and an animal capture device 50B. The combination of the information processing device 100 and the animal capture device 50B is an example of a "vermin capture system."

The animal capture device 50B is, for example, a drop net that is stretched through the trees in the forest and has a mechanism for capturing deer by dropping the net when the deer comes under it. The animal capture device 50B is controlled by the information processing device 100, and operates to capture deer when it is determined that the device's own position is at the future position predicted by the position prediction unit 140.

As in the first embodiment, the position prediction unit 140 predicts the deer's future position using Kalman filter processing or a trained model generated using a deep neural network, based on the deer's state information derived in time series by the state derivation unit 130 and information about the surrounding environment.

In this case, in the above-mentioned formula (2), for example, _xt , _yt , and _zt are the x-, y-, and z-components of the deer's position P, _φt is the orientation of the deer's abdomen, _θt is the orientation of the deer's head, and _ωt is the orientation of the deer's legs. Instead of or in addition to these parameters, the size of the deer's abdomen may be set as the status information, or more generally, _Yt may include at least the positions _xt , _yt , and _zt of the flying pests.

Furthermore, in the above-mentioned equation (3), the surrounding environment information _θt includes at least one of information related to, for example, wind direction, wind speed, air temperature, temperature, precipitation, topography, tree position, light source position, light source spectral intensity, sound source position, sound source intensity, season, and time of day.

Furthermore, in the above equation (6), _xt , _yt , ^and _zt are the x- _, y-, and z-components of the deer's position P, respectively; ^xt , _yt , _and ^zt are the velocities obtained by differentiating the x-, y-, and z-components of the deer's position P, respectively; _φt is the orientation of the deer's abdomen; _θt is the orientation of the deer's head; and _ωt is the orientation of the deer's legs.

Furthermore, in the above-mentioned equation (7), _xt , _yt , and _zt are the x-, y-, and z-components of the deer's position P, respectively, _φt is the orientation of the deer's abdomen, _θt is the orientation of the deer's head, and _ωt is the orientation of the deer's legs. Under the above conditions, the position prediction unit 140 can predict the deer's future position by Kalman filter processing, as in the first embodiment.

Next, the flow of processing executed by the information processing device 100 according to the second embodiment will be described with reference to FIG. 12. FIG. 12 is a flowchart showing an example of the flow of processing executed by the information processing device 100 according to the second embodiment.

First, the image acquisition unit 110 acquires images of flying pests captured in chronological order by the camera 10 (step S100). Next, the point cloud extraction unit 120 extracts point cloud information representing the flying pests in chronological order from each acquired image (step S101). Next, the state derivation unit 130 derives state information of the flying pests in chronological order based on the point cloud information extracted in chronological order (step S102).

Next, the position prediction unit 140 predicts the future position of the flying pest using Kalman filter processing or a trained model generated using a deep neural network, based on the flying pest status information derived in time series by the state derivation unit 130 and the surrounding environment information (step S103). Next, the information processing device 100 transmits the predicted future position to the animal capture device 50B, and the animal capture device 50B operates when it determines that its own position is at the future position predicted by the position prediction unit 140 (step S104). This completes the processing of this flowchart.

According to the second embodiment described above, the future position of a deer is predicted based on the deer's state information derived in time series and the surrounding environment information. This makes it possible to predict the position of a moving object even if the object is moving at high speed.

REFERENCE SIGNS LIST 10 Camera 50 Laser irradiation device 50B Animal capture device 100 Information processing device 110 Image acquisition unit 120 Point cloud extraction unit 130 State derivation unit 140 Position prediction unit 150 Storage unit

Claims (10)

a derivation unit that derives time-series state information of the moving object, including at least a position, based on point cloud information representing the moving object extracted from each of a plurality of images obtained by capturing the moving object in time series;
a prediction unit that predicts a future position of the moving object based on the time-series state information and surrounding environment information of the moving object ,
the prediction unit predicts the future position by repeating a process of correcting a state value of the moving object estimated in advance using the observed value of the time-series state information to obtain a state value of the moving object estimated ex post.
Information processing device. a derivation unit that derives time-series state information of the moving object, including at least a position, based on point cloud information representing the moving object extracted from each of a plurality of images obtained by capturing the moving object in time series;
a prediction unit that predicts a future position of the moving object based on the time-series state information and surrounding environment information of the moving object,
the prediction unit predicts the future position by inputting the time-series state information and the surrounding environment information into a trained model that receives the time-series state information and the surrounding environment information of a moving object as input and is trained to output a future position of the moving object;
Information processing device. the surrounding environment information includes at least one of information regarding wind direction, wind speed, air temperature, temperature, amount of precipitation, position of a light source, spectral intensity of a light source, season, and time of day in the environment where the moving object is located;
3. The information processing device according to claim 1 or 2 . the moving object is a flying pest,
The time-series state information further includes at least one of the roll angle, pitch angle, wing flapping frequency, wing direction, wing angle, abdomen direction, abdomen twist, abdomen bending, head direction, head twist, antenna direction, difference in movement of both wings, leg direction, and abdomen size of the pest.
The information processing device according to claim 1 . the moving object is an animal belonging to the avian family,
the time-series state information further includes at least one of a wing flapping frequency, a wing orientation, a wing angle, an abdomen orientation, a head orientation, a leg orientation, and an abdomen size of the animal;
The information processing device according to claim 1 . the moving object is a quadrupedal animal,
The time-series state information further includes at least one of an orientation of the abdomen, an orientation of the head, an orientation of the legs, and a size of the abdomen of the animal.
The information processing device according to claim 1 . The information processing device according to claim 4 or 5 ;
a laser irradiation device that is controlled by the information processing device and that irradiates a laser toward the future position;
A flying creature extermination system equipped with: The information processing device according to claim 6 ;
an animal capture device controlled by the information processing device and operable when the location of the device itself is determined to be at the future location;
A pest capture system comprising: The computer
deriving time-series state information of the moving object, including at least a position, based on point cloud information representing the moving object extracted from each of a plurality of images obtained by capturing the moving object in time series;
predicting a future position of the moving object based on the time-series state information and surrounding environment information of the moving object ;
The prediction predicts the future position by repeating a process of correcting a state value of the moving object estimated in advance using the observed value of the time-series state information to obtain a state value of the moving object estimated ex post.
Information processing methods. On the computer,
deriving time-series state information of the moving object, including at least a position, based on point cloud information representing the moving object extracted from each of a plurality of images obtained by capturing the moving object in time series;
predicting a future position of the moving object based on the time-series state information and surrounding environment information of the moving object ;
The prediction predicts the future position by repeating a process of correcting a state value of the moving object estimated in advance using the observed value of the time-series state information to obtain a state value of the moving object estimated ex post.
program.