JP7808065B2 - Data processing device, data processing system, and data processing method - Google Patents

Description

The present invention relates to a data processing device, a data processing system, and a data processing method.

In marine ecosystems where seaweeds such as eelgrass and kelp (hereinafter referred to as "seaweed") exist, carbon dioxide ( _CO2 ) is absorbed through photosynthesis and other processes, with some of the absorbed _CO2 remaining in the soil and seawater. The Japan Blue Economy Technology Research Association (JBE) has established the J Blue Credit (registered trademark) system, which targets _CO2 absorption by marine ecosystems. This system certifies and issues credits equivalent to the amount of _CO2 absorbed, and aims to promote credit trading with companies or organizations that are working to reduce _CO2 .

A company or organization wishing to issue credits (in other words, a credit seller) submits an application for credits to JBE, for example, along with the results of seaweed bed observations. JBE then issues an appropriate amount of credits to the applicant based on the results of the seaweed bed observations. JBE has published an application guide (for example, Non-Patent Document 1), which states that the amount of _CO2 absorbed by a seaweed bed (also known as the amount of blue carbon) is determined by the product of the area of the seaweed bed and the _CO2 absorption coefficient. While values found in literature are acceptable for the absorption coefficient, actual measurements of the seaweed bed's area are required.

One example of a technology for observing seaweed beds is a method using a sonar system (see, for example, Patent Document 1). According to the method described in Patent Document 1, ultrasonic waves are transmitted from the ocean surface to the sea, and the reflected waves are measured, making it possible to observe the distribution of seaweed beds over a wide area with high resolution. However, since it is difficult to receive GPS (Global Positioning System) radio waves underwater, the method described in Patent Document 1 links the location information of seaweed beds with the distribution of seaweed beds by placing measuring instruments on the sea surface.

Japanese Patent Application Laid-Open No. 2019-24377

Japan Blue Economy Technology Research Association (JBE), "J Blue Credit® (Trial) Certification Application Guide - Climate Change Countermeasures Using Blue Carbon - Ver. 2.1," [online], September 2022, [accessed January 24, 2023], Internet: <https://www.blueeconomy.jp/files/jbc2022/20220916_J-BlueCredit_Guideline_v2.1.pdf>

The method described in Patent Document 1 does not take into account the depth-wise length of seaweed, which results in a large error in the surface area of seaweed (or seaweed beds) calculated from observation data, and this error also results in a large error in the amount of _CO2 absorbed by seaweed.

One of the objects of the present invention is to improve the accuracy in determining the amount of CO ₂ absorbed by seaweed.

A data processing device according to a first aspect of the present invention comprises an acquisition unit that acquires image data generated by using an underwater camera to capture images of underwater seaweed from a direction different from above the seaweed; a modeling unit that performs image analysis of the captured image data to model a shape of the seaweed, including its depth-wise length and its width; and an estimation unit that estimates the amount of carbon dioxide absorbed by the seaweed based on the modeled shape.

The captured image data may include multiple captured image data sets generated by capturing images of the seaweed from multiple different directions, including a direction capturing an image of the side of the seaweed, and the modeling unit may perform three-dimensional modeling of the shape based on the multiple captured image data sets.

The estimation unit may estimate the amount of carbon dioxide absorption based on the surface area of the shape and the carbon dioxide absorption coefficient of the seaweed, which corresponds to the brightness depending on the water depth at which the seaweed is located as imaged by the underwater camera.

The estimation unit may divide the shape into a plurality of blocks in the longitudinal direction and estimate the amount of carbon dioxide absorption for each block using the carbon dioxide absorption coefficient of the seaweed corresponding to the brightness depending on the water depth at which the block is located.

The estimation unit may determine the number of blocks into which the shape is divided based on the depth-wise length of the shape.

The estimation unit may determine the number of blocks into which the shape is divided based on data regarding the distribution of brightness according to the water depth.

The estimation unit may determine a larger number of blocks to divide the shape into in the longitudinal direction for an area in which the degree of change in brightness indicated by the brightness distribution according to the water depth is greater.

The modeling unit may identify the type of seaweed by inputting the captured image data into a machine learning model that, when image data is input, outputs a type of seaweed that is more likely to be included in the input image data than a reference value, and the estimation unit may estimate the amount of carbon dioxide absorption using a carbon dioxide absorption coefficient corresponding to the identified type of seaweed.

The captured image data includes multiple captured image data sets generated by capturing images of the seaweed from multiple different directions, including a direction capturing the side of the seaweed, and the modeling unit may determine whether to apply 3D modeling or 2D modeling to model the shape of the seaweed, depending on the result of determining in the image analysis whether the seaweed is a seaweed bed where the seaweed grows in colonies.

A data processing system according to a second aspect of the present invention comprises an underwater camera that captures images of underwater seaweed; an acquisition unit that acquires image data generated by using the underwater camera to capture images of the seaweed from a direction different from above the seaweed; a modeling unit that performs image analysis of the captured image data to model a shape of the seaweed including its depth length and its width; and an estimation unit that estimates the amount of carbon dioxide absorbed by the seaweed based on the modeled shape.

A third aspect of the data processing method of the present invention involves a computer functioning as a data processing device acquiring image data generated by using an underwater camera to capture images of underwater seaweed from a direction different from above the seaweed, performing image analysis of the captured image data to model a shape of the seaweed including its depth-wise length and its width, and estimating the amount of carbon dioxide absorbed by the seaweed based on the modeled shape.

The above comprehensive or specific aspects may be realized by a system, device, method, integrated circuit, computer program, or recording medium, or by any combination of two or more of a system, device, method, integrated circuit, computer program, and recording medium.

According to the present invention, it is possible to improve the accuracy in determining the amount of CO ₂ absorbed by seaweed.

1 is a diagram schematically illustrating an overview of a seaweed bed observation system according to an embodiment. FIG. FIG. 1 is a diagram illustrating the difference in CO ₂ absorption amount by seaweed depending on water depth. FIG. 1 is a diagram showing a schematic overview of processing by the seaweed bed observation system. 1 is a block diagram showing an example of the configuration of a data processing device according to an embodiment; 4 is a flowchart illustrating an example of data processing by the data processing device according to the embodiment.

Embodiments will be described below with reference to the drawings as appropriate. The same elements will be designated by the same reference numerals throughout this specification unless otherwise specified. The matters described below with reference to the accompanying drawings are intended to explain exemplary embodiments and are not intended to represent the only embodiments. For example, when an order of operations is indicated in an embodiment, the order of operations may be changed as appropriate as long as no inconsistencies arise in the overall operation.

In addition, in the embodiments, more detailed explanations than necessary may be omitted. For example, detailed explanations of publicly known or well-known technical matters may be omitted to avoid unnecessarily lengthy explanations and/or ambiguity of technical matters or concepts, and to facilitate understanding by those skilled in the art. Furthermore, duplicate explanations of substantially identical configurations, functions, and/or operations may be omitted.

<Outline of the Seaweed Bed Observation System S>
Fig. 1 is a diagram showing a schematic overview of a seaweed bed observation system S according to one embodiment. The seaweed bed observation system S is a system for estimating the amount of carbon dioxide (CO ₂ ) absorbed by seaweed. As shown in Fig. 1 , the seaweed bed observation system S illustratively includes an underwater camera 2 for capturing images of seaweed in the ocean, and a ship-type surface drone 3. Before describing the configuration and operation of the seaweed bed observation system S, a method for observing seaweed beds where seaweed grows will be described.

The main methods for observing seaweed beds can be divided into two categories: aerial photography by drones and surveys by divers. While aerial photography by drones allows for efficient observation of wide areas, the accuracy of the observations can be reduced due to factors such as the inability to distinguish between seaweed beds and reefs or rocks. In contrast, surveys by divers allow for detailed observation of seaweed beds, but are not suitable for wide-area observations, making them less efficient. As such, each of the two methods has its advantages and disadvantages, making it difficult to achieve both accuracy and efficiency in seaweed bed observations.

Furthermore, when measuring the area of a seaweed bed, if the bed is photographed from above with an underwater camera, it is difficult to grasp the height of the seaweed (in other words, the length of the seaweed in the direction of the water depth), which reduces the accuracy of measuring the surface area of the seaweed.Since seaweed in water (for example, in the ocean) absorbs _CO2 through photosynthesis, any error in the surface area of the seaweed will also result in an error in the amount of _CO2 absorbed, which is calculated based on the surface area.

Another source of error is the fact that the brightness of light (or, in other words, light intensity) received by seaweed in the ocean varies depending on the depth from the ocean surface. For example, as shown diagrammatically in Figure 2, the deeper the water is from the ocean surface, the less brightness of light received by seaweed, and therefore the lower the photosynthetic activity of seaweed.

Therefore, even for the same seaweed, the activity of photosynthesis differs between the parts of the seaweed close to the water surface and those further from the water surface (deeper), resulting in different amounts of _CO2 absorption. Therefore, the depth length of the seaweed is one of the important parameters for calculating the amount of _CO2 absorption by seaweed.

Therefore, in the present embodiment described below, we propose to perform image analysis of captured image data of seaweed to model the shape of the seaweed, including its depth length and width, and to calculate the amount of _CO2 absorption based on the modeled shape of the seaweed. As will be described later, modeling of the shape of the seaweed may be either three-dimensional (3D) modeling or two-dimensional (2D) modeling.

In this specification, a place in the ocean where seaweed grows in colonies is referred to as a "seaweed bed," and types of "seaweed beds" include, for example, "eelgrass beds," "sardine beds," "kelp beds," "Eisenia bicolor beds," "wakame beds," and "gemonia beds."

The types of seaweed (hereinafter sometimes abbreviated as "algae species") that primarily make up "eelgrass beds" include, for example, "Zostera marina," "Zostera sieboldii," "Sugamo," and "Ryukyu-usugamo," while the types of seaweed that primarily make up "Sargassum beds" include, for example, "Akamoku," "Sargassum sieboldii," "Sargassum sieboldii," and "Sargassum sieboldii."

The algae species that primarily make up "kelp beds" are, for example, "Laminaria japonica," "Narrow Laminaria," "Chigaiso," "Aname," and "Sujime," while the algae species that primarily make up "Eisenia brasiliensis" beds are, for example, "Eisenia brasiliensis," "Sagarame," "Ecklonia cava," "Kurome," and "Arame japonica." The algae species that primarily make up "Wakame beds" are, for example, "Undaria pinnatifida" and "Hirome," while the algae species that primarily make up "Gengusa beds" are, for example, "Makusa," "Obosa maxima," and "Obakusa." Note that plants that can photosynthesize in the sea belong to seaweed.

Below, an overview of each component of the seaweed bed observation system S is provided. The surface drone 3 is an example of an aquatic vehicle (or navigation vehicle) that can move on water. The movement of the surface drone 3 is controlled by a wireless or wired controller from, for example, a vessel such as a boat located on the sea, a structure installed on the sea, a natural object on the sea such as a reef, or from the ground.

The underwater camera 2 is connected, for example, via a communication cable to a computer (for convenience, referred to as a drone computer; not shown in Figure 1) mounted on the surface drone 3, and moves underwater as the surface drone 3 moves (navigates).

Furthermore, the underwater camera 2's movement or posture in the sea can be controlled, for example, in response to control from a drone computer via a communication cable, allowing it to capture images of underwater seaweed from multiple different imaging directions. For example, the underwater camera 2 generates captured image data by capturing images of underwater seaweed from one or more directions other than above the seaweed.

Here, a direction different from above the seaweed refers to, for example, a direction excluding the direction in which the seaweed is imaged from directly above in the water depth direction, and is an imaging direction in which the side of the seaweed fits within the angle of view of the underwater camera 2. In other words, the imaging direction of the underwater camera 2 is set or controlled so that the length of the seaweed in the water depth direction can be identified from the captured image data. The captured image data may be either moving image data or still image data.

Image data captured by the underwater camera 2 is transmitted to a drone computer via a communication cable, for example, and stored in the drone computer's storage device. In addition, when photographing seaweed, the drone computer records in the storage device, for example, the coordinates of the photographing location, the photographing direction of the underwater camera 2 (for example, the angle between the water depth direction and the photographing direction), the water depth, and the distance between the sea surface and the underwater camera 2.

The coordinates of the shooting location are determined, for example, based on location information acquired by a drone computer or a GPS device mounted on the surface drone 3. The shooting direction of the underwater camera 2 is acquired, for example, by the underwater camera 2 itself, or as data obtained when the drone computer controls the attitude of the underwater camera 2 when shooting with the underwater camera 2.

The water depth is obtained, for example, by a water depth sensor attached to the underwater camera 2. The distance between the sea surface and the underwater camera 2 is obtained, for example, by the difference between the water depth obtained by the water depth sensor and the sea surface. Note that the distance between the sea surface and the underwater camera 2 may also be obtained based on the length of the communication cable equivalent to the distance the underwater camera 2 is lowered into the sea from the surface drone 3.

In the above-described seaweed bed observation system S, images of seaweed living in the sea are taken using an underwater camera 2 submerged in the sea from a surface drone 3 located above the sea. However, images of seaweed living in the sea may also be taken using a camera (which may be a built-in camera or an external camera) attached to an underwater vehicle that can move underwater, such as an underwater drone.

Furthermore, the underwater camera 2 in the seaweed bed observation system S is not limited to one, and may be equipped with two or more underwater cameras 2. For example, the underwater positions and attitudes of two or more underwater cameras 2 may be individually controlled, allowing captured image data with multiple different angles of view to be acquired at the same time.

<Outline of processing by the seaweed bed observation system S>
Fig. 3 is a diagram showing a schematic overview of the processing performed by the seaweed bed observation system S. In the seaweed bed observation system S shown in Fig. 1, observation data including captured image data of seaweeds living in the sea is acquired.

For example, as shown in the dotted frame A1, one or more underwater cameras 2 capture images of underwater seaweed from multiple different imaging directions to obtain image data. The coordinates of the imaging location, the imaging direction (angle) of the underwater camera 2, the water depth, and the distance between the sea surface and the underwater camera 2 are also obtained.

The acquired observation data is input into the data processing device 1, which will be described later with reference to Figure 4. As shown in the dotted frame A2 in Figure 3, the data processing device 1 generates a seaweed shape model by modeling the shape of the seaweed in the captured image data in 3D or 2D based on the observation data. For convenience, the shape of the seaweed modeled by the data processing device 1 based on the captured image data may be referred to as the "seaweed shape model."

The data processing device 1 also derives the length of the seaweed shape model based on, for example, the water depth at the location where the seaweed was captured and the distance between the underwater camera 2 and the sea surface, and derives the surface area of the seaweed shape model based on the derived length. Additionally, the data processing device 1 identifies the type of seaweed in the captured image data, for example, using a machine learning model that has learned about the type of seaweed. The data processing device 1 also acquires data (in units of, for example, "lux") that quantifies the brightness (in other words, light intensity) according to the water depth at the location where the seaweed was captured.

The brightness corresponding to the water depth at the location where the seaweed was photographed may be detected and quantified, for example, by a measuring device such as an optical sensor attached to the underwater camera 2, or it may be detected and quantified by changes in pixel brightness values in the water depth direction during image analysis of the captured image data of the seaweed.

The brightness of light received by seaweed in the ocean can vary not only with water depth but also with the turbidity of the ocean. By detecting the brightness according to the water depth where the seaweed was photographed as described above, the accuracy of estimating the amount of _CO2 absorbed by seaweed can be improved compared to when using theoretical values for brightness in the water depth direction.

Since the captured image data may contain multiple types of seaweed, the data processing device 1 determines the type of seaweed, for example, based on the degree of similarity between the image data of each type of seaweed and the image data of the seaweed contained in the captured image data. The data processing device 1 may also identify the type of seaweed contained in the captured image data using a machine learning model that has learned about types of seaweed.

Next, the data processing device 1 divides the modeled seaweed shape model into multiple blocks (four blocks #1 to #4 in the example of Figure 3) in the water depth direction (in other words, in the length direction of the seaweed), as shown in the dotted frame A3 in Figure 3.

The data processing device 1 calculates the amount of _CO2 absorption for each of the divided blocks by multiplying the surface area (e.g., in ^m2 ) by the CO2 absorption coefficient (e.g., in t- _CO2 / ^m2 ) corresponding to the brightness depending on the water depth.The data processing device 1 then calculates the amount of _CO2 absorption for the entire surface area of the seaweed imaged by adding up _{the CO2} absorption amounts for each block.

Regarding the correspondence relationship between the brightness of the ocean according to the water depth and the _CO2 absorption coefficient, for example, a database showing the correspondence between the brightness of the ocean according to the water depth and the _CO2 absorption coefficient is created in advance by collecting samples, etc., and stored in the memory unit 13 of the data processing device 1 (described later with reference to Figure 4).

As shown in the dotted frame A3, a database is created for each seaweed type #1 to #m (m is an integer equal to or greater than 2). The data processing device 1 calculates the _CO2 absorption amount using the CO2 absorption coefficient corresponding to the brightness of each block #i (i is any one of 1 to m) in the database corresponding to the seaweed type identified by the machine learning model. Hereinafter, the database showing _the correspondence between brightness and _CO2 absorption coefficient may be referred to as "brightness vs. _CO2 absorption coefficient data." The correspondence between brightness and _CO2 absorption coefficient according to water depth may also be expressed by a mathematical formula.

<Configuration example of data processing device 1>
Next, a configuration example of a data processing device 1 for realizing the above-described data processing will be described with reference to Fig. 4. Fig. 4 is a block diagram showing a configuration example of a data processing device 1 according to this embodiment. The data processing device 1 shown in Fig. 4 includes, for example, an interface (IF) 11, a processor 12, and a storage unit 13.

The interface 11 of the data processing device 1 is an input/output IF with an external device, and for example, observation data observed by the seaweed bed observation system S illustrated in Fig. 1 (e.g., captured image data stored in a storage device of a drone computer) is input, and the input observation data is output to the processor 12. The IF 11 also outputs the processing results (e.g., _CO2 absorption amount) by the processor 12 to the external device.

The external device may be a drone computer or another computer. Because real-time performance is not required for the estimation of _{the CO2} absorption amount (which may be interchangeably read as "calculation" or "determination"), observation data obtained by the drone computer may be input from another computer or storage medium to the data processing device 1 via IF11. Furthermore, the _CO2 absorption amount, which is the processing result of the data processing device 1, is output to, for example, a display device or printer as an external device via IF11 and displayed or printed.

Non-limiting examples of data items of observation data include image data of seaweed captured by underwater camera 2, coordinate data of the location where the image data was obtained, water depth data of the location where the image data was obtained, and angle data indicating the imaging direction of underwater camera 2 when the image data was obtained. The angle data is expressed, for example, by the angle formed between the water depth direction and the imaging direction of underwater camera 2, with the water depth direction as the reference. The image data of seaweed may include multiple image data pieces generated by capturing images of seaweed from multiple different directions, including a direction capturing the side of the seaweed.

The processor 12 estimates at least the amount of _CO2 absorption by seaweed based on, for example, observation data input from the IF 11, and outputs the estimation result to the IF 11. The processor 12 executes a program stored in the storage unit 13, thereby functioning as an observation data processing unit 121, a modeling unit 122, a _CO2 absorption amount estimation unit 123, and a setting reception unit 124.

The observation data processing unit 121 is an example of an acquisition unit that acquires captured image data including seaweed. It receives observation data including at least the captured image data input from IF11 and outputs the observation data to the modeling unit 122. The observation data processing unit 121 also acquires, for example, data relating to brightness according to the water depth at which seaweed is located as captured by the underwater camera 2 (hereinafter also referred to as "light intensity data") from the underwater camera 2. The observation data processing unit 121 may determine the brightness according to the water depth by analyzing the image data of the seaweed captured by the underwater camera 2. The light intensity data is, for example, data relating to the distribution (or change) of brightness according to the water depth.

The modeling unit 122, for example, performs image analysis on the captured image data input from the observation data processing unit 121 to model the shape of the seaweed, including its depth length and width. For example, the modeling unit 122 determines (or sets) whether to model the shape of the seaweed in 3D or 2D, depending on the result of the image analysis that determines whether the location where the captured image data was captured is a seaweed bed where seaweed grows in large numbers.

For this reason, the modeling unit 122 includes, for example, a 2D/3D setting unit 1221. For example, when the location where the captured image data was taken is not a seaweed bed (for example, when the captured image data contains a small number of seaweeds below a threshold), the 2D/3D setting unit 1221 models the shape of the seaweed in 3D (this may also be referred to as "three-dimensionalization"). 3D modeling can improve the modeling accuracy of the shape of the seaweed. On the other hand, when the location where the captured image data was taken is a seaweed bed, the 2D/3D setting unit 1221 models the shape of the seaweed or seaweed bed in 2D. 2D modeling can reduce the amount of calculation required for modeling.

Whether 3D or 2D modeling is applied may be determined based on the settings received from the setting reception unit 124, in addition to the result of determining whether the location where the captured image data was taken is a seaweed bed. A detailed example of the operation will be described later with reference to Figure 5.

The modeling unit 122 also includes a machine learning model 1222 to, for example, determine the type of seaweed contained in the captured image data. The machine learning model 1222 is a model that has learned the type of seaweed using, for example, image data of multiple types of seaweed as training data, and when image data is input, it outputs the type of seaweed that is more likely to be contained in the input image data than a reference value. By inputting captured image data that shows seaweed into the machine learning model 1222, the type of seaweed is identified and data indicating the seaweed species is output.

The _CO2 absorption amount estimating unit 123 estimates the _CO2 absorption amount of seaweed based on, for example, a seaweed shape model. For example, the _CO2 absorption amount estimating unit 123 estimates the CO2 absorption amount based on the surface area of the seaweed shape model, the depth-direction length of the seaweed in the underwater area, light intensity data according to the depth in the underwater area, and _{the CO2} absorption coefficient of the seaweed corresponding to the brightness according to the depth.

Here, the amount of _CO2 absorption is calculated by multiplying the surface area of the seaweed by the _CO2 absorption coefficient, but in this embodiment, the _CO2 absorption coefficient by which the surface area is multiplied varies depending on the brightness in the water depth direction. As a non-limiting example, multiple _CO2 absorption coefficients corresponding to multiple different brightness levels in the water depth direction are stored in the storage unit 13. The _CO2 absorption coefficients are stored in the storage unit 13 in the form of a table in which brightness levels are associated with _CO2 absorption coefficient data, for example.

The _CO2 absorption amount estimation unit 123 acquires the _CO2 absorption coefficient of brightness corresponding to the water depth of each block, which is a block obtained by dividing the shape of the seaweed shape model into a plurality of blocks, from the storage unit 13. For this purpose, the _CO2 absorption amount estimation unit 123 includes a block division unit 1231 that divides the seaweed shape model into a plurality of blocks in the length direction, as shown in Fig. 3 .

The number of blocks divided by the block dividing unit 1231 may be fixed or variable. If it is variable, the block dividing unit 1231 determines the number of blocks to divide based on, for example, the length of the seaweed shape model. If there are too many blocks to divide (in other words, if the block size is too small), the amount of calculation increases, so the block dividing unit 1231 reduces the number of blocks to divide, for example, as the length of the seaweed shape model becomes longer.

Furthermore, if the number of divisional blocks is too small (in other words, if the size of the divisional blocks is too large), when there is a change in brightness along the water depth, the difference in photosynthetic activity corresponding to that change is not reflected in the calculated _CO2 absorption amount, which is likely to result in large errors.

Therefore, the block dividing unit 1231 determines the number of blocks into which the seaweed shape model is divided, for example, based on data on the distribution of brightness according to water depth. As a non-limiting example, the block dividing unit 1231 determines the size of the blocks into which the seaweed shape model is divided, so that the greater the degree of change indicated by the distribution of brightness according to water depth, the smaller the size of the blocks into which the seaweed shape model is divided. This improves the accuracy of the _CO2 absorption amount calculated for each block compared to when the block size is constant.

The block dividing unit 1231 may determine the number of blocks by prioritizing data on brightness according to water depth over the length of the seaweed shape model. For example, if the length of the seaweed shape model is short but the brightness of the seaweed shape model varies greatly according to water depth, the block dividing unit 1231 may increase the number of blocks divided into areas of the seaweed shape model where the degree of brightness variation according to water depth is greater, regardless of the length of the seaweed shape model. This reduces errors in the _CO2 absorption amount calculated per block, improving accuracy.

The _CO2 absorption coefficient of seaweed may vary depending on the type of seaweed. Therefore, the _CO2 absorption amount estimation unit 123 uses, for example, a _CO2 absorption coefficient corresponding to the algae species to estimate the _CO2 absorption amount. For example, as shown in the dotted frame A3 in Fig. 3 , a plurality of "brightness vs. _CO2 absorption coefficient data" showing the correspondence between brightness and _CO2 absorption amount according to water depth for each algae species is stored in the storage unit 13. The _CO2 absorption amount estimation unit 123 uses the _CO2 absorption coefficient from the "brightness vs. _CO2 absorption coefficient data" corresponding to the algae species identified by the machine learning model 1222 to estimate the _CO2 absorption amount.

The setting reception unit 124 receives 2D modeling or 3D modeling settings, for example, via IF11, and sets the received settings in the modeling unit 122 (2D/3D setting unit 1221).

The storage unit 13 stores the various data described above, such as observation data, a threshold value used to determine whether the location where the captured image data was captured is a seaweed bed, light intensity data in the water depth direction, and "brightness versus _CO2 absorption coefficient data" for each algae species. One or both of the threshold value and the "brightness versus _CO2 absorption coefficient data" can be set or updated as appropriate via, for example, the IF 11 and the processor 12.

In the above-described exemplary configuration, the machine learning model 1222 is provided in the modeling unit 122, for example, but may be provided in the observation data processing unit 121 or the _CO2 absorption amount estimation unit 123. Alternatively, the machine learning model 1222 may be provided in any location within the data processing device 1 or any location within the processor 12.

The functions of the data processing device 1 described above (for example, various functions of the processor 12) may be realized by a plurality of data processing devices 1. For example, various processes such as shape modeling of seaweed and estimating the amount of _CO2 absorption for each block obtained by dividing the modeled shape into a plurality of blocks may be performed in a distributed manner by a plurality of data processing devices 1 (for example, processors mounted on each of a plurality of servers).

<Computer program for realizing data processing device 1>
The various functions (blocks) of the data processing device 1 described above can be realized by a computer executing a program. For example, the computer includes an input device such as a keyboard, a mouse, or a touchpad, an output device such as a display or a speaker, a central processing unit (CPU), a graphics processing unit (GPU), a storage device such as a read-only memory (ROM), a random access memory (RAM), a hard disk drive or a solid-state drive (SSD), a reading device that reads information from a recording medium such as a digital versatile disk read-only memory (DVD-ROM) or a universal serial bus (USB) memory, and a transmitting/receiving device that communicates via a network.

The reading device then reads the data processing program for realizing the functions of the above-mentioned data processing device 1 from a recording medium on which the program is recorded, and stores the program in a storage device. Alternatively, the transmitting/receiving device communicates with a server device connected to the network, and stores the data processing program downloaded from the server device in a storage device.

The CPU then copies the data processing program stored in the storage device to the RAM, and reads and executes the instructions contained in the data processing program, thereby realizing the functions of the data processing device 1. Note that the data processing program may also be incorporated into the computer's operating system (OS) as part of the OS.

<Example of operation of data processing device 1>
Fig. 5 is a flowchart showing an example of data processing by the data processing device 1 according to the embodiment. The flowchart shown in Fig. 5 is started in response to the start-up of the data processing device 1 or in response to the start-up of a program or software that realizes the processing (S11 to S21) shown in Fig. 5.

As shown in FIG. 5, the data processing device 1 acquires observation data including at least image data of underwater seaweed captured from multiple different imaging directions by the processor 12 (e.g., the observation data processing unit 121) via the IF 11 (S11).

The observation data processing unit 121 outputs the acquired observation data to the modeling unit 122. The modeling unit 122 checks whether a setting has been made to apply 3D modeling or 2D modeling to the shape modeling of the seaweed based on the captured image data of the seaweed (S12).

If the setting is found to be valid (YES in S12), the modeling unit 122 checks whether the setting is valid for applying 3D modeling (S13). If the setting is valid for applying 3D modeling (YES in S13), the modeling unit 122 models the shape of the seaweed using 3D modeling based on the captured image data of the seaweed (S15).

On the other hand, if 3D modeling is not set to be applied in S13 (NO in S13), the modeling unit 122 applies 2D modeling to the modeling and models the shape of the seaweed in 2D (S16).

Furthermore, if it is determined in S12 that no setting has been made as to whether 3D modeling or 2D modeling is to be applied (NO in S12), the modeling unit 122 determines whether the captured image data including seaweed is image data of a seaweed bed where seaweed grows in large numbers (S14).

If the result of the determination is that the captured image data is image data of a seaweed bed (YES in S14), the modeling unit 122 performs 2D modeling of the shape of the seaweed to reduce the amount of calculation (S16). On the other hand, if the captured image data is not image data of a seaweed bed (NO in S14), the modeling unit 122 performs 3D modeling of the shape of the seaweed to improve the accuracy of estimating the amount of _CO2 absorption (S15).

The modeling unit 122 also identifies the type of seaweed (algae species) captured in the captured image data using, for example, a machine learning model 1222 that has learned about types of seaweed. Data indicating the identified algae species is output to the _CO2 absorption amount estimation unit 123.

The _CO2 absorption estimation unit 123 divides the shape of the seaweed modeled in 3D or 2D (seaweed shape model) into multiple blocks along the length (water depth direction) using the block division unit 1231, as illustrated in the dotted frame A3 in Figure 3 (S17).

The _CO2 absorption estimation unit 123 calculates the surface area of each divided block, and also accesses the memory unit 13 to obtain the _CO2 absorption coefficient corresponding to the brightness according to the water depth of each block from the ``brightness vs. _CO2 absorption coefficient data'' corresponding to the algae species identified by the machine learning model 1222.

Furthermore, the _CO2 absorption amount estimation unit 123 determines the CO2 absorption amount of each block by multiplying the surface area by the _CO2 absorption coefficient corresponding to the water depth and algae species (S18).The _{CO2 absorption amount estimation unit 123 then calculates the CO2 absorption} amount of the seaweed by adding up the _CO2 absorption amounts of each block (S19), and outputs data indicating _the calculated _CO2 absorption amount to an external device (e.g., a display device) via IF11 (S20).

Then, the processor 12 of the data processing device 1 monitors whether or not other observation data has been input (S21). If other observation data has been input (YES in S21), the processor 12 repeats the processes of S11 to S20 until no more observation data is input (until a NO determination is made in S21).

If the processor 12 determines that no other observation data has been input (NO in S21), the processor 12 ends processing of the observation data.

As described above, the data processing device 1 performs image analysis on image data of underwater seaweed captured by the underwater camera 2 from multiple imaging directions, including directions other than above the seaweed, to model the shape of the seaweed, including its depthwise length and width. The data processing device 1 then calculates the amount of _CO2 absorption by the seaweed based on the modeled shape of the seaweed. This allows for more accurate calculation of the amount of _CO2 absorption by the seaweed than when the depthwise length of the seaweed is not taken into account, resulting in improved reliability of the calculated amount of _CO2 absorption.

Furthermore, the data processing device 1 calculates the CO2 absorption coefficient of the modeled seaweed using the _CO2 absorption coefficient corresponding to the underwater brightness depending on the water depth, thereby further improving the accuracy in calculating the amount _{of CO2} absorption by seaweed compared to when the underwater brightness depending on the water depth is not taken into account.

Therefore, this method can provide evidence that seaweed (or seaweed beds) absorb a greater amount of _CO2 than if the brightness corresponding to the seaweed's depth is not taken into account, contributing to the spread of blue carbon credits. Furthermore, since the measurement method is published in Non-Patent Document 1, the adoption of the blue carbon credit system by many organizations, such as local governments and companies, is expected to lead to the expansion of solutions businesses related to seaweed observation and an increase in licensing revenue. Furthermore, for example, by collecting and analyzing seaweed observation data from various locations, it is possible to predict the growth status of seaweed or seaweed beds. This can also contribute to predicting credit prices and providing financial products such as insurance.

In the above-described embodiment, the case where seaweed (or seaweed beds) living in the ocean are the object of observation has been described, but seaweed (or seaweed beds) living in freshwater or brackish water such as lakes, ponds, and rivers may also be the object of observation.

Furthermore, the term "~ unit" used in the configurations illustrated in the above-mentioned embodiments may be interchangeably read as other terms, such as "~ means," "~ circuit," or "~ device."

Furthermore, this invention will make it possible to contribute to Goal 9 "Build resilient infrastructure, promote inclusive and sustainable industrialization," and Goal 17 "Protect and sustainably use the water below sea level" of the United Nations' Sustainable Development Goals (SDGs).

The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and alterations are possible within the spirit of the invention. For example, all or part of the device can be configured by functionally or physically distributing or integrating any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of new embodiments resulting from the combination will also have the effects of the original embodiments.

S Seaweed bed observation system 1 Data processing device 2 Underwater camera 3 Surface drone 11 IF
12 Processor 13 Memory unit 121 Observation data processing unit 122 Modeling unit 123 _CO2 absorption amount estimation unit 124 Setting reception unit 1221 2D/3D setting unit 1222 Machine learning model 1231 Block division unit

Claims (13)

an acquisition unit that acquires captured image data generated by capturing images of underwater seaweed from a direction different from above the seaweed using an underwater camera;
a modeling unit that models a shape including a depth direction length and a width of the seaweed by performing image analysis on the captured image data;
An estimation unit that estimates the amount of carbon dioxide absorbed by the seaweed based on the modeled shape;
and
The estimation unit estimates the carbon dioxide absorption amount based on a surface area of the shape and a carbon dioxide absorption coefficient of the seaweed corresponding to brightness according to the water depth at which the seaweed is located as imaged by the underwater camera.
Data processing device. an acquisition unit that acquires captured image data generated by capturing images of underwater seaweed from a direction different from above the seaweed using an underwater camera;
a modeling unit that models a shape including a depth direction length and a width of the seaweed by performing image analysis on the captured image data;
An estimation unit that estimates the amount of carbon dioxide absorbed by the seaweed based on the modeled shape;
and
The estimation unit divides the shape into a plurality of blocks in the longitudinal direction, and estimates the carbon dioxide absorption amount for each block using the carbon dioxide absorption coefficient of the seaweed corresponding to the brightness according to the water depth at which the block is located.
Data processing device. an acquisition unit that acquires captured image data generated by capturing images of underwater seaweed from a direction different from above the seaweed using an underwater camera;
a modeling unit that models a shape including a depth direction length and a width of the seaweed by performing image analysis on the captured image data;
An estimation unit that estimates the amount of carbon dioxide absorbed by the seaweed based on the modeled shape;
and
the modeling unit inputs the captured image data into a machine learning model that, when image data is input, outputs a type of seaweed that is more likely to be included in the input image data than a reference value, thereby identifying the type of seaweed;
The estimation unit estimates the carbon dioxide absorption amount using a carbon dioxide absorption coefficient corresponding to the identified type of seaweed.
Data processing device. an acquisition unit that acquires captured image data generated by capturing images of underwater seaweed from a direction different from above the seaweed using an underwater camera;
a modeling unit that models a shape including a depth direction length and a width of the seaweed by performing image analysis on the captured image data;
An estimation unit that estimates the amount of carbon dioxide absorbed by the seaweed based on the modeled shape;
and
The captured image data includes a plurality of captured image data generated by capturing images of the seaweed from a plurality of different directions, including a direction capturing an image of a side surface of the seaweed,
the modeling unit determines whether to apply three-dimensional modeling or two-dimensional modeling to model the shape of the seaweed, depending on the result of determining whether the seaweed is a seaweed bed in which the seaweed grows in colonies in the image analysis.
Data processing device. The captured image data includes a plurality of captured image data generated by capturing images of the seaweed from a plurality of different directions, including a direction capturing an image of a side surface of the seaweed,
the modeling unit performs three-dimensional modeling of the shape based on the plurality of captured image data.
5. A data processing device according to claim 1. the estimation unit determines the number of blocks into which the shape is divided based on the length of the shape in the water depth direction.
3. The data processing device according to claim 2 . the estimation unit determines the number of blocks to divide the shape into based on data on brightness distribution according to water depth.
3. The data processing device according to claim 2 . The estimation unit determines a larger number of blocks to divide the shape in a longitudinal direction in an area where the degree of change in brightness indicated by the distribution of brightness according to the water depth is greater.
8. A data processing device according to claim 7. An underwater camera that captures images of underwater seaweed,
A data processing device according to any one of claims 1 to 4 ;
A data processing system comprising: A computer functioning as a data processing device,
Acquiring captured image data generated by capturing images of underwater seaweed from a direction different from above the seaweed using an underwater camera;
Modeling a shape of the seaweed including a depth direction length and a width of the seaweed by performing image analysis on the captured image data;
The amount of carbon dioxide absorbed by the seaweed is estimated based on the surface area of the modeled shape and the carbon dioxide absorption coefficient of the seaweed corresponding to the brightness depending on the water depth at which the seaweed is located as imaged by the underwater camera.
Data processing methods. A computer functioning as a data processing device,
Acquiring captured image data generated by capturing images of underwater seaweed from a direction different from above the seaweed using an underwater camera;
Modeling a shape of the seaweed including a depth direction length and a width of the seaweed by performing image analysis on the captured image data;
The modeled shape is divided into a plurality of blocks in the longitudinal direction, and for each block, the carbon dioxide absorption amount of the seaweed is estimated using the carbon dioxide absorption coefficient of the seaweed corresponding to the brightness according to the water depth at which the block is located.
Data processing methods. A computer functioning as a data processing device,
Acquiring captured image data generated by capturing images of underwater seaweed from a direction different from above the seaweed using an underwater camera;
Modeling a shape of the seaweed including a depth direction length and a width of the seaweed by performing image analysis on the captured image data;
Identifying the type of seaweed by inputting the captured image data into a machine learning model that, when image data is input, outputs a type of seaweed that is more likely to be included in the input image data than a reference value;
Estimating the amount of carbon dioxide absorption by the seaweed based on the modeled shape and the carbon dioxide absorption coefficient corresponding to the identified type of seaweed.
Data processing methods. A computer functioning as a data processing device,
Acquire a plurality of pieces of image data generated by capturing images of the underwater seaweed from a plurality of different directions using an underwater camera, the direction being different from above the seaweed, including a direction capturing an image of a side of the seaweed;
The captured image data is analyzed to determine whether the seaweed is a seaweed bed where the seaweed grows in colonies. Depending on the result, it is determined whether to apply three-dimensional modeling or two-dimensional modeling to model the shape of the seaweed, and a shape including the length of the seaweed in the water depth direction and the width of the seaweed is modeled.
Estimating the amount of carbon dioxide absorbed by the seaweed based on the modeled shape;
Data processing methods.