JP7489249B2 - DATABASE SYSTEM, DATA DISTRIBUTION MANAGEMENT DEVICE, AND DATA DISTRIBUTION MANAGEMENT METHOD - Google Patents

Description

The present invention relates to a database system, a data deployment management device, and a data deployment management method.

...

Patent Application No. 2019-511705

According to Patent Document 1, it is possible to determine that the degree of clustering has fallen below a clustering threshold, and in response to an explicit user command from a user and/or one or more portions of a DML command, re-cluster one or more partitions of the table to improve the degree of clustering of the table. However, in order to perform appropriate clustering, the user needs to specify a key, and specialized skills are required to achieve good performance.

The present invention aims to provide a physical design for a database that achieves good performance without relying on the user's skills.

In order to achieve the above-mentioned objective, one representative database system of the present invention comprises a catalog management server that manages a data catalog indicating the location of data, a storage node, one or more computing nodes that form a distributed database, and a management node that manages data deployment by causing the computing nodes to retain a portion of the data stored in the storage nodes, and when the management node receives a deployment request for the data deployment, it retrieves information from the data catalog and determines, based on the retrieved information, which computing node should retain the deployment target data, which is the data to be deployed, thereby performing a physical design of the distributed database.
Furthermore, one representative data deployment management device of the present invention is a data deployment management device that manages data deployment by replicating a portion of data stored in a storage node and storing it on one or more computing nodes that form a distributed database, and includes a deployment request receiving unit that receives a deployment request for the data deployment, a catalog information acquisition unit that, when the deployment request is received, acquires information from a data catalog that indicates the location of the data, and a physical design processing unit that performs physical design of the distributed database by determining, based on the information acquired from the data catalog, which computing node should store deployment target data, which is the data to be deployed.
Furthermore, one representative data deployment management method of the present invention is a data deployment management method that manages data deployment by replicating a portion of data stored in a storage node and storing the replicated data in one or more computing nodes that form a distributed database, and includes a deployment request receiving step in which a processor receives a deployment request for the data deployment, a catalog information acquisition step in which, upon receiving the deployment request, information is acquired from a data catalog indicating the location of the data, and a physical design processing step in which a physical design of the distributed database is performed by determining, based on the information acquired from the data catalog, in which computing node the deployment target data, which is the data to be deployed, should be stored.

The present invention makes it possible to carry out a physical design of a database that achieves good performance. Problems, configurations, and advantages other than those mentioned above will become clear from the description of the embodiment below.

FIG. 1 is an explanatory diagram of a database system according to an embodiment. FIG. 2 is a diagram showing the configuration of a management node. Example of database system configuration (part 1). Example of database system configuration (part 2). FIG. 13 is an explanatory diagram of the input/output processing amount of a computation node. FIG. 13 is an explanatory diagram of a data placement that reduces the input/output processing amount of a computing node. FIG. 13 is a diagram illustrating the amount of data transfer between computing nodes. FIG. 1 is an explanatory diagram of a data placement that reduces the amount of data transfer between computation nodes. FIG. 1 is a diagram illustrating a bias in the amount of processing among computing nodes. FIG. 13 is an explanatory diagram of a data placement that reduces the imbalance in the amount of processing between computing nodes. FIG. 1 is an explanatory diagram of data duplication. FIG. 1 is an explanatory diagram of data placement to avoid data duplication. FIG. 11 is an explanatory diagram of a specific example of data. 11 is a flowchart showing a processing procedure of a management node. 15 is a flowchart showing details of the determination of the distribution method shown in FIG. 14 . A concrete example of combined data deployment. A concrete example of data deployment using existing data. This is the sequence of operations when building a new distributed database. The sequence of operations when adding data to a distributed database. A modified example of additional data deployment.

The following describes an embodiment of the present invention with reference to the drawings. Note that the embodiment described below does not limit the invention as claimed, and not all of the elements and combinations thereof described in the embodiment are necessarily essential to the solution of the invention.

In the following explanation, information that produces an output for an input may be described using expressions such as "xxx table," but this information may be data of any structure. Therefore, a "xxx table" can also be called "xxx information."

In addition, in the following description, the configuration of each table is an example, and one table may be divided into two or more tables, or all or part of two or more tables may be one table.

In the following explanation, the processing may be described with the "program" as the subject, but since the program is executed by the processor unit to perform the specified processing while appropriately using the memory unit and/or interface unit, the subject of the processing may be the processor unit (or a device such as a controller that has the processor unit).

The program may be installed on a device such as a computer, or may be, for example, on a program distribution server or a computer-readable (e.g., non-transitory) recording medium. In addition, in the following description, two or more programs may be realized as one program, and one program may be realized as two or more programs.

The "processor unit" is one or more processors. The processor is typically a microprocessor such as a CPU (Central Processing Unit), but may be other types of processors such as a GPU (Graphics Processing Unit). The processor may be a single-core or multi-core processor. The processor may also be a processor in the broader sense, such as a hardware circuit that performs part or all of the processing (for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)).

In addition, in the following description, identification numbers are used as identification information for various objects, but other types of identification information (e.g., identifiers including letters or codes) may also be used.

In the following description, when elements of the same type are described without distinction, reference signs (or common signs among reference signs) are used, and when elements of the same type are described with distinction, the identification numbers (or reference signs) of the elements are used.

Figure 1 is an explanatory diagram of a database system according to an embodiment. The database system shown in Figure 1 has a management node 10, a catalog management server 20, a storage node 31, and a distributed database 40.

The distributed database 40 is formed by including a DBMS (database management system) management node 41 and one or more computing nodes 42. The computing nodes 42 are cached computing nodes equipped with a database, and can hold a portion of the data stored in the storage node 31.

The catalog management server 20 is a server that holds and manages the data catalog 21. The data catalog 21 is metadata about the data held by the storage nodes 31 and the computing nodes 42. The details will be described later, but the data catalog 21 at least indicates the location of the data.

The management node 10 manages data deployment in which a portion of the data stored in the storage node 31 is held in the computation node 42. Specifically, when the management node 10 receives a deployment request for data deployment (1), it acquires information from the data catalog 21 (2). Based on the acquired information, the management node 10 determines which computation node 42 should hold the deployment target data, which is the data to be deployed, thereby performing physical design of the distributed database 40 (3). In other words, based on the location of existing data, the management node 10 determines the deployment destination of the deployment target data indicated by the deployment request so that the distributed database 40 will achieve good performance.

Figure 2 is a configuration diagram showing the configuration of the management node 10. As shown in Figure 2, the management node 10 has a CPU (Central Processing Unit) 11, a memory 12, a storage unit 13, an input/output unit 14, and a communication unit 15.

The input/output unit 14 is a user interface such as a display and a keyboard. The communication unit 15 is a communication interface that communicates with the catalog management server 20 and the distributed database 40.

The storage unit 13 is an auxiliary storage device composed of a hard disk drive, a flash memory, etc. The CPU 11 reads out the programs functioning as the deployment request reception unit 12a, the catalog information acquisition unit 12b, and the physical design processing unit 12c from the storage unit 13, and deploys them in the memory 12 as the main storage device and executes them.

The deployment request receiving unit 12a is a processing unit that receives a deployment request via the input/output unit 14 and the communication unit 15. The catalog information acquiring unit 12b is a processing unit that acquires information of the data catalog 21 via the communication unit 15. The physical design processing unit 12c is a processing unit that performs physical design of the distributed database 40 by determining which computing node 42 should hold the deployment target data based on the acquired information.

Figures 3 and 4 show examples of database system configurations. In Figure 3, a management node 10, a catalog management server 20, a storage node 31, and a distributed database 40 are connected via a specific network. Also, within the distributed database 40, a DBMS 41 and multiple computation nodes 42 are connected via a network.

A user of the distributed database 40 is, for example, a medical researcher. The medical researcher refers to the data catalog 21 in the catalog management server 20 and informs the operations manager of the data he or she wishes to use. The operations manager generates a deployment request based on the medical researcher's needs and inputs it to the management node 10, thereby constructing a distributed database in which data that satisfies the medical researcher's needs is deployed. At this time, the management node 10 determines the data distribution method for improving the performance of the distributed database, so the operations manager does not need to specify the placement of the data.

In FIG. 4, a management node 10, a catalog management server 20, a storage network 30, and multiple distributed databases 40 are connected via a specific network. The storage network 30 is formed by connecting multiple storage nodes 31 via a network, and functions as a single virtual storage.

The multiple distributed databases 40 are each formed by including a DBMS 41 and multiple computation nodes 42. The multiple distributed databases 40 are constructed individually for different users and are used independently. In this way, the database system of this embodiment can be implemented as a system including multiple distributed databases.

In FIG. 4, as in FIG. 3, the user of the distributed database 40 is, for example, a medical researcher. The medical researcher refers to the data catalog 21 in the catalog management server 20 and informs the operations manager of the data he or she wishes to use. The operations manager generates a deployment request based on the medical researcher's needs and inputs it to the management node 10, thereby constructing a distributed database in which data that meets the medical researcher's needs is deployed. As in FIG. 3, the operations manager does not need to specify the placement of the data.

Next, the physical design by the management node 10 will be explained. The management node 10 arranges data to avoid various bottlenecks that occur in a distributed database. Bottlenecks that occur in a distributed database include the input/output processing volume of a computation node, the amount of data transferred between computation nodes, imbalances in the amount of processing between computation nodes, and data duplication.

Figure 5 is an explanatory diagram of the input/output processing volume of a computation node. As shown in Figure 5, when data is stored in a memory area without being sorted, the range of data searched for becomes wider, and the input/output processing volume increases. Figure 5 shows a case where data with an id value of "1" is searched for from the medical record (SI) held by computation node C1, but since data with an id value of "1" may exist in any block, it is necessary to read all blocks.

Figure 6 is an explanatory diagram of a data arrangement that reduces the amount of input/output processing on a computing node. In Figure 6, data is stored in a clustered state based on the search key item. For this reason, for example, when searching for data with an id value of "1" from the medical record (SI), only block 1 needs to be read. In this way, by specifying clustering of column values, it is possible to reduce the amount of input/output processing on a computing node.

Figure 7 is an explanatory diagram of the amount of data transferred between calculation nodes. A calculation node 42 may combine multiple pieces of data for use. In this case, if the data to be combined is stored in a different calculation node 42, it becomes necessary to transfer the data to the other calculation node 42 when combining, which causes a decrease in performance. In Figure 7, an injury table (SY) and a medicine table (IY) are combined using the ID column, but calculation nodes C1 to C3 do not have the data they will combine and use, and instead receive data transferred from the other calculation nodes, which causes a decrease in performance due to the data transfer.

Figure 8 is an explanatory diagram of data placement that reduces the amount of data transferred between computation nodes. In Figure 8, the amount of data transferred between computation nodes is reduced by specifying that the data to be joined be placed on the same node. Specifically, as in Figure 7, the injury table (SY) and medicine table (IY) are used by joining them using the ID column, but since computation nodes C1 to C3 themselves have the data they will use for joining, they do not need to receive data transferred from other computation nodes. For convenience, this data distribution method in which joined data is placed on the same node is called "clustering by join column".

Figure 9 is an explanatory diagram of the imbalance in processing volume between computing nodes. First, if the amount of data is uneven between computing nodes, it will cause a decrease in performance. Also, if the master table referenced by all computing nodes is stored only in one computing node, communication will occur between the computing nodes, causing a decrease in performance.

Figure 9 shows a case where the medical records (SI) and the medical master (M) are used in combination. Here, if the computing node C1 holds more medical records (SI) than the computing nodes C2 to C3, it will take a long time to process the computing node C1, which has a large amount of data. In addition, since the computing node C3 holds the medical master (M) and all the computing nodes C1 to C3 access the computing node C3 where the master is located, the load on the computing node C3 where the master is located increases and communication occurs, resulting in processing delays.

Figure 10 is an explanatory diagram of data placement that reduces the bias in the amount of processing between the computing nodes. In Figure 10, the master is placed on all computing nodes C1 to C3, and the history table is placed as evenly as possible between the computing nodes using a round robin method or the like, thereby reducing the bias in the amount of processing between the computing nodes.

For convenience, the data distribution method in which the master is placed on all computing nodes is called "replicate to all nodes." Also, for convenience, the data distribution method in which the data is placed as evenly as possible among computing nodes using round robin is called "round robin."

Figure 11 is an explanatory diagram of data duplication. When a deployment request is made for data that includes some data that has already been deployed to a computing node, deploying all of the requested data again would result in unnecessary data transfer and capacity duplication.

In FIG. 11, medical chart (SI) ids = 1 to 20 have already been deployed for medical trainee U1. Of these, medical chart (SI) ids = 1 to 10 are stored in computing node C1, and medical chart (SI) ids = 11 to 20 are stored in computing node C2.

In this state, if medical researcher U2 requests deployment of SI ids = 11 to 30, deploying all of them will result in duplicate SI ids = 11 to 20, and the capacity will be wasted by transferring that data.

Figure 12 is an explanatory diagram of data placement that avoids data duplication. To avoid data duplication, the management node 10 checks whether data has been placed on the computing nodes using an appropriate distribution method. If the data has been placed using an appropriate distribution method, only the missing parts are placed. If the data has been placed but the distribution method is different, all of the requested data is placed.

In Figure 12, of the data already deployed for medical researcher U1, ids 11 to 20 of the medical chart (SI) are also available for medical researcher U2, and the remaining ids 21 to 30 of the medical chart (SI) are allocated and placed on compute nodes C1 and C2.

Figure 13 is an explanatory diagram of a specific example of data. The input information shown in Figure 13 is an example of information input as a deployment request. The input information for a deployment request may include the data name and the data usage method.

The data name is used to specify the name of the data stored in the storage node 31 or the computation node 42. When specifying data stored in the computation node 42, it is also possible to specify combined data generated by combining or extracted data generated by extracting.

A data usage method requests data merging or extraction. When requesting data merging, you specify the name of the merged data to be generated, the source data to be used in the merge, the merge conditions, the deployment destination, the user, etc. When requesting data extraction, you specify the name of the extracted data that is the extraction result, the source data from which it was extracted, the extraction conditions, the deployment destination, the user, etc.

In Figure 13, the input information is shown as "Usage: Injury/Illness_Medicine (SY_IY) = Combine (Injury/Illness claim (SY), Medicine claim (IY), Combine condition: SY.id = IY.id), Destination = C3, 4, User = U1" and "Usage: Medical claim (SI) (id = 11-30) = Extract (Medical claim (SI), Extract condition: 11 <= id <= 30), Destination = C1, 2, User = U3".

As shown in FIG. 13, the data catalog includes data list@storage node, deployed data@computation node, lineage information, data access rights, etc. The data list@storage node indicates the name and type of data stored in the storage node 31. The type may be "master" which is basic information or "history" which corresponds to transaction data. The deployed data@computation node indicates the name, placement destination, and distribution method of the data stored in the computation node 42.

Lineage information is data that indicates the origin of joined data and extracted data, and includes items such as data name, source data, joined columns, and extraction conditions. Data access rights are data that associates the name of data with users who have access rights. Information regarding this access right may be managed by a separate server that manages access rights.

Figure 14 is a flowchart showing the processing procedure of the management node 10. First, the deployment request receiving unit 12a receives a deployment request via the input/output unit 14 and the communication unit 15 (step S101). The physical design processing unit 12c determines a distribution method for the deployment target data indicated in the deployment request (step S102). After that, the physical design processing unit 12c determines whether the deployment target data has already been deployed, i.e., whether there is an overlap with existing data (step S103).

If the data to be deployed has already been deployed (step S103; Yes), the physical design processing unit 12c determines whether the distribution method of the data to be deployed and the deployed data is the same (step S104). If the distribution methods are the same (step S104; Yes), the physical design processing unit 12c deploys only the undeployed data to the computing nodes according to the determined distribution method (step S105).

If all of the deployment target data has not been deployed (step S103; No), or if the distribution method is different (step S104; No), the physical design processing unit 12c deploys the entire deployment target to the computing nodes according to the determined distribution method (step S106).

After step S105 or step S106, the physical design processing unit 12c updates the data catalog, sets the access rights (step S107), and ends the process.

Fig. 15 is a flow chart showing the details of the distribution method decision shown in Fig. 14. In this flow chart, the following steps are performed:
Step S201
When the process starts, the physical design processing unit 12c identifies input information of the deployment request, and the process proceeds to step S102.
Step S202
The physical design processing section 12c judges whether the input information is a data name or a method of use. If it is a data name, the process proceeds to step S203, and if it is a method of use, the process proceeds to step S206.

Step S203
The catalog information acquiring unit 12b acquires information about the input information from the data list of the data catalog 21, and proceeds to step S204.
Step S204
The physical design processing unit 12c determines the storage location of the data name. If the storage location is a data list@storage, the process proceeds to step S207, and if the storage location is a lineage, the process proceeds to step S205.
Step S205
The catalog information acquisition unit 12b acquires the lineage information from the data catalog 21, and proceeds to step S206.
Step S206
The physical design processing unit 12c obtains the source data and the join key from the lineage information, and proceeds to step S207.

Step S207
The physical design processing section 12c executes the processes of steps S208 to S212 for each source data, and then determines whether to end the loop in step S213.

Step S208
The physical design processing unit 12c judges the type of data. If it is combined data, the process proceeds to step S205. If it is history data, the process proceeds to step S210. If it is master data, the process proceeds to step S209.
Step S209
The physical design processing unit 12c sets the distribution method to "duplicate to all nodes" and proceeds to step S213.
Step S210
The physical design processing section 12c judges whether or not there is a connection between the histories. If there is a connection, the process proceeds to step S211, and if there is no connection, the process proceeds to step S212.
Step S211
The physical design processing unit 12c sets the distribution method to "clustering by join column" and proceeds to step S213.
Step S212
The physical design processing unit 12c sets the distribution method to "round robin" and proceeds to step S213.

Step S213
The physical design processing unit 12c determines whether all source data has been processed. If unprocessed source data remains, the process proceeds to step S207. If all source data has been processed, the process ends the determination of the distribution method and returns to the original process.

Next, we will explain the changes to the data catalog due to data deployment. Figure 16 is a specific example of the deployment of combined data. Figure 16 shows the state after data deployment has been performed on the data catalog shown in Figure 13 based on the input information "Usage: Injury/Illness_Medicine (SY_IY) = Join (Injury/Illness Claim (SY), Medicine Claim (IY), Join Condition: SY.id = IY.id), Deployment Destination = C3, 4, User = U1."

By deploying this combined data, injury/illness claim (SY) and pharmaceutical claim (IY) are added to the deployed data @ computation node in the data catalog. The deployment destinations for these are C3 and C4, and the distribution method is combined column clustering. In addition, injury/illness_medicine (SY_IY) is added to the lineage information. The source data for injury/illness_medicine (SY_IY) is injury/illness claim (SY) and pharmaceutical claim (IY), and the combined column is id. Furthermore, injury/illness claim (SY) and pharmaceutical claim (IY) are added to the access rights of the computation node data, and user U1 is associated with them.

Figure 17 is a specific example of data deployment using existing data. Figure 17 shows the state after data deployment has been performed on the data catalog shown in Figure 13 based on the input information "Usage: Medical Receipt (SI) (id = 11-30) = Extract (Medical Receipt (SI), Extraction Condition: 11 <= id <= 30), Deployment Destination = C1, 2, User = U3."

When deploying this data, as shown in Deployed Data @ Computation Node, since medical receipt (SI) (id = 1-20) has already been deployed, medical receipt (SI) (id = 21-30) is newly deployed. This deployment destination is C1 and C2, and the distribution method is round robin. In addition, medical receipt (SI) (id = 11-30) is added to the lineage information. The source data of medical receipt (SI) (id = 11-30) is medical receipt (SI), and the extraction condition is 11 <= id <= 30. The access rights of the computation node data are such that U3 is added to the existing user of medical receipt (SI) (id = 1-20), and new medical receipt (SI) (id = 11-30) is added and associated with user U3.

Next, the operational sequence of the database system will be explained. Figure 18 shows the operational sequence when constructing a new distributed database. As shown in Figure 18, first, the medical researcher requests a catalog search from the catalog management server 20. The catalog management server 20 executes the catalog search and returns data definitions and the like to the medical researcher.

The medical researcher then makes a data deployment request to the operations manager. Based on the request from the medical researcher, the operations manager inputs a DBMS deployment request to the management node 10. The management node 10 accepts the deployment request, requests catalog information from the catalog management server 20, and obtains lineage information and data type.

The management node 10 uses the lineage information and data type obtained from the catalog management server 20 to perform DBMS deployment and table definition creation, and requests the DBMS management node 41 to create a table. Once the DBMS management node 41 has created the table, the management node 10 requests the DBMS management node 41 to load data. Once the DBMS management node 41 has executed the data load, the management node 10 notifies the operations manager that deployment is complete.

In this way, during the DBMS deployment process, the management node 10 can request catalog information from the catalog management server 20, obtain lineage information and data types, and create table definitions.

Figure 19 shows the operation sequence when additional data is deployed to a distributed database. The operation sequence shown in Figure 19 differs from the operation sequence in Figure 18 in that the operations manager inputs a data deployment request instead of a DBMS deployment request, and there is no DBMS deployment by the management node 10; the rest of the operations are the same. That is, even in data deployment processing, the management node 10 can request catalog information from the catalog management server 20, obtain lineage information and data type, and create table definitions.

Figure 20 shows a modified example of additional deployment of data. In Figure 20, a medical researcher requests a catalog search from the catalog management server 20 to obtain data definitions, etc., and then requests the catalog management server 20 to deploy data. The catalog management server 20 provides lineage information and data type to the management node 10 based on the deployment request received from the medical researcher. Subsequent operations are similar to those in Figure 19, so a description will be omitted. In this way, a configuration in which the catalog management server 20 is responsible for receiving deployment requests can also be used.

As described above, the database system disclosed as an embodiment includes a catalog management server 20 that manages a data catalog 21 that indicates the location of data, a storage node 31, one or more computation nodes 42 that form a distributed database 40, and a management node 10 that manages data deployment that causes a portion of the data stored in the storage node 31 to be held in the computation node 42. When the management node 10 receives a deployment request for data deployment, it acquires information from the data catalog 21 and performs physical design of the distributed database 40 by determining which computation node 42 should hold the deployment target data, which is the data to be deployed, based on the acquired information. This makes it possible to perform physical design of a database that achieves good performance without relying on the skills of the user.

Furthermore, according to the above embodiment, the management node 10 determines the distribution method of the deployment target data based on the data name and/or data usage method indicated in the deployment request. Specifically, when the deployment target data is master data, the management node 10 adopts a distribution method in which the deployment target data is held by all the computing nodes included in the same distributed database. When the deployment target data is history data and is data obtained by combining multiple data, the management node 10 adopts a distribution method in which the multiple data related to the combination are held by the same computing node. When the deployment target data is history data and is not data obtained by combining multiple data, the management node 10 adopts a distribution method in which the deployment target data is distributed and held by multiple computing nodes.
In this way, by adopting a distribution method according to the data name and/or data usage method indicated in the deployment request, an appropriate data deployment can be automatically determined.

In addition, if the data to be deployed overlaps with data that has already been deployed and the distribution method of the data to be deployed is the same as that of the data that has already been deployed, the management node 10 selectively deploys the non-overlapping portion. In addition, when deploying the data to be deployed to the computing node, the management node 10 clusters the data to be deployed by items that serve as search keys and stores them. This makes it possible to streamline processing and improve performance.

The data catalog includes a list of data stored in the storage nodes, a list of data generated from the data stored in the storage nodes and stored in the computation nodes, and lineage information indicating how the data stored in the computation nodes was generated. The database system disclosed in the embodiment realizes the physical design of a distributed database by referring to and using such metadata.

The present invention is not limited to the above-described embodiment, but includes various modified examples. For example, the above-described embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to having all of the configurations described. Furthermore, the configurations may be replaced or added, rather than being deleted.

For example, the present invention is applicable to searches in any database, not just the databases shown as examples. In addition, the functions of the management node 10 may be implemented by having another device perform the functions.

10: Management node, 12a: Deployment request reception unit, 12b: Catalog information acquisition unit, 12c: Physical design processing unit, 20: Catalog management node, 21: Data catalog, 30: Storage network, 31: Storage node, 40: Distributed database, 41: DBMS management node, 42: Computation node

Claims (8)

A catalog management server that manages a data catalog indicating the location of data;
A storage node;
one or more computing nodes forming a distributed database;
a management node that manages data allocation for causing the computing node to hold a portion of the data stored in the storage node,
The management node,
When a deployment request for the data deployment is received, information is acquired from the data catalog, and a physical design of the distributed database is performed by determining which computing node should hold the deployment target data, which is the data to be deployed, based on the acquired information;
The management node determines a distribution method for the data to be deployed based on the data name and/or data usage method indicated in the deployment request, and when the data to be deployed is master data, adopts a distribution method in which the data to be deployed is held in all computing nodes included in the same distributed database.
A database system comprising: 2. The database system according to claim 1, wherein the management node adopts a distributed method in which, when the data to be deployed is history data and is data obtained by combining multiple data, the multiple data related to the combination are stored in the same calculation node. 2. The database system according to claim 1, wherein the management node adopts a distributed method in which, when the data to be deployed is history data and is not data that is a combination of multiple data, the data to be deployed is distributed to multiple calculation nodes and stored therein. 2. The database system according to claim 1, wherein the management node selectively deploys the non-overlapping portion if the data to be deployed overlaps with data already deployed and the distribution method of the data to be deployed is the same as the distribution method of the data already deployed. 2. The database system according to claim 1 , wherein the management node clusters the deployment target data by an item serving as a search key and stores the data when deploying the deployment target data to the computing nodes. The database system according to claim 1, characterized in that the data catalog includes a list of data stored in the storage node, a list of data generated from the data stored in the storage node and stored in the computation node, and lineage information indicating how the data stored in the computation node was generated. A data allocation management device that manages data allocation by replicating a portion of data stored in a storage node and storing the replicated data in one or more computing nodes that form a distributed database, comprising:
a deployment request receiving unit that receives a deployment request for the data deployment;
a catalog information acquisition unit that acquires information from a data catalog indicating a location of the data when the deployment request is received;
a physical design processing unit that performs a physical design of the distributed database by determining which computing node should hold deployment target data, which is data to be deployed, based on information acquired from the data catalog ;
The physical design processing unit determines a distribution method for the data to be deployed based on the data name and/or data usage method indicated in the deployment request, and when the data to be deployed is master data, adopts a distribution method in which the data to be deployed is held in all computing nodes included in the same distributed database.
A data deployment management device comprising: A data allocation management method for managing data allocation in which a part of data stored in a storage node is replicated and stored in one or more computing nodes forming a distributed database, the method comprising:
The processor:
a deployment request receiving step of receiving a deployment request for the data deployment;
a catalog information acquisition step of acquiring information from a data catalog indicating a location of the data when the deployment request is received;
a physical design processing step of performing a physical design of the distributed database by determining which computing node should hold the deployment target data, which is the data to be deployed, based on the information acquired from the data catalog ;
The physical design processing step determines a distribution method for the data to be deployed based on the data name and/or data usage method indicated in the deployment request, and adopts a distribution method in which the data to be deployed is held in all computing nodes included in the same distributed database when the data to be deployed is master data.
A data deployment management method comprising:

Images