JP6156380B2 - Distributed system, data processing method for information processing apparatus, and program - Google Patents

Description

  The present invention relates to a distributed system, a data processing method for an information processing apparatus, and a program.

  In recent years, analysis such as machine learning and data mining for large-scale data has been performed. MapReduce was proposed by Google as a framework for handling large-scale data, and the use of Hadoop, which is an open source implementation, has come to be used for data analysis. Yes.

  MapReduce operates as follows. First, it is assumed that a file on the distributed file system is used as an input of MapReduce. In this case, the file is split and its fragments exist on multiple machines. In MapReduce, the “map” function is executed by distributing the file fragments. The output of the “map” function is a key-value pair. The system groups key-value pairs generated from multiple machines by key. Therefore, communication is performed so that a key-value pair having the same key is transferred to the same machine. Then, using this as an input, the “reduce” function is executed. This is shown in FIG.

  It is known that various algorithms can be described by using MapReduce, but it is also known that the speed may be reduced in some cases. As an example of a case where the speed is low, there is a case where it is necessary to randomly access data shared by a plurality of machines from a map function or a reduce function.

  As an example in which such random access processing to shared data is necessary, there is an update of a “model” in machine learning. The model is data representing a learning result and is updated by input data (teacher data). The model data is updated many times with the input data until plausible model data is obtained. Here, the place where the model data is updated is random.

  MapReduce does not directly support data structures shared by multiple machines. Although it is conceivable to use a distributed file system used by MapReduce, random access is very slow and impractical.

  An example of a system aimed at solving this problem is described in Non-Patent Document 1. This system uses memcached, which is a general-purpose distributed memory cache system, as data shared by a plurality of machines, and attempts to solve this problem by using it in combination with Hadoop. Here, memcached is a data store in which a key-value pair is stored in a memory and a value can be subtracted from the key. By operating memcached on a plurality of machines and determining the range of keys assigned to each machine, an on-memory data store distributed on the plurality of machines is realized.

In Non-Patent Document 1, an attempt is made to solve the above-described problem by accessing memcached on a plurality of machines from a map function or a reduce function. This is shown in FIG.
Patent Document 1 describes an example of a distributed processing system realized by using Hadoop or the like, which is open source software that implements MapReduce.

JP 2012-58836 A

Jimmy Lin, 3 others, “Low-Latency, High-Throughput Access to Static Global Resources within the Hadoop Framework”, [online], 2009, HCIL Technical Report HCIL-2009-01, [May 25, 2012 Search], Internet <URL: http://hcil.cs.umd.edu/trs/2009-01/2009-01.pdf>

Non-Patent Document 1 uses memcached, but hereinafter, it will be referred to as a “distributed table” as a general name.
The method disclosed in Non-Patent Document 1 has a problem that the latency when accessing data on the distribution table is large. When acquiring values on other machines via the network, it takes a long time such as 1 ms. In this case, data can be accessed only 1000 times per second, and this data access becomes a bottleneck in the entire execution.

  In order to improve this, it is only necessary to read and write as many data as possible at one time. That is, for example, if you want to get the value corresponding to Key1 and Key2, if Key1 and Key2 are in charge of the same machine, instead of getting the value of Key2 after getting the value of Key1 , Request the value of both Key1 and Key2 to the machine at once. In this way, access to two pieces of data can be performed at a time, so the influence of latency can be reduced.

Such an improvement technique is also suggested in Non-Patent Document 1, but it is not described how to achieve this.
That is, in the system described in Non-Patent Document 1 described above, it is easy to combine a plurality of accesses appearing at the same place on the program into one, but appearing at different places on the program or in the MapReduce program ( There has been a problem that it is not easy to combine accesses across a plurality of map functions (operating on the same machine).

  An object of the present invention is to provide a distributed system, a data processing method for an information processing apparatus, and a program that solve the processing delay due to the influence of latency when accessing a distributed table, which is the above-described problem.

The distributed system of the present invention
Receiving means for receiving a key for read access processing for reading data from a plurality of distributed tables respectively possessed by a plurality of servers, and a callback for determining processing to be executed using a result read based on the key;
Registration means for waiting the read access process received and registering the key and the callback in an access waiting list as standby read access to be issued later to a corresponding server;
For the standby read access registered in the access waiting list, in accordance with a predetermined condition, a determination unit that determines an issue destination server that issues the standby read access and an issue timing;
Read access issuing means for issuing the standby read access to the server of the issue destination determined by the determining means at the determined issue timing and receiving the access result from the server;
Using the access result received by the read access issuing means, the callback of the corresponding standby read access registered in the access waiting list is executed, and the standby read access of the executed callback is waited for the access Callback executing means for deleting from the list.

The data processing method of the information processing apparatus of the present invention includes:
Information processing device
Receiving a key for read access processing for reading data from a plurality of distributed tables respectively possessed by a plurality of servers, and a callback for determining processing to be executed using a result read based on the key;
Waiting for the accepted read access processing, registering the key and the callback in the access waiting list as standby read access to be issued later to the corresponding server,
For the read access registered in the access waiting list, in accordance with a predetermined condition, determine a server that issues the standby read access and an issue timing,
Issuing the standby read access at the determined issue timing to the determined server, receiving the access result from the server,
Information processing for executing the callback of the corresponding standby read access registered in the access waiting list using the received access result and deleting the standby read access of the executed callback from the access waiting list A data processing method of the apparatus.

The computer program of the present invention is:
In a computer that implements an information processing device,
A procedure for receiving a key for read access processing for reading data from a plurality of distributed tables respectively possessed by a plurality of servers, and a callback for determining processing to be executed using a result read based on the key;
A procedure for causing the received read access process to wait and registering the key and the callback in an access waiting list as standby read access to be issued later to a corresponding server,
A procedure for determining a server that issues the standby read access and an issue timing for the standby read access registered in the access waiting list according to a predetermined condition;
A procedure of issuing the standby read access to the determined server at the determined issue timing and receiving the access result from the server;
Executing the callback of the corresponding standby read access registered in the access waiting list using the received access result, and deleting the standby read access of the executed callback from the access waiting list; Is a program for executing

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  The various components of the present invention do not necessarily have to be independent of each other. A plurality of components are formed as a single member, and a single component is formed of a plurality of members. It may be that a certain component is a part of another component, a part of a certain component overlaps with a part of another component, or the like.

  In addition, although a plurality of procedures are described in order in the data processing method and the computer program of the present invention, the described order does not limit the order in which the plurality of procedures are executed. For this reason, when implementing the data processing method and computer program of this invention, the order of the several procedure can be changed in the range which does not have trouble in content.

  Furthermore, the plurality of procedures of the data processing method and the computer program of the present invention are not limited to being executed at different timings. For this reason, another procedure may occur during the execution of a certain procedure, or some or all of the execution timing of a certain procedure and the execution timing of another procedure may overlap.

  According to the present invention, there are provided a distributed system, a data processing method for an information processing apparatus, and a program for reducing the influence of latency when accessing a distributed table and increasing the processing speed.

  The above-described object and other objects, features, and advantages will be further clarified by the preferred embodiments described below and the accompanying drawings.

It is a functional block diagram which shows the structure of the distributed system which concerns on embodiment of this invention. It is a figure which shows an example of the access waiting list | wrist of the distributed system which concerns on embodiment of this invention. It is a block diagram which shows the structure of the computer which implement | achieves the client apparatus of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows an example of operation | movement of the client apparatus in the distributed system which concerns on embodiment of this invention. It is a block diagram which shows one of the utilization forms of the distributed system which concerns on embodiment of this invention. It is a block diagram which shows one of the utilization forms of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows an example of the process sequence by the 1st method of the determination part of the client apparatus of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows an example of the process sequence by the 2nd method of the determination part of the client apparatus of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows an example of the process sequence by the 3rd method of the determination part of the client apparatus of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows an example of the process sequence by the 4th method of the determination part of the client apparatus of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows the other example of the process sequence by the 4th method of the determination part of the client apparatus of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows the other example of the process sequence by the 4th method of the determination part of the client apparatus of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows an example of the process sequence by the 5th method of the determination part of the client apparatus of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows an example of the process sequence by the 6th method of the determination part of the client apparatus of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows an example of the process sequence by the 7th method of the determination part of the client apparatus of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows the other example of the process sequence by the 7th method of the determination part of the client apparatus of the distributed system which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the distributed system which concerns on embodiment of this invention. It is a figure which shows the example when write access is additionally registered into the access waiting list of the distributed system which concerns on embodiment of this invention. It is a figure which shows the example when accumulative access is additionally registered into the access waiting list of the distributed system which concerns on embodiment of this invention. It is a figure which shows the example when the read access is additionally registered after the accumulative access to the access waiting list of the distributed system according to the embodiment of the present invention. It is a flowchart which shows an example of operation | movement of the read function execution part of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows an example of operation | movement of the write function execution part of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows an example of operation | movement of the accumulation function execution part of the distributed system which concerns on embodiment of this invention. It is a flowchart which shows an example of operation | movement of the read access issuing part of the distributed system which concerns on embodiment of this invention. It is a figure showing the operation | movement of MapReduce. 1 is a diagram illustrating a configuration of a system of Non-Patent Document 1. FIG. It is a functional block diagram which shows the logical structure of the information processing apparatus which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

(First embodiment)
FIG. 1 is a functional block diagram showing a configuration of a distributed system 1 according to an embodiment of the present invention.
The distributed system 1 according to the embodiment of the present invention includes a key for read access processing for reading data from a plurality of distributed tables (not shown) respectively included in a plurality of servers (distributed table server 10), and , A reception unit 102 that receives a callback that defines a process to be executed using a result read out based on the key (Value: Value), and waits for the received read access process, and causes the corresponding distributed table server 10 to The registration unit 106 for registering the key and callback in the access waiting list 104 as the standby read access issued in step 1, and the issue destination that issues the standby read access according to a predetermined condition for the standby read access registered in the access waiting list 104 Of determining the distributed table server 10 and the issue timing 108 and a read access issuing unit 110 that issues a standby read access to the distribution table server 10 of the issue destination determined by the determination unit 108 at the determined issue timing and receives the access result (value) from the distribution table server 10. Using the access result received by the read access issuing unit 110, the corresponding standby read access callback registered in the access waiting list 104 is executed, and the standby read access of the executed callback is deleted from the access waiting list 104. A callback execution unit 112.

  As shown in FIG. 1, in this embodiment, the distributed system 1 is a client connected to a plurality of distributed table servers 10 (in FIG. 1, three distributed table servers S1, S2, and S3) via a network 3. A device 100 is provided.

  The distributed system 1 of the present invention is a system that reads and writes values from and to a table that is distributed on a plurality of machines. In particular, it is a distributed table access system that can be accessed at high speed even when keys used for reading and writing are random. It can be suitably applied.

  The distributed system 1 of the present invention can be suitably applied to, for example, an apparatus that performs analysis such as machine learning or data mining on large-scale data. The distributed system 1 of the present invention can be applied to an application such as, for example, a system that handles large-scale data such as MapReduce, when the data processing unit needs shared data to be randomly accessed and speeds up the access. . FIG. 5 shows an example in which the client device of the present invention (shown as “distributed table client 92” in the figure) is applied to a system that uses a distributed table from the map function and reduce function of the MapReduce program executed by the data processing unit 90. Indicates.

The distributed system 1 of the present invention can also be applied to a Web application such as a Web server or an application (AP) server that uses a distributed table to cache the contents of a database. FIG. 6 shows a client apparatus (FIG. 6) when the Web server or AP server (shown as “Web server / AP server 94” in the figure) uses a distributed table to cache the contents of the database 96. In this example, “distributed table client 98” is applied.
The client device 100 of the present invention corresponds to the distributed table client (92 or 98) of FIG. 5 or FIG.

  Returning to FIG. 1, the client device 100 of the present embodiment is realized by a server computer or a personal computer connected to the distributed table server 10 via the network 3, or an information processing device equivalent thereto (computer 60 in FIG. 3). can do. The client device 100 may be configured by a virtual server or the like. In FIG. 1, only one client device 100 is shown. However, in the distributed system 1 of the present invention, a plurality of client devices 100 (not shown) can access the distributed table server 10.

Each component of the client device 100 of the distributed system 1 of this embodiment shown in FIG. 1 includes the components shown in FIG. 1 loaded in the CPU (Central Processing Unit) 62, RAM (Random Access Memory) 66, and RAM 66 shown in FIG. Arbitrary combination of hardware and software of an arbitrary computer 60 having a program 70 to be realized, a ROM (Read Only Memory) 64 for storing the program 70, and an I / O (Input / Output) 68 including an interface for network 3 connection It is realized by. The CPU 62 is connected to each element of the computer 60 via the bus 69 and controls the entire computer 60 together with each element. It will be understood by those skilled in the art that there are various modifications to the implementation method and apparatus. Each functional block diagram described below shows a block of logical functional units, not a configuration of hardware units. The computer 60 can also be connected to an input / output device (not shown) via the I / O 68.
Moreover, in each figure shown below, about the structure of the part which is not related to the essence of this invention, it has abbreviate | omitted and is not illustrated.

In the present embodiment, the distributed system 1 is assumed to be a cluster or grid system including a plurality of distributed table servers 10 installed in a data center or the like, for example.
In FIG. 1, the client device 100 includes a reception unit 102, an access waiting list 104, a registration unit 106, a determination unit 108, a read access issue unit 110, and a callback execution unit 112.

  The accepting unit 102 executes a process to be executed using a read access process key for reading data from a plurality of distributed tables (not shown) respectively included in the plurality of distributed table servers 10 and a result (value) read based on the key. Accept the specified callback.

For example, the access processing to the distributed table appears at a plurality of locations on one program text.
Data access on the distributed table has a large latency and causes performance degradation. In order to improve this, in the access processing to the distributed table, it is only necessary to read and write as many data as possible at one time. However, the access to the distributed table that appears in different places on the program is combined into one. Is not easy. Here, the program includes a program including access to a distributed table executed by the client apparatus 100, or a program executed on at least one other computer that accesses the distributed table via the client apparatus 100. it can.

In the present invention, a mechanism called a callback is used in order to consolidate accesses to a plurality of distributed tables appearing at different locations in the program. The callback is a mechanism for registering the subsequent processing as an object when it is desired to continue processing after performing some processing.
In the present invention, these access processes are collectively performed for each server that is an access destination, thereby achieving high speed.

  In the present invention, the read access process can include, for example, a process executed by an API (Application Program Interface) such as a read function that performs a process of reading data from a distributed table. In the present embodiment, a key for reading data from the distributed table and a callback for executing processing using the read result are designated as arguments in the read function. The accepting unit 102 accepts a key and a callback specified as arguments in the read function when the read function is called when the program is executed.

As shown in FIG. 2A, the access waiting list 104 includes a distributed table server 10 (responsible server) having a distributed table that holds data to be read access processed, and a key (Key) as standby read access. Corresponding and holding a set of callbacks.
In the access waiting list 104, identification information assigned in advance, an IP (Internet Protocol) address on the network 3, or the like can be held as information of the distributed table server 10 (server in charge).

  In the client device 100 of this embodiment, a method for searching for the distributed table server 10 in charge of a key can be performed using a general-purpose technique of a distributed system, and is not particularly limited. For example, a method for searching the distributed table server 10 is to obtain a distributed table server 10 by taking a hash value of a key and taking mod (the remainder obtained as a result of division), or a prepared response. The distributed table server 10 may be searched using the relationship.

  In the present embodiment, the client device 100 is configured to register a set of a key and a callback as standby read access for each server in charge (FIG. 2A). However, the present invention is not limited to this. For example, the client device 100 may be configured to hold a callback for each key (FIG. 2B) or for each of a plurality of keys grouped in advance by attributes or the like. Alternatively, the client device 100 may be configured to hold the callback by classifying each server and each key (or each key group). In the case where the callback is held for each key (or each key group), the read access issuing unit 110 searches for the distributed table server 10 in charge of the key (or key group) to be accessed, and performs standby read access. Can be issued. In the case of a configuration in which the keys are classified and held for each key (or for each key group), the read access issuing unit 110 can flexibly cope with a change in the key (or key group) assigned to the server.

  Returning to FIG. 1, the registration unit 106 does not immediately execute the received read access processing, but waits, and accesses the key and callback as standby read access to be issued later to the corresponding distributed table server 10 (server in charge). Register in the waiting list 104 and buffer. As will be described later, in this embodiment, when registering the standby read access in the access waiting list 104, the registration unit 106 may notify the determination unit 108 of the registration. The registration unit 106 may be a read function execution unit that executes a read function, and may notify that the read function execution unit has registered in the access waiting list 104.

  The determination unit 108 determines the issue destination distributed table server 10 that issues a plurality of standby read accesses and issues the standby timing for standby read accesses registered in the access waiting list 104 according to a predetermined condition. For example, the determination unit 108 determines that the issue timing is when the number of standby read accesses registered in the access waiting list exceeds a certain number.

  That is, for each distributed table server 10, when the number of standby read access processes registered in the access waiting list 104 exceeds a certain number, the determining unit 108 uses the distributed table server 10 as an issue destination and distributes the distributed read The read access processing issuance timing for the table server 10 is used. Alternatively, for example, when the number of standby read access processes registered in the access waiting list 104 exceeds a certain number for each key, the determination unit 108 performs the read access process on the distributed table server 10 in charge of the key. The issue timing of the read access processing for the distributed table server 10 may be determined as the issue destination of.

  Hereinafter, the determination method of the issuance timing of the standby process is referred to as a first method. In this first method, the memory amount required for holding the access waiting list 104 and the difference in communication speed for each assigned distributed table server 10 are not considered. Therefore, in the client apparatus 100, it is desirable to select and employ the optimum method in consideration of the system configuration of the distributed system 1 from among the first methods combined with other methods described later.

  The read access processing issuance timing is determined not by the server but by the number of standby read access processes registered in the access waiting list 104 for each key or for each of a plurality of keys grouped in advance according to attributes or the like. It may be when it becomes more than a certain number. Then, the distributed table server 10 in charge of the key whose number of standby read access processes is a certain number or more is set as the read access process issue destination. The plurality of standby read accesses may be issued to the distributed table server 10, the key of the distributed table server 10, or a plurality of keys of the same group including the key.

  The read access issuing unit 110 issues a plurality of standby read accesses to the issue destination distributed table server 10 determined by the determining unit 108 at the determined issue timing, and sends the access result (value) from the distributed table server 10. receive. That is, a plurality of read accesses from the same server are issued together.

  If the read accesses are not issued all at once and are executed each time, the time required for one process is latency + read data size / throughput. This processing time is required for each read access. On the other hand, if a plurality of read accesses are issued together, the latency of only one processing out of the processing time is required, and the processing time can be shortened.

  The callback executing unit 112 executes the corresponding standby read access callback registered in the access waiting list 104 using the access result (value) received by the read access issuing unit 110, and waits for the executed callback. The read access is deleted from the access waiting list 104.

In the client device 100 according to the present embodiment, various processing operations corresponding to the computer program 70 (FIG. 3) are executed by the CPU 62 (FIG. 3) of the computer 60, whereby the various units as described above are realized as various functions. Is done.
The computer program 70 of the present embodiment causes a computer 60 for realizing an information processing apparatus (client apparatus 100) to read access processing keys for reading data from a plurality of distributed tables respectively included in a plurality of distributed table servers 10, and , A procedure for accepting a callback for determining a process to be executed using a result read out based on a key, a wait for the accepted read access process, and a key and a callback as a standby read access issued later to the corresponding distributed table server 10 For the standby read access registered in the access waiting list 104, the procedure for determining the distributed table server 10 issuing the standby read access and the issuing timing in accordance with predetermined conditions, and the determined amount The standby read access is issued to the table server 10 at the determined issuance timing, the procedure for receiving the access result from the distributed table server 10, and the corresponding standby read registered in the access waiting list 104 using the received access result It is described that an access callback is executed and a procedure for deleting the standby read access of the executed callback from the access waiting list 104 is executed.

  The computer program 70 of this embodiment may be recorded on a recording medium readable by the computer 60. The recording medium is not particularly limited, and various forms can be considered. The program 70 may be loaded from a recording medium into the memory (ROM 64 or RAM 66) of the computer 60, or may be downloaded to the computer 60 through a network and loaded into the memory.

A data processing method by the client device 100 in the distributed system 1 according to the present embodiment having the above-described configuration will be described below. FIG. 4 is a flowchart showing an example of the operation of the distributed system 1 of the present embodiment. Hereinafter, description will be made with reference to FIGS. 1 and 4.
In the data processing method of the information processing apparatus (client apparatus 100 in FIG. 1) according to the embodiment of the present invention, the client apparatus 100 receives data from a plurality of distributed tables respectively included in the plurality of distributed table servers 10 (FIG. 1). A read access process key to be read and a callback for determining a process to be executed using a result read based on the key are received (step S101), the received read access process is waited, and the corresponding distributed table server 10 is later processed. The distributed table server that registers the key and callback in the access waiting list 104 as standby read access issued in step S103 and issues standby read access according to a predetermined condition for the read access registered in the access waiting list 104 10 and issue Taimin (Step S105), issues a standby read access to the determined distributed table server 10 at the determined issue timing (YES in Step S107), receives the access result from the distributed table server 10 (Step S109), Using the received access result, a corresponding standby read access callback registered in the access waiting list 104 is executed (step S111), and the standby read access of the executed callback is deleted from the access waiting list 104 (step S111). S113).

FIG. 7 is a flowchart illustrating an example of read access issuance timing and issue destination server decision processing procedures of the decision unit 108 of the client apparatus 100 of the distributed system 1 according to the present embodiment. In the present embodiment, the determination unit 108 performs determination processing using the first method described above.
First, every time the determination unit 108 receives a notification from the registration unit 106 (read function) that it has been registered in the access waiting list 104 (YES in step S121), this processing starts.

Then, the determination unit 108 refers to the access waiting list 104 and determines whether there is a distributed table server 10 in which the number of standby read access processes is equal to or greater than a certain number (step S123). If there are no more than a certain number of servers (NO in step S123), the procedure returns to step S121, and the determination unit 108 waits until the next notification arrives (NO in step S121).
When there are more than a certain number of servers (YES in step S123), the determination unit 108 issues a plurality of read accesses waiting for the server collectively (step S125).

  Alternatively, each time the registration unit 106 registers in the access waiting list 104, the registration number may be counted for each distributed table server 10 and stored in a memory (RAM 66 or the like). Then, the determination unit 108 may monitor the presence or absence of the distributed table server 10 in which each count value becomes a certain value or more with reference to the memory.

  As described above, according to the distributed system 1 according to the embodiment of the present invention, the client apparatus 100 targets an arbitrary program and uses a read function to the distributed table server 10 that appears at an arbitrary place on the program. A plurality of read access processes can be collectively executed for each server by registering in the access waiting list 104 and waiting for a certain number or more instead of performing each read access each time. With this configuration, in the distributed system 1 according to the embodiment of the present invention, compared to the case where multiple read access processes appearing on the program are processed each time, the access processing speed due to the latency at the time of read access can be reduced. The influence can be reduced and the speed can be increased.

  In particular, according to the distributed system 1 according to the embodiment of the present invention, the client device 100 can be suitably applied to a plurality of read access processes in which it is not known where and when it appears on the program. This is because the client device 100 has a configuration in which when a read access process appears, the client apparatus 100 accepts the process and waits appropriately.

(Second Embodiment)
The distributed system 1 of the present embodiment is different from the above-described embodiment in the method for determining the issuance timing of the standby access process, considering the system configuration, and the network configuration between the nodes of the cluster system (distributed table server 10). In consideration of a short distance, a node that is optimal for speeding up is selected and standby access processing is executed.
The configuration of the distributed system 1 of the present embodiment is the same as that of the distributed system 1 of the above-described embodiment of FIG. 1 and will be described below with reference to FIG.

  In the above-described embodiment, the client device 100 can reduce the influence of latency by processing a plurality of accesses to the distributed table into one, and can increase the processing speed. In the present embodiment, the client apparatus 100 further includes a system of the distributed system 1 such as which access is given priority to which server at which timing, a difference in memory amount, communication speed between machines, and the like. The performance is further improved by making a decision in consideration of conditions such as the configuration.

First, as a second method, a method of determining the issue timing and the issue destination server in consideration of the memory amount will be described.
In the present embodiment, the determination unit 108 determines that the issue timing has come when the memory amount of the access waiting list 104 exceeds a certain amount, and when the memory amount exceeds the certain amount, the access waiting list 104 is determined. The distributed table server 10 having a large number of standby read accesses registered in is selected as a standby read access issue destination.

The memory amount of the access waiting list 104 can be obtained by adding the memory amount required for the key and the callback. However, if there is memory shared between multiple Callback objects, it may be difficult to accurately determine the amount of memory required for the callback. In this case, the programmer may specify the amount of memory necessary for registering the access waiting list 104 as an argument of the read function. The used memory amount of the access waiting list 104 is subtracted as soon as the execution of the callback is completed and the memory used by the object is released.
When the access waiting list 104 is classified and registered for each key, the distributed table server 10 corresponding to the key having a large number of standby read accesses registered in the access waiting list 104 is set as the standby read access issue destination. select.
In the second method, the difference in communication speed for each server in charge is not considered. Therefore, it is desirable that the client apparatus 100 selects and employs the optimum method in consideration of the system configuration of the distributed system 1 among the second methods combined with other methods.

Next, as a third method, a method for determining an issue timing and an issue destination server in consideration of a difference in communication speed between machines will be described.
In the present embodiment, the determination unit 108 is a server that is a destination of standby read access based on the acquisition time of the access result (value) per key obtained based on the performance information of each distributed table server 10. Select.
In the third method, when the memory amount of the access waiting list 104 exceeds a certain amount, the determination unit 108 determines that the distributed table server 10 having the shortest value acquisition time per key is the standby read access issue destination. Choose as. That is, the determination unit 108 selects the distribution table server 10 having a high communication speed as the issue destination.

Specifically, the value acquisition time per key can be obtained by the following formula (1) for each distributed table server 10.

  Here, considering that the distributed system 1 of the present embodiment is a cluster system, there are nodes that are close to each other in terms of network and nodes that are far from each other (distributed table server 10). Here, the network distance can be evaluated by the acquisition time of the access result (value) per key, and can be obtained from the communication latency and throughput value from each node to other nodes.

The acquisition time of the access result (value) per key is the minimum for the node where the standby access processing is the largest if the distance between the nodes is the same.
If the number of standby access processes is the same, the node having the shortest distance between nodes has the shortest acquisition time of the access result (value) per key.

  Therefore, in the client device 100, if the distance between the nodes is the same, the standby access process is executed from the node having the largest number of standby access processes. It is desirable to execute access processing. In other words, it is desirable to execute more standby access processes for nodes that are far away. In the present embodiment, the determination unit 108 can select a node having the least influence of latency by selecting the node having the shortest value acquisition time per key as the standby read access issue destination.

  Here, the client device 100 obtains in advance information such as communication latency and throughput to each assigned distributed table server 10 necessary for considering the difference in communication speed for each assigned server. For these pieces of information, values obtained from the measurement results and values calculated from the system configuration (network configuration including the number of stages of switches / routers) may be registered in the client apparatus 100 in advance. Alternatively, the client device 100 may include a calculation unit (not shown) that measures data necessary for system initialization and obtains such information.

  There is a possibility that a plurality of read accesses to the same key may be registered in the access waiting list 104. However, in the equation (1), the determination unit 108 collects them as one and counts them as one key. Find the number of keys. When it is difficult to obtain the total number of value bytes corresponding to a plurality of keys, for example, the byte size of the value differs depending on the key, the determining unit 108 may use an approximate value in Expression (1). When the influence of the communication latency is overwhelmingly larger than the influence of the throughput, the determination unit 108 sets the term (total number of bytes of values corresponding to a plurality of keys) / throughput in Expression (1). You may delete and approximate.

  In this way, if all the distributed table servers 10 have the same condition, the client apparatus 100 can also perform the same for the server with the largest number of access waiting queues in the third method as in the second method. Can issue read access. If all the distributed table servers 10 have the same access waiting number, a server close to the network is selected. In other words, the client device 100 issues a read access after as much as possible for a server far from the network.

Next, a fourth method in which the determination unit 108 determines the issue timing and the issue destination server will be described.
In the fourth method, the determination unit 108 calculates “value acquisition time per key” of each distributed table server 10 described in the third method at a predetermined timing, The minimum value is obtained, and the issue timing and the issue destination server are determined based on the minimum value.

The timing at which the determination unit 108 calculates “value acquisition time per one key” is determined every time read access is registered in the access waiting list 104, every time read access is registered in the access waiting list 104 a predetermined number of times, Or it is every fixed time. The determining unit 108 determines that the issue timing is when the calculated minimum value is equal to or less than the threshold value, and sets the server having the minimum value as the issue destination. Here, the determination unit 108 can detect the presence / absence of registration in the access waiting list 104 and the number of registrations based on the notification from the registration unit 106.
The determination unit 108 can obtain the “value acquisition time per key” by the above-described formula (1) for each server, as in the third method, and a detailed description thereof will be omitted.

  Here, the determination unit 108 can detect that the read access has been registered in the access waiting list 104 based on a registration notification from the registration unit 106 (read function). The reason why this notification (read access) waits for a predetermined number of times or more is that the read function registers in the access waiting list 104 whether or not there is a server whose minimum value acquisition time per key is less than or equal to the threshold. This is because when it is inefficient to calculate each time it is received, it is calculated together and the overhead is reduced.

  In the fourth method, since the client device 100 does not refer to the memory amount of the access waiting list 104, there is a possibility that the memory for the access waiting list 104 is consumed too little or too much depending on the set threshold value. . Therefore, it is desirable that the client apparatus 100 selects and employs the optimum method in consideration of the system configuration of the distributed system 1 from among the fourth methods combined with other methods.

As the fifth method, the following method is also conceivable.
The determination unit 108 selects the distributed table server 10 that has not been issued a read access for a certain period of time (for example, 10 ms) as a read access issue destination. The issue timing is when there is a server for which read access has not been issued for a certain period of time.

  The reason why the distributed table server 10 for which read access has not been issued for a certain period of time in this method is the issue destination is that when the distributed system 1 of the present invention is used for the Web server / AP server 94 shown in FIG. This is because it is not desirable to make the user wait. However, in this case, since the memory amount of the access waiting list 104 and the value acquisition time per key are not referred to, the fifth method is not necessarily an efficient server selection. Therefore, it is desirable that the client apparatus 100 selects and employs the optimum method in consideration of the system configuration of the distributed system 1 from among the fifth methods combined with other methods.

  Furthermore, in the sixth method, the programmer explicitly designates the server and issue timing for issuing read access on the program. For example, in an example of a function described in C ++, by executing a function such as flush (server name), the read access issuing unit 110 issues a read access to the server at the timing when this function is executed. To do.

  Moreover, the determination part 108 may employ | adopt as a determination method combining the 1st-6th method at least 2 as a 7th method. Details will be described later.

8 to 16 are flowcharts illustrating examples of processing procedures according to the respective methods of the determination unit 108 of the client device 100 of the distributed system 1 according to the present embodiment.
The processing procedure in FIG. 8 is an example when the determination unit 108 performs the determination process using the second method described above. In other words, this processing procedure is an example of a determination process for issuing a read access to a server having the largest number of standby read accesses when the memory amount is equal to or greater than a certain level.

  In the example of FIG. 8, each time the determination unit 108 receives notification from the read function that it has been registered in the access waiting list 104 (YES in step S121), this process starts. When the notification is received (YES in step S121) and the amount of memory in the access waiting list 104 exceeds a certain amount (YES in step S131), the determination unit 108 determines that the issue timing has come.

Then, the determination unit 108 determines the distribution table server 10 having the largest number of standby read accesses registered in the access waiting list 104 as the issue destination at that time. Then, the read access issuing unit 110 issues a read access to the distributed table server 10 determined as the issue destination (step S133).
On the other hand, if the memory amount is not equal to or greater than the predetermined amount (NO in step S131), the procedure returns to step S121, and the determination unit 108 waits until the next notification arrives (NO in step S121).

Next, the processing procedure of FIG. 9 is an example when the determination unit 108 performs the determination process using the third method described above. That is, this processing procedure is an example of a determination process for issuing a read access to a server having the shortest value acquisition time per key when the amount of memory is a certain level or more.
In the example of FIG. 9, when a notification indicating that it has been registered in the access waiting list 104 is received from the read function (YES in step S121), and the memory amount in the access waiting list 104 becomes equal to or greater than a certain amount (YES in step S131). ), The determination unit 108 determines that the issue timing has come. The processing procedure of FIG. 9 so far is the same as the example of FIG.

  Then, the determination unit 108 determines the distribution table server 10 having the shortest value acquisition time per key as the issue destination, and the read access issue unit 110 reads the read table to the distribution table server 10 of the determined issue destination. An access is issued (step S141).

  Next, the processing procedure of FIGS. 10 to 12 is an example in the case where the determination unit 108 performs the determination process using the fourth method described above. The procedure in each figure is different in timing for calculating the value acquisition time per key (or comparing the minimum value acquisition time with a threshold value).

FIG. 10 shows an example in which the determination process according to the fourth method is performed every time read access is registered in the access waiting list 104. FIG. 11 shows an example in which the determination process by the fourth method is performed when the read access is registered in the access waiting list 104 a predetermined number of times or more. FIG. 12 shows an example in which determination processing by the fourth method is performed at predetermined time intervals.
First, in the example of FIG. 10, each time the determination unit 108 receives a notification that it has been registered in the access waiting list 104 from the read function (YES in step S121), this process starts.

Then, the determination unit 108 determines whether there is a server whose minimum value acquisition time per key is equal to or less than a threshold value (step S151). If there is no server whose minimum value is equal to or smaller than the threshold value (NO in step S151), the procedure returns to step S121, and the determination unit 108 waits until the next notification arrives (NO in step S121).
When there is a server whose minimum value is equal to or smaller than the threshold (YES in step S151), the determination unit 108 determines that server as an issue destination, and the read access issue unit 110 issues read access to the server (step S125).

  In the example of FIG. 11, the determination unit 108 receives a notification that it has been registered in the access waiting list 104 from the read function (YES in step S121), and the determination unit 108 counts the number of times the notification is received. In addition, when the notification is received a predetermined number of times or more (YES in step S161), this processing starts. The subsequent procedure is the same as in FIG. When the determination unit 108 executes this process, the count is reset. That is, this process is executed every predetermined number of times.

In the example of FIG. 12, a predetermined time interval is measured by a timer (not shown), and every time the determination unit 108 receives a notification (timer interrupt) from the timer (YES in step S171), that is, at regular intervals. This process starts. The subsequent procedure is the same as in FIG. The timer time can be set to 10 ms, for example, and may be a preset value or may be changed according to conditions.
Further, in the client device 100, the procedures in FIG. 10 or FIG. 11 and FIG. 12 may be combined.

  Furthermore, the processing procedure of FIG. 13 is an example in the case where the determination unit 108 performs the determination process using the fifth method described above. In the example of FIG. 13, a predetermined time interval is measured by a timer (not shown), and every time the determination unit 108 receives a notification (timer interrupt) from the timer (YES in step S171), that is, at regular intervals. Processing starts. Then, the determination unit 108 determines whether or not there is a distributed table server 10 for which a read access has not been issued for a certain time or more (step S181). If there is no distributed table server 10 for which read access has not been issued (NO in step S181), the procedure returns to step S171, and the determination unit 108 waits until the next notification arrives (NO in step S171). When there is a distributed table server 10 for which read access has not been issued (YES in step S181), the determination unit 108 determines that server as an issue destination, and the read access issue unit 110 issues read access to the server. (Step S125).

  Further, the processing procedure of FIG. 14 is an example in the case where the determination unit 108 performs determination processing using the above-described sixth method. In the example of FIG. 14, first, the determination unit 108 calls a function that specifies a server that issues read access and an issue timing (step S <b> 191). When the timing specified by the function is reached (YES in step S193), the determination unit 108 determines the specified distributed table server 10 as an issue destination, and issues a read access to the issue destination distributed table server 10. The unit 110 issues a read access (step S195).

As a seventh method, FIGS. 15 and 16 show processing procedures when the determination unit 108 adopts a determination method by combining at least two of the first to sixth methods. Here, the flows of FIGS. 15 and 16 and FIG. 14 can be executed in parallel as will be described later.
If there is a server whose “value acquisition time per key” is equal to or less than a threshold value as in the fourth method (YES in step S151 in FIG. 15), the read access issuing unit 110 performs read access to the server. Is issued (step S125 in FIG. 15).

  Even if there is no such server (NO in step S151 in FIG. 15), when the amount of memory in the access waiting list 104 exceeds a certain amount as in the third method (YES in step S131 in FIG. 15). Then, the determination unit 108 selects the server having the smallest “value acquisition time per key” as the issue destination, and the read access issue unit 110 issues the read access (step S141 in FIG. 15). Even if these conditions are not satisfied (NO in step S131 in FIG. 15 and NO in step S151), a server that has not issued read access for a certain period of time as in the fifth method (in FIG. 16). In step S181, the read access issuing unit 110 issues a read access (step S125 in FIG. 16). Also, a read access is issued when explicitly specified on the program as in the sixth method (step S195 in FIG. 14).

  Further, when issuing a read access by these methods, the read access issuing unit 110 may execute the issuance in an overlap manner in parallel with processing other than the read access or the previously issued read access. Further, when issuing read access by the first to fourth, sixth, and seventh methods, the read access issuance by the read access issuing unit 110 is mainly triggered by the execution of the read function. Become. If the read access is issued from among the read functions that trigger this, and the read access is issued when the read function is completed, the overlap is not performed. Furthermore, when issuing a read access from a read function that triggers, if the read access is issued in a separate thread, the read access is issued in a manner other than processing other than read access or other read access issuance. You can wrap and run.

  When the read access issuing unit 110 issues read access to a server to which read access has not been issued by the fifth method or the seventh method (YES in step S181 in FIG. 16), Alternatively, the “value acquisition time per key” is calculated for each “certain time” by the fourth method, and the read is performed for a server that is below the threshold (YES in step S151 in FIG. 15 or FIG. 16). When issuing an access, an overlap can be realized by performing the determination performed for each of these “certain times” with independent threads and issuing a read access with the thread. As described above, in the client device 100, the processing time can be effectively used by asynchronously overlapping and executing read access.

As described above, according to the distributed system 1 according to the embodiment of the present invention, it is possible to achieve the same effect as the above embodiment, and to optimize the memory amount and the difference in communication speed between machines. An access method can be provided.
The reason is that, when the memory consumed in the access waiting list 104 exceeds a certain amount, the determination unit 108 selects a server having the shortest acquisition time per element and reads the plurality of keys from the server. This is because by issuing access, it is possible to select the most appropriate server at that time while using the memory of the machine as much as possible.

In the technique described in Non-Patent Document 1, even if a plurality of accesses to the distribution table can be combined, it is not clear at what timing it is optimal to prioritize which server.
That is, in the technique described in Non-Patent Document 1, it is necessary to delay the processing in some form in order to combine a plurality of accesses. Therefore, it is necessary to realize the most efficient access method while keeping the amount of memory required for that purpose constant.

  Furthermore, considering the environment where the system actually operates, the machines constituting the distributed table are not necessarily in the same condition. In other words, when the number of machines increases, the connection is often configured as a multi-stage tree. For example, a machine in the same rack may have a higher communication speed than a machine in another rack. .

  According to the distributed system 1 according to the embodiment of the present invention, it is possible to realize an optimum access method taking such an environment into consideration.

(Third embodiment)
FIG. 17 is a functional block diagram showing the configuration of the distributed system 1 according to the embodiment of the present invention.
The distributed system 1 of the present embodiment is different from the above-described embodiment in that not only read access to the distributed table, but also write access or accumulated access is registered in the access waiting list 104.

  The distributed system 1 according to the embodiment of the present invention includes a client device 200 connected to a plurality of distributed table servers 10 via a network 3 (not shown in FIG. 17). Note that the client apparatus (shown as “distributed table client” in the figure) 200 of this embodiment is the same determination process as the determination unit 108 of the client apparatus 100 of the first embodiment or the second embodiment. Procedures including procedures can also be performed.

  The client device 200 of the distributed system 1 according to the present exemplary embodiment includes the same access waiting list 104, a read access issuing unit 110, and a callback execution unit 112 as the client device 100 according to the exemplary embodiment of FIG. Further, the client device 200 includes a read function execution unit 202, a write function execution unit 204, an accumulate function execution unit 206, and a write / accumulate access issue unit. 208 and a determination unit 210.

  When the program is executed and the read function is called, the read function execution unit 202 sets the key (Call) and callback of the read function for each distributed table server 10 in charge of the key on the distributed table. The information is classified and registered in the access waiting list 104. That is, the read function execution unit 202 registers the key and callback in the access waiting list 104 as buffer for standby read access and buffers them. In the present embodiment, the read function execution unit 202, the write function execution unit 204, and the accumulation function execution unit 206 correspond to the reception unit 102 and the registration unit 106 in the above embodiment.

  When the program is executed and the write function is called, the write function execution unit 204 requests the write / accumulate access issue unit 208 to issue a write access, or performs a write access process. The write function is a function for writing data in the distributed table, and a data key (Key) for writing data and data (Value) to be written are designated as arguments. The write access process is a process of writing the argument Value data to the argument Key data specified by the write function.

  When the program is executed and the accumulation function is called, the accumulation function execution unit 206 requests the write / accumulate access issuing unit 208 to issue an accumulation access or performs an accumulation access process. The Accumulate function performs the arithmetic processing specified as an argument between the value (data) originally in the distributed table corresponding to the key (Key) specified as the argument and the value (Value) specified as the argument. Access and storing the result as key data (value). Here, it is assumed that the specified arithmetic processing satisfies the combining law. An example of the calculation process is addition, multiplication, or obtaining a maximum value or a minimum value. Accumulate access processing is the operation processing specified by the argument between the Key data of the argument specified by the Accumulation function and the Value specified by the argument, and the result of the operation is added to the Key data of the argument. This is a process of writing a value.

  The write / accumulate access issuance unit 208 sends a write or an accumulative access to the corresponding distributed table server at a request from the write function execution unit 204 or the accumulation function execution unit 206 or at an issuance timing determined by the determination unit 210. Issue to 10.

  In the client device 200 of the distributed system 1 according to the present embodiment, the reception unit (write function execution unit 204) further receives a write access process for writing a value to the key of the distribution table. Then, the read access issuing unit 110 refers to the access waiting list 104, and if there is a standby read access for the same key as the write access process, executes a standby read access callback with the value (value) of the write access process. The standby read access is deleted from the access waiting list 104.

  Further, the reception unit (write function execution unit 204) receives a prohibition flag for prohibiting the write access process received after the read access process from being processed first together with the key and callback of the read access process. Then, the registration unit (write function execution unit 204) registers the prohibition flag received by the write function execution unit 204 in the access waiting list 104 together with the standby read access key and callback. Further, the registration unit (write function execution unit 204) refers to the access wait list 104, and if the standby read access prohibition flag for the same key as that of the write access process is registered, the write access process waits, and the corresponding. A key and a value are registered in the access waiting list 104 as a standby write access to be issued later to the server.

  Here, when the write function or the accumulation function is not registered in the access waiting list 104 for the read access of the target key, or is registered, the write function execution unit 204 or The accumulation function execution unit 206 requests the immediate write / accumulate access issuance unit 208 to issue a write or an accumulative access. When a read access that cannot be overtaken is registered in the access waiting list 104, the write function executing unit 204 or the accumulating function executing unit 206 registers the write or accumulating access process in the access waiting list 104 as a standby access. .

  In this embodiment, the determination unit 210 determines when (issue timing) and which distributed table server 10 (issue destination) for the read access registered in the access waiting list 104, the write access that waits for it, and the accumulative access. ) To issue.

  In this embodiment, when the memory consumed in the access waiting list 104 exceeds a certain amount, the determination unit 210 selects the distributed table server 10 having the minimum acquisition time per element, and the read access issue unit 110. Issues read access to a plurality of keys to the server. Then, the callback execution unit 112 executes the callback (Callback) registered using the acquired key (Key) and value (Value). Thereafter, the write / accumulate access issuing unit 208 issues the waiting write access or the accumulated access to the corresponding distributed table server 10.

  Other methods are also conceivable in the determination unit 210 for determining when and to which server the lead is issued, as in the above embodiment. That is, the determination method includes a method of issuing a server for which read access has not been issued for a certain period of time, a method of issuing a server having an acquisition time per element of a certain value or less, a method of explicitly specifying by a programmer, There are ways to combine these.

  By adopting such a configuration and operation, the client device 200 according to the present embodiment collectively issues a plurality of read accesses appearing at arbitrary locations in the program, thereby reducing the influence of read access latency. can do. In the present embodiment, the determination unit 210 can provide an optimal access method that takes into account the amount of memory, the difference in communication speed between machines, and the like. Thereby, the object of the present invention can be achieved.

Hereinafter, each access process will be described with specific examples.
First, a case where data is read from the distributed table (read access) will be described.
In the present invention, in a program including a process for accessing the distributed table server 10, a mechanism called a callback is used to combine accesses to a plurality of distributed tables appearing at different locations in the program into one. .
Here, the processing executed after accessing the distributed table is realized by callback.
An example in which a callback is described in C ++ is shown below.

class Callback {/ * Provided by the system * /
public:
virtual void run (string Key, string Value) = 0;
};

class MyCallback: public Callback {/ * Described in individual programs * /
public:
MyCallback (...) {
/ * Initialize data necessary to execute run () when creating an object * /
}
virtual void run (string Key, string Value) {
/ * Describe the processing to be performed as a result of obtaining the value for Key * /
}
private:
/ * Save the data necessary for executing run () for each program * /
};

Here, an example is shown in which the system provides a Callback class and inherits it to generate a callback (in this case, MyCallback) class for use in individual programs. The Callback class provided by the system is an abstract class whose run () member function is only provided as a virtual function.
Individual programs that are executed on the system and access the distributed table server 10 are determined in advance. By describing the declaration of the above MyCallback class and the following read function in the individual programs in advance, the system Realized.

An example in which a program interface (read function) for performing read access to a distributed table is described in C ++ is shown below.

read (Key, new MyCallback (...));

In each program, when the read access is made to the distributed table, the read function of the program interface may be called.

  This read function performs read access to the first argument Key for the distributed table, and calls the run of MyCallback using the result as an argument. However, the description of the read function means that the access to the distributed table is not necessarily finished when the read function is finished.

Based on the meaning defined in this way, the read function operates as follows.
The read function registers the Key and Callback object given as arguments in the access waiting list 104. An example of the access waiting list 104 at this time is as shown in FIG. 2 shown in the above embodiment. The access waiting list 104 in the example of FIG. 2A is a collection of given keys and callbacks for each distributed table server 10 in charge of keys. The access waiting list 104 in the example of FIG. 2B is a collection of given keys and callbacks for each key. In the following description, it is assumed that the access waiting list 104 collected for each server in FIG.

  The read access request registered in the access waiting list 104 is accumulated to some extent, and the issuing unit 210 determines the issue timing and the issue destination server. About operation | movement of the determination part 210, it is the same as that of the determination part 108 of the said embodiment.

  When the determination unit 210 determines to issue a read access to a certain distributed table server 10, the read access issue unit 110 determines that the read access to at least one key in the server column of the access waiting list 104 is the distribution table. This is performed on the server 10. Then, the read access issuing unit 110 receives a value for each key.

  The callback execution unit 112 executes the callback registered in the server field of the access waiting list 104 using the key and the received value. If the example in C ++ mentioned above is used, the callback execution part 112 will execute the run member function of Callback object by using Key and Value as an argument.

  As described above, the client apparatus 200 can reduce the influence of the latency in accessing the distributed table by collecting a plurality of accesses to the same server in the read access to the distributed table existing in a place distant from the program.

  In addition, in the callback, not only the read function but also the write function or the accumulation function can be executed, and the write function or the accumulation function is processed in the access waiting list 104 together with the read function for the same key. You may register as a plurality together. Further, when the read function for the same key is executed by the program a plurality of times, a plurality of read functions may be registered in the access waiting list 104.

Next, a case where there is a write access (write access) to the distributed table will be described.
Normally, when the shared data is accessed from a plurality of machines (for example, the client device 200 or other machines connected to the client device 200), exclusive control such as locking is performed. Such control is not targeted. That is, in this embodiment, the case where the read and write access to the distributed table in each machine can be executed in an arbitrary order is targeted.

  First, the client device 200 classifies the write access into a write access and an accumulate access. Write access is normal writing that overwrites a value corresponding to a key. Accumulated access is an access that performs an operation between the original value and a specified value and stores the result. Here, it is assumed that the specified operation is one in which a combination rule is established. Examples of addition and multiplication are operations for obtaining a maximum value or a minimum value.

  For example, in the case of addition, if access that adds Y to the value of Key X is assumed to be accumulate (add, X, Y), etc., if the value of Key X is 1, accumulate (add, X, 1), accumulate (add , X, 2) is issued, the value of Key X becomes 4. This result does not depend on the order in which accumulate (add, X, 1) and accumulate (add, X, 2) are executed.

  As in the case of reading, both the write access and the accumulative access accepted by the client apparatus 200 are called from the write function and the accumulating function described in each program. As in the case of the read function, both the write function and the accumulation function are completed on the program when the write access and the accumulation access to the distributed table server 10 via the client device 200 are completed. Not necessarily.

First, write access will be described. An example in which the write function is described in C ++ is shown below.

Write (Key, Value);

  The write access does not use the return value, so the processing speed is not affected by the latency. Therefore, the client device 200 may issue a write access basically unconditionally. However, depending on the contents of the program, if the read function is executed for the same key before the write function is executed, it may be possible to prevent the write from overtaking the read. This is in the same machine, and as described above, the order control of read access and write access among a plurality of machines is not considered. Therefore, in the present embodiment, the case where the write access may pass the read access and the case where the write access does not pass are considered.

  When the write access may pass the read access, when the write function is executed by the write function execution unit 204, the write function execution unit 204 first refers to the access wait list 104, and the read for the same key is the access wait list 104. Check whether it is registered in. If registered, according to the request from the write function execution unit 204, the write / accumulate access issuing unit 208 issues a write request to the server in charge of the key, and also writes the value to be written with the key and the write function. The registered callback is executed as an argument. If not registered, according to the request from the write function execution unit 204, the write / accumulate access issuing unit 208 simply issues a write request to the server in charge of the key.

Next, consider a case where it is desired that write access does not overtake read access. In this embodiment, in order to cope with such a case, a flag (true) is set for a lead that is not to be overtaken. In each program, for example, the argument of the read function can be expanded to indicate that the read access to this key is not overtaken by the write as follows. This is hereinafter referred to as a lead that cannot be overtaken.

read (Key, new MyCallback (...), true);

  When such an instruction is given, that is, when the overtaking impossible flag (true) is designated, the registration unit 106 (write function execution unit 204) puts the information (overtaking impossible flag (true)) in the access waiting list 104. ). That is, as shown in FIG. 18, the registration unit 106 (write function execution unit 204) sets a key (Key1) and a callback (Callback1) as standby read access, and an overtaking impossible flag (true) as an access waiting list 104. Register with.

  Consider a case where a write function is executed for the same key (Key1) as a read access that cannot be overtaken. When the write function is executed by the write function execution unit 204, the write function execution unit 204 first refers to the access waiting list 104. When there is a read that cannot be overtaken for the same key (Key1) (when the flag true is registered), the write function execution unit 204 (registration unit 106) also registers the write in the access waiting list 104. This is shown in FIG.

Next, consider the operation of each function in a state where the write is registered in the access waiting list 104 (the state shown in FIG. 18).
When the read function for the same key is executed, the callback execution unit 112 executes the callback using the value of the waiting write.
When the write function for the same key is executed, the write / accumulate access issuing unit 208 overwrites the value of the waiting write with the value of the newly executed write function.
When the accumulation function for the same key is executed, the write / accumulate access issuing unit 208 calculates the result of executing the accumulation function using the value specified in the write function as the original value, and the value of the write function is calculated. To change. For example, if the value specified in the write function is 1 and the accumulate function is to add 2, the write / accumulate access issuing unit 208 changes the value specified in the write function to 3.

  If the read access is issued from the access waiting list 104 by the read access issuing unit 110 in a state where the write is registered in the access waiting list 104 (the state shown in FIG. 18), the read waiting is completed. The write / accumulate access issuing unit 208 issues the write access that has been waiting.

Next, accumulated access will be described. An example in which the accumulation function is described in C ++ is shown below.

accumulate (TYPE, Key, Value);

Here, the TYPE of the first argument specifies an operation to be executed (add for addition, mul for multiplication, etc.).

  First, a case where there is no lead that cannot be overtaken in the access waiting list 104 for the key specified in the accumulation function will be described. In this case, according to the request from the accumulation function execution unit 206, the write / accumulate access issuing unit 208 immediately issues the accumulation function. Unlike the case of the write function, regardless of whether there is a read access that cannot be overtaken in the access waiting list 104, the callback of the read access is not executed at this time. This is because if the value corresponding to the current key is not known, the value is not determined only by the argument of the accumulation function.

  Next, a case where a lead that cannot be overtaken is in the access waiting list 104 will be described. In this case, as in the case of the write function, the accumulation function execution unit 206 registers the accumulation access in the access waiting list 104. This is shown in FIG.

Consider the operation of each function in a state where the accumulation is registered in the access waiting list 104 (the state shown in FIG. 19).
When the read function for the same key is executed, the read function execution unit 202 (registration unit 106) further registers the read access in the access waiting list 104 (unlike the case of the write access described above). This is shown in FIG. If there is further read, write, or accumulated access to the same key from this state, the same operation as described above is performed.

  When a write function for the same key is executed in a state where the accumulation is registered in the access waiting list 104, the write function execution unit 204 (registration unit 106) determines the accumulation access in the access waiting list 104 as the write access. Replace with and overwrite.

  When the accumulation function is executed for the same key while the accumulation is registered in the access waiting list 104, the accumulation function execution unit 206 (registration unit 106) uses the same operation type (for example, addition (add)). If it is, etc.), update the accumulated value. For example, when accumulate (add, Key1,2) is executed in a state where accumulate (add, Key1,1) is registered, the accumulation function execution unit 206 (registration unit 106) displays the registered accumulation. Change rate access to accumulate (add, Key1,3). In the case of different calculation types, the accumulation function execution unit 206 (registration unit 106) registers the accumulation access further thereafter (not shown).

  If the read access is issued from the access waiting list 104 by the read access issuing unit 110 in a state where the accumulation is registered in the access waiting list 104 (the state shown in FIG. 19), the read / write accumulation is waited for. The rate access issuing unit 208 issues the accumulated access that has been waiting. Here, when the read access is further waiting for the accumulated access that has been waiting (the state of FIG. 20), the value after accumulation can be calculated by the read access, so the callback execution unit 112 uses this value. Execute the callback for the read access that has been waiting.

  For example, it is assumed that the value of Key1 is found to be 1 by the read access issuing unit 110 issuing a read from the state of FIG. The callback execution unit 112 executes Callback1 using this value. When Value2 of the registered accumulation is 2, the write / accumulate access issuing unit 208 issues an accumulation, and the callback execution unit 112 executes Callback5 using 1 + 2 = 3.

  In addition, as described above, the issue timing and issue destination server decision processing procedure by the decision unit 210 may be the first to seventh methods. Here, the description is omitted.

Next, the overall operation of the present embodiment will be described in detail with reference to the flowcharts of FIGS.
FIG. 21 is a flowchart illustrating an example of the operation of the read function execution unit 202 of the client device 200 of the distributed system 1 according to the present embodiment.
The read function execution unit 202 first checks the access waiting list 104, and checks the access waiting list 104 for the access waiting for the same key from the back of the access waiting list 104 (step S201). That is, as shown in FIGS. 18 to 20, there are cases where the write access, the accumulative access, and the read access are registered afterwards in the access waiting list 104, so the read function execution unit 202 checks this later.

  When write access is registered (YES in step S203: state shown in FIG. 18), the callback execution unit 112 executes a callback of an argument of the read function using the registered value (step S205). ), This process is terminated. At this time, the access waiting list 104 remains in the state shown in FIG.

  When accumulating access is registered (YES in step S207: state of FIG. 19), the read function execution unit 202 registers the Key and Callback object given as arguments after the accumulating access (step S209). . That is, the state shown in FIG. Thereafter, the read function execution unit 202 notifies the determination unit 210 that the read access has been registered (step S213). In addition, when the determination part 210 does not require this notification, this step can be omitted.

  When the write access and the accumulative access are not registered (NO in step S203 and NO in step S207), the read function execution unit 202 registers the Key and Callback object given as arguments in the access waiting list 104. (Step S211). The read function execution unit 202 notifies the determination unit 210 that the read access has been registered (step S213). As before, this step can be omitted if the determination unit 210 does not require this notification. Further, when registering the read access, the determination unit 210 adds the memory amount necessary for the registration to the memory consumption amount of the access waiting list 104 as necessary.

  As described above, by adding the read access key and the amount of memory required for the callback, the amount of memory consumed in the access wait list 104 can be obtained, and the memory amount to be added for each registration is specified by the programmer. You may make it do. The determination unit 210 also adds the memory amount when registering write / accumulate access, which will be described later. In these cases, the determination unit 210 adds the memory amount required for the write / accumulate access key and value (in the case of accumulation, the amount of memory required for TYPE information).

FIG. 22 is a flowchart illustrating an example of the operation of the write function execution unit 204 of the client device 200 of the distributed system 1 according to the present embodiment.
As in the case of the read function, the write function execution unit 204 checks the access waiting list 104 from the back of the access waiting list 104 for the same key in the access waiting list 104 (step S221). When write access is registered (YES in step S223: state of FIG. 18), the write function execution unit 204 overwrites the value of the registered write access with the value given as an argument (step S225). Then, the write function execution unit 204 ends this process.

  When the accumulated access is registered (YES in step S227: the state in FIG. 19), the write function execution unit 204 overwrites the registered accumulated access with the write access (step S229). That is, the state shown in FIG. 20 is obtained. Then, the write function execution unit 204 ends this process. If the accumulated access is not registered (NO in step S227), and if the read access is not registered (NO in step S231), the write function executing unit 204 sets the write / accumulate access issuing unit 208. Use write access is issued (step S233). Then, the write function execution unit 204 ends this process.

  When the read access is registered (YES in step S231), that is, in the state of FIG. 20 or FIG. 2, the write function execution unit 204 checks whether the registered read access can be overtaken. (Step S235). If the overtaking is not possible (NO in step S235), that is, if the flag true is registered, the write function execution unit 204 registers the write access behind this read access in the access waiting list 104 (step S237). ). When the overtaking is possible (YES in step S235), the write function execution unit 204 causes the callback execution unit 112 to execute the read access callback using the value of the argument of the write function, and accesses the read access. Delete from the waiting list 104 (step S239). Further, paying attention to the element immediately before the access waiting for the same key in the access waiting list 104, the process returns to step S221, and the write function execution unit 204 repeats the same processing as above (step S241). When the process is completed for all elements waiting for the same key in the access waiting list 104, the write function execution unit 204 ends the process after repeating the process for the empty element at the end.

  For example, when a write function having Key1 as an argument is executed from the state of FIG. 20 (YES in step S231), the read access at the end of the access waiting list 104 can be overtaken (YES in step S235). The execution unit 204 causes the callback execution unit 112 to execute Callback5 of the read access, and removes the read access from the access waiting list 104 (Step S239). Then, the access waiting list 104 is in the state shown in FIG. Then, returning to step S221, paying attention to the accumulated access at the end of the access waiting list 104, the write function execution unit 204 repeats the process (step S241).

  Further, when the write function having Key1 as an argument is executed from the state of FIG. 2 (YES in step S231), the read access can be overtaken (YES in step S235), so the write function execution unit 204 executes the callback. The read access callback is executed by the unit 112 and deleted from the access waiting list 104 (step S239). Since the next element of interest is not in the access wait list 104, the write function execution unit 204 performs the next iteration process on the empty element. Here, since any of the write access, the accumulation access, and the read access is not registered, the write function execution unit 204 finally issues the write access using the write / accumulate access issuing unit 208 and ends the process. (Step S237).

FIG. 23 is a flowchart illustrating an example of the operation of the accumulation function execution unit 206 of the client device 200 of the distributed system 1 according to the present embodiment.
As in the case of the write function, the accumulation function execution unit 206 checks the access waiting list 104 for the access waiting for the same key from the back of the access waiting list 104 (step S251). When write access is registered (YES in step S253: state shown in FIG. 18), the accumulation function execution unit 206 executes accumulation calculation based on the value of the registered write access and registers as a result. The written write access value is overwritten (step S255). For example, when the value specified in the light function is 1 and the accumulation function is to add 2, the accumulation function execution unit 206 changes the value specified in the light function to 3. Then, the accumulation function execution unit 206 ends this process.

  If the accumulation access is registered (YES in step S257: state of FIG. 19), the accumulation function execution unit 206 first checks whether or not the calculation type is the same as that registered (step S259). If the calculation types are not the same (NO in step S259), the accumulation function execution unit 206 registers the accumulation access behind the access waiting list 104 (step S261). Then, the accumulation function execution unit 206 ends this process. If the calculation types are the same (YES in step S259), the accumulation function execution unit 206 performs an accumulation calculation based on the registered accumulation access value and the function argument value, and the result The value of the accumulated access registered in (1) is overwritten (step S263). Then, the accumulation function execution unit 206 ends this process.

  For example, when accumulate (add, Key1, 2) is executed in a state where accumulate (add, Key1, 1) is registered (YES in step S257 and YES in step S259), the accumulation function execution unit 206 Then, the registered accumulation access is changed to accumulate (add, Key1, 3) (step S263).

  If read access is not registered (NO in step S265), the accumulation function execution unit 206 issues an accumulation access using the write / accumulate access issuing unit 208 (step S273). Then, the accumulation function execution unit 206 ends this process. When read access is registered (YES in step S265), that is, in the state of FIG. 20 or FIG. 2, the accumulation function execution unit 206 checks whether this read access can be overtaken (step) S267).

  If the overtaking is not possible (NO in step S267), the accumulation function execution unit 206 registers the accumulation access after the read access (step S261). Then, the accumulation function execution unit 206 ends this process. If the overtaking is possible (YES in step S267), the accumulation function execution unit 206 returns to step S251 for the element immediately before the access waiting for the same key in the access waiting list 104, and performs the same processing as above. Repeat (step S271). When the processing is completed for all the elements that are waiting for the same key in the access waiting list 104, the accumulation function executing unit 206 ends the process after repeating the process for the last empty element.

  For example, when the accumulation function having Key1 as an argument is executed from the state of FIG. 20 (YES in step S265), the read access at the end of the access waiting list 104 can be overtaken (YES in step S267). The rate function execution unit 206 repeats the process while paying attention to the accumulative access immediately before the end of the access waiting list 104 (step S271). When the processing is completed for all the elements that are waiting for the same key in the access waiting list 104, the accumulation function executing unit 206 ends the process after repeating the process for the last empty element.

  When the accumulation function is executed from the state shown in FIG. 2 (YES in step S265 and NO in step S267), the accumulation function executing unit 206 is empty because the next element of interest is not in the access waiting list 104. Perform the next iteration on the element. Here, since any of the write access, the accumulative access, and the read access is not registered (NO in step S253, NO in step S257, and NO in step S265), the accumulating function execution unit 206 performs the final operation. Thus, the write / accumulate access issuing unit 208 is used to issue an accumulative access and the process is terminated (step S273).

  FIG. 24 is a flowchart illustrating an example of the operation of the read access issuing unit 110 of the client device 200 of the distributed system 1 according to the present embodiment. The write / accumulate access issuing unit 208 simply issues to a server in charge of write access and accumulative access, but the read access issuing unit 110 is called by specifying a target server. The operation like this is performed.

First, the read access issuing unit 110 refers to the designated server from the top of the access waiting list 104 and repeatedly executes the following processing in order from the top standby read access (step S281). The read access issuing unit 110 issues a read access for each of the plurality of read accesses in order, and receives a value corresponding to each key (step S283). When required by the determination unit 210, the read access issuing unit 110 records this issuance time in the access waiting list 104.
Then, the read access issuing unit 110 executes the registered callback using the obtained key and value, and deletes it from the access waiting list 104 (step S285). If there is a write access waiting for the issuance of the read access for the key issued in step S283 behind the access waiting list 104 (YES in step S287), the read access issuing unit 110 reads the write / accumulate access issuing unit. A write access is issued using 208 (step S289).

Similarly, when there is an accumulating access (YES in step S291), the read access issuing unit 110 issues an accumulating access using the write / accumulate access issuing unit 208 (step S293). In this case, the read access issuing unit 110 uses the key and value obtained by issuing the read access, and calculates a value corresponding to the key after the accumulative access (step S295). If neither write access nor accumulated access exists (NO in step S287 and NO in step S291), the read access issuing unit 110 calculates using the key and value received in step S283.
Then, the read access issuing unit 110 repeatedly executes the process for the read / write / accumulate access of the key later (YES in step S297) using the key and value values (step S298). If it does not exist (NO in step S297), the read access issuing unit 110 ends this process.
In the iterative process, if read access exists later (YES in step S299), the process returns to step S285, and if no read access exists (NO in step S299), the process returns to step S287.

  For example, if the read access is issued to the distributed table server S1 from the state of FIG. 20, the read access issuing unit 110 executes the accumulation process using the obtained value for Key1, and the value after the calculation Is obtained as the value for Key1, the subsequent read access is repeated.

  Next, the operation of the access issue process determined by the determination unit 210 of the client device 200 of this embodiment will be described below. Each of the first to seventh methods will be described with reference to FIGS. 7 to 16 showing the processing procedure of the determining unit 108 of the embodiment. The determining unit 210 basically operates in the same manner as the determining unit 108 of the above embodiment.

  In the determination process of the first method shown in FIG. 7, first, the determination unit 210 receives a notification registered in the access waiting list 104 from the read function (step S121). When this notification is received (YES in step S121), this processing is started, and the determination unit 210 refers to the access waiting list 104 to check whether there is a server having a certain number or more of read access in charge (step). S123). If there is, the read access issuing unit 110 issues a read access to the server (step S125). The operation of the read access issuing unit 110 is actually the above-described operation described with reference to FIG. 24. However, the detailed description will be omitted below, and it will be described as simply issuing a read access.

  In the determination process of the second method shown in FIG. 8, the determination unit 210 first receives a notification registered in the access waiting list 104 from the read function (step S121). When this notification is received (YES in step S121), this process starts, and the determination unit 210 checks whether the amount of memory consumed in the access waiting list 104 is equal to or larger than a certain amount (step S131). If it is equal to or greater than a certain amount (YES in step S131), the read access issuing unit 110 issues a read access to the server with the largest number of access waits (step S133).

  In the determination process of the third method shown in FIG. 9, the determination unit 210 first receives a notification registered in the access waiting list 104 from the read function (step S121). When this notification is received (YES in step S121), this process starts, and the determination unit 210 checks whether the amount of memory consumed in the access waiting list 104 is equal to or larger than a certain amount (step S131). If it is a certain amount or more (YES in step S131), the read access issuing unit 110 issues a read access to the server having the shortest value acquisition time per key (step S141). The value acquisition time per key can be obtained by the above-described equation (1).

  In the determination process of the fourth method shown in FIGS. 10 to 12, the determination unit 210 receives a notification registered in the access wait list 104 from the read function (YES in step S121 in FIG. 10) or waits for access from the read function. Either the notification registered in the list 104 is received a predetermined number of times (YES in step S121 in FIG. 11 and YES in step S161), or a notification is received at regular intervals from the timer (YES in step S171 in FIG. 12). This triggers the process. Then, the determining unit 210 refers to the access waiting list 104 and checks whether there is a server whose minimum value acquisition time per key is equal to or less than a threshold (step S151). If there is (YES in step S151), the read access issuing unit 110 issues a read access to the server (step S125).

  In the determination process of the fifth method shown in FIG. 13, the determination unit 210 receives a notification from the timer every predetermined time (YES in step S171), and there is a server for which read access has not been issued for a predetermined time or more. The previous read issue time recorded in the waiting list 104 is checked (step S181). If there is such a server (YES in step S181), the read access issuing unit 110 issues a read access to the server (step S125).

  In the determination process of the sixth method shown in FIG. 14, the determination unit 210 calls a function that explicitly designates a server that issues read access from a program (step S191). Then, the read access issuing unit 110 issues read access to the specified server at the specified timing (step S195).

  In the determination process of the seventh method shown in FIGS. 15 and 16, the determination unit 210 receives the notification registered in the access waiting list 104 from the read function (step S121 in FIG. 15). When this notification is received (YES in step S121 in FIG. 15), the determination unit 210 checks whether the amount of memory consumed in the access waiting list 104 is equal to or larger than a certain amount (step S131 in FIG. 15). If it is a certain amount or more (YES in step S131 in FIG. 15), the read access issuing unit 110 issues read access to the server having the shortest value acquisition time per key (step in FIG. 15). S141). Otherwise (NO in step S131 in FIG. 15), the determination unit 210 checks whether or not the notification registered in the access waiting list has been received a predetermined number of times or more from the read function (step S161 in FIG. 15). Note that this step may be omitted and the process may proceed to the next as it is established.

  If the predetermined number of times has been reached (YES in step S161 in FIG. 15), the determination unit 210 checks whether there is a server whose minimum value acquisition time per key is equal to or less than a threshold (step in FIG. 15). S151). If there is (YES in step S151 in FIG. 15), the read access issuing unit 110 issues a read access to the server (step S125 in FIG. 15).

  Also, in this method, the determination unit 210 receives a notification from the timer at regular intervals (YES in step S171 in FIG. 16), and whether there is a server for which read access has not been issued for a certain period of time or not is in the access waiting list 104. The previous read issue time recorded is checked (step S181 in FIG. 16). If there is such a server (YES in step S181 in FIG. 16), the read access issuing unit 110 issues a read access to the server (step S125 in FIG. 16). If there is not (NO in step S181 in FIG. 16), the determination unit 210 checks whether there is a server whose minimum value acquisition time per key is equal to or less than a threshold (step S151 in FIG. 16). If there is (YES in step S151 in FIG. 16), the read access issuing unit 110 issues a read access to the server (step S125 in FIG. 16).

  In this method, as shown in FIG. 14, the determination unit 210 may call a function that explicitly designates a server that issues read access from a program (step S191). Then, the read access issuing unit 110 issues a read access to the designated server at the designated timing (step S195).

  As described above, according to the distributed system 1 of the embodiment of the present invention, the same effects as those of the above embodiment can be obtained, and efficient processing can be performed together with write access and accumulated access.

  As mentioned above, although embodiment of this invention was described with reference to drawings, these are the illustrations of this invention, Various structures other than the above are also employable.

  For example, the minimum configuration in the embodiment of the present invention may be an information processing apparatus 1000 as shown in FIG. The information processing apparatus 1000 reads the data from a plurality of distributed tables (not shown) that each of the plurality of servers (distributed table server 10) has, and a result of reading based on the key (Key) of read access processing A reception unit 102 that receives a callback that defines a process to be executed using the server, and waits for the received read access processing, and a key and callback as standby read access to be issued later to the corresponding distributed table server 10 The registration unit 106 registered in the access waiting list 104 and the determination of determining the issuing destination distributed table server 10 issuing the standby read access and the issuing timing according to predetermined conditions for the standby read access registered in the access waiting list 104 Unit 108 and the source determined by the determination unit 108 A read access issuance unit 110 that issues a standby read access to the distributed table server 10 at the determined issue timing and receives the access result from the distributed table server 10, and an access result (value) that is received by the read access issue unit 110 And a callback execution unit 112 that executes a callback of the corresponding standby read access registered in the access waiting list 104 and deletes the standby read access of the executed callback from the access waiting list 104. .

The information processing apparatus 1000 in FIG. 27 is realized by any combination of computer hardware and software similar to the computer 60 in FIG. 3 that implements the client apparatus of the above embodiment. FIG. 27 shows a logical functional unit block, not a hardware unit configuration.
In the information processing apparatus 1000 according to the present embodiment, the various units shown in FIG. 27 are implemented as various functions by the CPU of the computer executing various processing operations corresponding to the computer program.

  According to this configuration, the information processing apparatus 1000 does not perform, for each read access, a plurality of read access processes using a read function to the distributed table server 10 appearing at an arbitrary place on the program for an arbitrary program. Instead, by registering in the access waiting list 104 and waiting for a certain number or more, it is possible to execute for each server collectively. In addition, with this configuration, it is possible to reduce the influence on the access processing speed due to the latency at the time of read access, compared to the case where multiple read access processes appearing on the program are processed each time, and to increase the speed. Can do.

  While the present invention has been described with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

Hereinafter, examples of the reference form will be added.
1. Receiving means for receiving a key for read access processing for reading data from a plurality of distributed tables respectively possessed by a plurality of servers, and a callback for determining processing to be executed using a result read based on the key;
Registration means for waiting the read access process received and registering the key and the callback in an access waiting list as standby read access to be issued later to a corresponding server;
For the standby read access registered in the access waiting list, in accordance with a predetermined condition, a determination unit that determines an issue destination server that issues the standby read access and an issue timing;
Read access issuing means for issuing the standby read access to the server of the issue destination determined by the determining means at the determined issue timing and receiving the access result from the server;
Using the access result received by the read access issuing means, the callback of the corresponding standby read access registered in the access waiting list is executed, and the standby read access of the executed callback is waited for the access And a callback execution means for deleting from the list.
2. 1. In the distributed system described in
The registration means classifies the standby read access for each server or each key and registers it in the access waiting list,
The determination unit is a distributed system that determines the issuance timing of the standby read access for each server or each key.
3. 1. Or in the distributed system according to 2,
The distributed system, wherein the determination unit selects a server that is the issue destination of the read access based on an acquisition time of the access result for each key obtained based on performance information of each server.
4). 1. To 3. In any one of the distributed systems,
The distributed system determines that the issue timing is when the number of standby read accesses registered in the access waiting list exceeds a certain number.
5. 1. To 4. In any one of the distributed systems,
The determining means includes
When the amount of memory in the access waiting list exceeds a certain amount, it is determined that the issue timing has been reached,
A distributed system that selects a server with a large number of standby read accesses registered in the access waiting list as the issue destination of the read access when the amount of memory exceeds a certain amount.
6). 1. To 5. In any one of the distributed systems,
The distributed system selects the server to which the read access is not issued by the read access issuing means as the issue destination of the read access for a predetermined time or more.
7). 1. To 6. In any one of the distributed systems,
The accepting means further accepts a write access process for writing a value to the key of the distributed table;
If the standby read access to the same key as the write access process exists with reference to the access waiting list, the callback execution means executes the callback of the standby read access with the value of the write access process And a distributed system for deleting the standby read access from the access waiting list.
8). 7). In the distributed system described in
The accepting unit accepts a prohibition flag that prohibits processing the write access process received after the read access process, together with the key and the callback of the read access process,
The registration means registers the received prohibition flag in the access waiting list together with the key of the standby read access and the callback,
Further, the registration means refers to the access waiting list, and when the prohibition flag of the standby read access for the same key as the accepted write access process is registered, the accepted write access process is made to wait, A distributed system for registering the key and the value in the access waiting list as a standby write access to be issued later to a corresponding server.
9. 3. In the distributed system described in
The determination unit calculates a time for obtaining the access result per key, for each server, and selects a server having a minimum value equal to or less than a threshold value as the issue destination of the read access.
10. 9. In the distributed system described in
The determination means calculates the acquisition time of the access result per one key each time the registration means registers the standby read access in the access waiting list, and A distributed system that makes prior choices.
11. 9. In the distributed system described in
The determination unit calculates the acquisition time of the access result per one key for each predetermined number of registrations of the standby read access to the access waiting list by the registration unit. A distributed system for selecting the issue destination.
12 9. In the distributed system described in
The distributed system is a distributed system that calculates the acquisition time of the access result per one key at regular time intervals, and selects the issue destination each time it is calculated.
13. 1. To 12. In any one of the distributed systems,
The distributed system receives a designation of the issue destination of the read access and the issue timing, and determines the issue destination of the read access and the issue timing according to the designation.

14 Information processing device
Receiving a key for read access processing for reading data from a plurality of distributed tables respectively possessed by a plurality of servers, and a callback for determining processing to be executed using a result read based on the key;
Waiting for the accepted read access processing, registering the key and the callback in the access waiting list as standby read access to be issued later to the corresponding server,
For the read access registered in the access waiting list, in accordance with a predetermined condition, determine a server that issues the standby read access and an issue timing,
Issuing the standby read access at the determined issue timing to the determined server, receiving the access result from the server,
Information processing for executing the callback of the corresponding standby read access registered in the access waiting list using the received access result and deleting the standby read access of the executed callback from the access waiting list Device data processing method.
15. 14 In the data processing method of the information processing device described in
The information processing apparatus is
When the read access processing is accepted, the standby read access is classified and registered in the access waiting list for each server or for each key,
A data processing method of an information processing apparatus for determining the issuance timing of the standby read access for each of the servers or for each key of the read access registered in the access waiting list.
16. 14 Or 15. In the data processing method of the information processing device described in
The information processing apparatus is
A data processing method of an information processing apparatus for selecting a server that is the issue destination of the read access based on an acquisition time of the access result for each key obtained based on performance information of each of the servers.
17. 14 To 16. In the data processing method of the information processing apparatus according to any one of the above,
The information processing apparatus is
A data processing method of an information processing apparatus that determines that the issue timing is when the number of standby read accesses registered in the access waiting list exceeds a certain number.
18. 14 To 17. In the data processing method of the information processing apparatus according to any one of the above,
The information processing apparatus is
When the amount of memory in the access waiting list exceeds a certain amount, it is determined that the issue timing has been reached,
A data processing method of an information processing apparatus that selects a server having a large number of standby read accesses registered in the access waiting list as the issue destination of the read access when the memory amount becomes a predetermined amount or more.
19. 14 To 18. In the data processing method of the information processing apparatus according to any one of the above,
The information processing apparatus is
A data processing method of an information processing apparatus that selects a server to which the read access is not issued for a certain time or more as the issue destination of the read access.
20. 14 Thru 19. In the data processing method of the information processing apparatus according to any one of the above,
The information processing apparatus is
Further accepting a write access process for writing a value to the key of the distributed table;
If the standby read access to the same key as the write access processing exists with reference to the access waiting list, the callback of the standby read access is executed with the value of the write access processing, and the access waiting list A data processing method of an information processing apparatus for deleting the standby read access.
21. 20. In the data processing method of the information processing device described in
The information processing apparatus is
The prohibition flag for prohibiting the write access process received after the read access process from being processed first is received together with the key and the callback of the read access process,
Registering the accepted prohibition flag in the access waiting list together with the key of the standby read access and the callback;
Further, referring to the access waiting list, if the prohibit flag for the standby read access for the same key as the accepted write access process is registered, the accepted write access process is made to wait and the corresponding server is A data processing method of an information processing apparatus for registering the key and the value in the access waiting list as standby write access issued in step (1).
22. 16. In the data processing method of the information processing device described in
The information processing apparatus is
A data processing method of an information processing apparatus that calculates an acquisition time of the access result per one key for each server and selects a server having a minimum value equal to or less than a threshold as the issue destination of the read access.
23. 22. In the data processing method of the information processing device described in
The information processing apparatus is
Each time the standby read access is registered in the access waiting list, the acquisition time of the access result for each key is calculated, and each time it is calculated, data of the information processing apparatus that selects the issue destination Processing method.
24. 22. In the data processing method of the information processing device described in
The information processing apparatus is
Information processing for calculating the acquisition time of the access result per one key every time the registration of the standby read access to the access waiting list is performed a predetermined number of times, and selecting the issue destination each time it is calculated Device data processing method.
25. 22. In the data processing method of the information processing device described in
The information processing apparatus is
A data processing method of an information processing apparatus that calculates an acquisition time of the access result for each key at a certain time, and selects the issue destination each time it is calculated.
26. 14 To 25. In the data processing method of the information processing apparatus according to any one of the above,
The information processing apparatus is
A data processing method of an information processing apparatus that accepts specification of the issue destination of the read access and the issue timing, and determines the issue destination of the read access and the issue timing according to the specification.

27. In a computer that implements an information processing device,
A procedure for receiving a key for read access processing for reading data from a plurality of distributed tables respectively possessed by a plurality of servers, and a callback for determining processing to be executed using a result read based on the key;
A procedure for causing the received read access process to wait and registering the key and the callback in an access waiting list as standby read access to be issued later to a corresponding server,
A procedure for determining a server that issues the standby read access and an issue timing for the standby read access registered in the access waiting list according to a predetermined condition;
A procedure of issuing the standby read access to the determined server at the determined issue timing and receiving the access result from the server;
Executing the callback of the corresponding standby read access registered in the access waiting list using the received access result, and deleting the standby read access of the executed callback from the access waiting list; A program for running
28. 27. In the program described in
A procedure for classifying the standby read access and registering it in the access waiting list for each server or for each key when the read access processing is accepted,
A program for causing a computer to execute a procedure for determining the issuance timing of the standby read access for each of the servers or the keys for the read access registered in the access waiting list.
29. 27. Or 28. In the program described in
A program for causing a computer to execute a procedure for selecting a server to which the read access is issued based on the access result acquisition time per key obtained from the performance information of each server .
30. 27. Thru 29. In any program,
A program for causing a computer to execute a procedure for determining that it is the issue timing when the number of standby read accesses registered in the access waiting list exceeds a certain number.
31. 27. Thru 30. In any program,
A procedure for determining that the issue timing has been reached when the memory amount of the access waiting list exceeds a predetermined amount;
A program for causing a computer to execute a procedure for selecting a server having a large number of standby read accesses registered in the access waiting list as the issue destination of the read access when the memory amount becomes a predetermined amount or more .
32. 27. Thru 31. In any program,
A program for causing a computer to execute a procedure of selecting a server to which the read access has not been issued as the issue destination of the read access for a predetermined time or more.
33. 27. Thru 32. In any program,
A procedure for further receiving a write access process for writing a value to the key of the distributed table;
A step of referring to the access waiting list and executing the callback of the standby read access with the value of the write access process when the standby read access to the same key as the write access process exists;
A program for causing a computer to execute a procedure for deleting the standby read access from the access waiting list.
34. 33. In the program described in
A procedure of accepting a prohibition flag for prohibiting the processing of the write access processing received after the read access processing, together with the key and the callback of the read access processing;
Registering the accepted prohibition flag in the access waiting list together with the key of the standby read access and the callback;
If the prohibit flag of the standby read access for the same key as the accepted write access process is registered with reference to the access waiting list, the accepted write access process is waited and issued to the corresponding server later A program for causing a computer to execute a procedure for registering the key and the value in the access waiting list as standby write access to be performed.
35. 29. In the program described in
A program for causing a computer to execute a procedure for calculating, for each server, an access result acquisition time for each key, and selecting a server having a minimum value equal to or less than a threshold as the issue destination of the read access.
36. 35. In the program described in
Each time the standby read access is registered in the access waiting list, the acquisition time of the access result for each key is calculated, and each time the calculation is performed, a procedure for selecting the issue destination is executed on the computer. Program to let you.
37. 35. In the program described in
A procedure for calculating the acquisition time of the access result per one key every time the registration of the standby read access to the access waiting list is performed a predetermined number of times, and selecting the issue destination each time it is calculated A program that causes a computer to execute.
38. 35. In the program described in
A program for calculating the access result acquisition time per one key at regular intervals, and causing the computer to execute a procedure for selecting the issue destination each time the key is calculated.
39. 27. Thru 38. In any program,
A procedure for accepting designation of the issue destination of the read access and the issue timing;
A program for causing a computer to execute a procedure for determining the issuance destination of the read access and the issuance timing in accordance with the designation.

40. Receiving means for receiving a key for read access processing for reading data from a plurality of distributed tables respectively possessed by a plurality of servers, and a callback for determining processing to be executed using a result read based on the key;
Registration means for waiting the read access process received and registering the key and the callback in an access waiting list as standby read access to be issued later to a corresponding server;
For the standby read access registered in the access waiting list, in accordance with a predetermined condition, a determination unit that determines an issue destination server that issues the standby read access and an issue timing;
Read access issuing means for issuing the standby read access to the server of the issue destination determined by the determining means at the determined issue timing and receiving the access result from the server;
Using the access result received by the read access issuing means, the callback of the corresponding standby read access registered in the access waiting list is executed, and the standby read access of the executed callback is waited for the access An information processing apparatus comprising: callback execution means for deleting from the list.
41. 40. In the information processing apparatus described in
The registration means classifies the standby read access for each server or each key and registers it in the access waiting list,
The information processing apparatus that determines the issuing timing of the standby read access for each server or each key.
42. 40. Or 41. In the information processing apparatus described in
The information processing apparatus that selects the server that is the issue destination of the standby read access based on the access result acquisition time per key that is obtained based on the performance information of each server. .
43. 40. Thru 42. In the information processing apparatus according to any one of the above,
The information processing apparatus, wherein the determining unit determines that the issue timing is when the number of the standby read accesses registered in the access waiting list exceeds a certain number.
44. 40. Thru 43. In the information processing apparatus according to any one of the above,
The determining means includes
When the amount of memory in the access waiting list exceeds a certain amount, it is determined that the issue timing has been reached,
An information processing apparatus that selects a server having a large number of standby read accesses registered in the access waiting list as the issuing destination of the standby read access when the memory amount becomes a predetermined amount or more.
45. 40. To 44. In the information processing apparatus according to any one of the above,
The information processing apparatus, wherein the determination unit selects a server to which the standby read access has not been issued by the read access issuing unit for a predetermined time or more as the issue destination of the standby read access.
46. 40. To 45. In the information processing apparatus according to any one of the above,
The accepting means further accepts a write access process for writing a value to the key of the distributed table;
If the standby read access to the same key as the write access process exists with reference to the access waiting list, the callback execution means executes the callback of the standby read access with the value of the write access process An information processing apparatus that deletes the standby read access from the access waiting list.
47. 46. In the information processing apparatus described in
The accepting unit accepts a prohibition flag that prohibits processing the write access process received after the read access process, together with the key and the callback of the read access process,
The registration means registers the received prohibition flag in the access waiting list together with the key of the standby read access and the callback,
Further, the registration means refers to the access waiting list, and when the prohibition flag of the standby read access for the same key as the accepted write access process is registered, the accepted write access process is made to wait, An information processing apparatus that registers the key and the value in the access waiting list as a standby write access to be issued later to a corresponding server.
48. 42. In the information processing apparatus described in
The information processing apparatus, wherein the determination unit calculates an access result acquisition time for each key for each server, and selects a server having a minimum value equal to or less than a threshold as the issue destination of the read access.
49. 48. In the information processing apparatus described in
The determination means calculates the acquisition time of the access result per one key each time the registration means registers the standby read access in the access waiting list, and An information processing apparatus that performs the previous selection.
50. 48. In the information processing apparatus described in
The determination unit calculates the acquisition time of the access result per one key for each predetermined number of registrations of the standby read access to the access waiting list by the registration unit. An information processing apparatus for selecting the issue destination.
51. 48. In the information processing apparatus described in
The information processing apparatus, wherein the determination unit calculates an acquisition time of the access result for each key at regular time intervals, and selects the issue destination each time it is calculated.
52. 40. To 51. In the information processing apparatus according to any one of the above,
The information processing apparatus, wherein the determination unit receives a specification of the issue destination of the read access and the issue timing, and determines the issue destination of the read access and the issue timing according to the specification.

53. 40. To 52. An information processing apparatus according to any one of the above;
And a plurality of servers each having a plurality of distributed tables accessed by the information processing apparatus.

  This application claims the priority on the basis of Japanese patent application Japanese Patent Application No. 2012-192453 for which it applied on August 31, 2012, and takes in those the indications of all here.

Claims (10)

Receiving means for receiving a key for read access processing for reading data from a plurality of distributed tables respectively possessed by a plurality of servers, and a callback for determining processing to be executed using a result read based on the key;
Registration means for waiting the read access process received and registering the key and the callback in an access waiting list as standby read access to be issued later to a corresponding server;
For the standby read access registered in the access waiting list, in accordance with a predetermined condition, a determination unit that determines an issue destination server that issues the standby read access and an issue timing;
Read access issuing means for issuing the standby read access to the server of the issue destination determined by the determining means at the determined issue timing and receiving the access result from the server;
Using the access result received by the read access issuing means, the callback of the corresponding standby read access registered in the access waiting list is executed, and the standby read access of the executed callback is waited for the access And a callback execution means for deleting from the list. The distributed system according to claim 1,
The registration means classifies the standby read access for each server or each key and registers it in the access waiting list,
The determination unit is a distributed system that determines the issuance timing of the standby read access for each server or each key. The distributed system according to claim 1 or 2,
The distributed system determines that the issue timing is when the number of standby read accesses registered in the access waiting list exceeds a certain number. The distributed system according to any one of claims 1 to 3,
The determining means includes
When the amount of memory in the access waiting list exceeds a certain amount, it is determined that the issue timing has been reached,
A distributed system that selects a server with a large number of standby read accesses registered in the access waiting list as the issue destination of the standby read access when the memory amount becomes a predetermined amount or more. The distributed system according to any one of claims 1 to 4,
The distributed system, wherein the determination unit selects a server that is the issue destination of the standby read access based on an acquisition time of the access result for each key obtained based on performance information of each server. The distributed system according to any one of claims 1 to 5,
The distributed system, wherein the determination unit selects a server to which the standby read access is not issued by the read access issuing unit for a predetermined time or more as the issue destination of the standby read access. The distributed system according to any one of claims 1 to 6,
The accepting means further accepts a write access process for writing a value to the key of the distributed table;
If the standby read access to the same key as the write access process exists with reference to the access waiting list, the callback execution means executes the callback of the standby read access with the value of the write access process And a distributed system for deleting the standby read access from the access waiting list. The distributed system according to claim 7.
The accepting unit accepts a prohibition flag that prohibits processing the write access process received after the read access process, together with the key and the callback of the read access process,
The registration means registers the received prohibition flag in the access waiting list together with the key of the standby read access and the callback,
Further, the registration means refers to the access waiting list, and when the prohibition flag of the standby read access for the same key as the write access process is registered, the write access process waits and the corresponding server is made to wait. A distributed system for registering the key and the value in the access waiting list as standby write access to be issued later. Information processing device
Receiving a key for read access processing for reading data from a plurality of distributed tables respectively possessed by a plurality of servers, and a callback for determining processing to be executed using a result read based on the key;
Waiting for the accepted read access processing, registering the key and the callback in the access waiting list as standby read access to be issued later to the corresponding server,
For the read access registered in the access waiting list, in accordance with a predetermined condition, determine a server that issues the standby read access and an issue timing,
Issuing the standby read access at the determined issue timing to the determined server, receiving the access result from the server,
Information processing for executing the callback of the corresponding standby read access registered in the access waiting list using the received access result and deleting the standby read access of the executed callback from the access waiting list Device data processing method. In a computer that implements an information processing device,
A procedure for receiving a key for read access processing for reading data from a plurality of distributed tables respectively possessed by a plurality of servers, and a callback for determining processing to be executed using a result read based on the key;
A procedure for causing the received read access process to wait and registering the key and the callback in an access waiting list as standby read access to be issued later to a corresponding server,
A procedure for determining a server that issues the standby read access and an issue timing for the standby read access registered in the access waiting list according to a predetermined condition;
A procedure of issuing the standby read access to the determined server at the determined issue timing and receiving the access result from the server;
Executing the callback of the corresponding standby read access registered in the access waiting list using the received access result, and deleting the standby read access of the executed callback from the access waiting list; A program for running

Images