JP3729573B2 - Processing takeover method for parallel processing system - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a process takeover method for a parallel processing system to which a function is added to execute an operation when an alternative apparatus is intensively operated when a parallel processing apparatus fails.
[0002]
In order to improve the processing speed, a parallel processing apparatus that executes a plurality of processes simultaneously is used. In such a parallel processing device, since a plurality of processes are executed simultaneously, the same resource (data) may be accessed simultaneously. In order to prevent such resource competition, exclusive control of resources is performed.
[0003]
However, exclusive control allows one process to use resources, but makes other processes wait. For this reason, from the viewpoint of speed improvement, a short exclusive control time is required.
[0004]
[Prior art]
FIG. 4 is an explanatory diagram of parallel processing, and FIG. 5 is an explanatory diagram of exclusive control.
[0005]
As shown in FIG. 4, in the parallel processing, the first processing and the second processing are performed in parallel. In such parallel processing, the same resource may be accessed simultaneously by a plurality of processes.
[0006]
For example, as shown in FIG. 4, a first process for moving 1000 yen from a bank A account to a bank B account and a second process for calculating the total amount of the bank A account and the bank B account When running simultaneously, the bank A account and the bank B account may be accessed simultaneously.
[0007]
In such a case, the first process first deducts 1000 yen from the bank A account (balance 1000 yen), and then the second process the bank A account and the bank B account (balance 0 yen). ), And when the process of adding 1,000 yen to the bank B account is performed in the first process, an error occurs in the result of the second process.
[0008]
That is, when a resource is changed by a certain process, consistency may be destroyed.
[0009]
In order to maintain this consistency, exclusive control is performed so that other processes do not access the resource while one process is being performed.
[0010]
As shown in FIG. 5, in the example of the process of FIG. 4, for the first process of transferring deposits from account to account, first, the data of the two accounts are exclusively controlled and the deposit transfer process is performed. Then, the exclusive control of the two accounts is canceled and the second process is started.
[0011]
While this exclusive control is being performed, other processes are kept waiting, and transient data reading can be prevented.
[0012]
FIG. 6 is an explanatory diagram of an alternative system.
[0013]
On the other hand, the parallel processing device 1 may be down due to a failure. In the event of this failure, it is necessary to add a fault tolerance function so that the system can continue operation. Therefore, as shown in FIG. 6, an alternative device (standby device) 2 is prepared for the first parallel processing device (active device) 1. The alternative device 2 is a parallel processing device. Reference numeral 3 denotes a file device.
[0014]
In this parallel processing system, when fault tolerance is added, the standby unit 2 needs to take over the data of the active unit 1 when the active unit 1 fails. For this reason, after the active machine 1 exclusively controls resources, it is necessary to change the resource contents and transmit the changed contents to the standby machine 2 and then release the exclusive resource control.
[0015]
Conventionally, the following data transfer methods have been considered.
[0016]
FIG. 7 is an explanatory diagram of the first prior art, and FIG. 8 is a processing flowchart of the first prior art.
[0017]
If the exclusive control time is long, the processing speed decreases. For this reason, in the case of a resource that is frequently referred to, there is a resource that does not perform exclusive control for maintaining consistency while allowing a temporarily inconsistent state to exist.
[0018]
Examples of such resources include the number of empty blocks in the UNIX file system and the file link count value.
[0019]
Conventional resource exclusive control will be described with reference to an example of file extension processing shown in FIG.
[0020]
First, the number of empty blocks is changed without performing exclusive control. In file extension, the number of free blocks is decreased by “1”. Next, an empty block bitmap (empty block list) and a block list are read from the file apparatus 3 accordingly. The free block bitmap manages blocks that are not used in any file. The block list is for managing blocks on the disk storing file contents.
[0021]
Next, the exclusive control device is requested to perform exclusive control of the empty block bitmap and the block list. Then, after the exclusive control of the empty block bitmap and the block list, the empty block bitmap and the block list are changed. Then, the exclusive control device is requested to release exclusive control. As a result, the exclusive control of the empty block bitmap and the block list is released.
[0022]
In this example, since the number of empty blocks can be determined by referring to the empty block bitmap and the block list, it is not necessary to exclusively control the number of empty blocks. That is, data contradiction is temporarily allowed in the number of free blocks. That is, the change of the number of free blocks and the data read process on the disk, which is a process accompanying this change, are excluded from exclusive control. As a result, the time for exclusive control is shortened by the time required for changing the number of empty blocks and the time required for reading the empty block bitmap and block list.
[0023]
This is generally shown in FIG. Here, the consistency ignoring resource is the number of empty blocks, and the other resources are an empty block bitmap and a block list.
[0024]
There is known a technique for recovering a resource when a failure occurs before the system returns to a consistent state after changing the resource ignoring the consistency. That is, in the middle of restarting, the consistency check program is activated to restore consistency. For example, in the case of the UNIX file system, a program called fsck does this.
[0025]
In order to give fault tolerance to the parallel processing system, it is conceivable to use this method when taking over from the active machine 1 to the standby machine 2. That is, it is conceivable that when the active machine 1 fails, the standby machine 2 is run with a consistency check program to restore the consistency of resources and resume processing of the standby machine 2.
[0026]
On the other hand, as another conventional method, as shown in FIG. 9, resources that ignore consistency are exclusively controlled to maintain consistency. Then, the contents of the resource are transferred from the active machine 1 to the standby machine 2. The standby machine 2 has a method of resuming processing based on the sent resources when the active machine 1 fails.
[0027]
[Problems to be solved by the invention]
The prior art described above has the following problems.
[0028]
When the first conventional technique for running the consistency check program shown in FIGS. 7 and 8 is applied at the time of failure of the active machine 1, the standby machine 2 resumes the process until the consistency check program is started and terminated. I had to wait. Since the consistency check program examines all the contents of the file device 3 and restores the consistency of resources, there is a problem that the processing takes time and the restart of the standby machine 2 is delayed.
[0029]
In the second prior art shown in FIG. 9, since the resources whose consistency may be ignored are exclusively controlled, the time required for the exclusive control is unnecessarily long. For this reason, the problem that the processing speed of parallel processing falls has arisen.
[0030]
Further, it is conceivable to write back resources to the file device 3 for each process. However, since it takes time, it is desirable to write to the file device 3 about once every 30 seconds, for example. For this reason, writing back resources to the file device 3 each time processing significantly reduces the parallel processing speed.
[0031]
An object of the present invention is to provide a process takeover method of a parallel processing system for preventing a reduction in the speed of parallel processing and preventing a process restart of a standby machine from being delayed.
[0032]
Another object of the present invention is to provide a process takeover method for a parallel processing system that can prevent unnecessary exclusive control and can eliminate the need for processing by a consistency check program.
[0033]
[Means for Solving the Problems]
FIG. 1 is a principle diagram of the present invention.
[0034]
The parallel processing system includes a parallel processing device 1 that executes a plurality of processes in parallel, a processing device 2 that performs processing with a tight operation when a failure occurs in the parallel processing device 1, and the parallel processing device 1 and the processing device 2. And data storage means 3 for storing data for file processing and file contents . The parallel processing apparatus 1, after changing the first data for the file processing, by changing the second data related to said first data, and performs file processing of the data storage means 3 is there.
[0035]
Then, the process takeover method is such that the parallel processing device 1 temporarily allows the first data and the first data that are the number of free blocks so as to allow a state inconsistent with the second data that is the free block list and the block list . After the first data in the main memory is changed without exclusive control of the file processing using the data, the step of reading the second data from the data storage means 3 to the main memory, and the parallel processing device 1 2 for exclusive control of the second data and the third data, which is the shadow object of the number of empty blocks of the first data for indicating the same content as the first data, A step of transferring at least the third data to the processing device 2 after changing the second data and the third data; Has a step of releasing the exclusive control of the third data processing apparatus 2, when a failure occurs in parallel processing apparatus 1, the third data transferred as the initial value, and performing taking over the process .
[0036]
In the present invention, data (resources) that allow the existence of a temporarily inconsistent state are doubled. Then, one of the first data (first data) is changed without exclusive control so as to allow a temporarily inconsistent state, and the time-consuming second data reading process is performed accordingly. The other (third data) is exclusively controlled so that the second data is changed so as to maintain consistency with the second data. Thereby, the third data is consistent with the second data.
[0037]
This third data (referred to as a shadow object of the first data) is transferred to the processing device 2. As a result, the processing device 2 can restart the processing with the third data having no contradiction as an initial value when a failure occurs in the parallel processing device 1.
[0038]
For this reason, since it is not necessary to run the consistency check program, the standby unit (processing device) 2 can prevent a delay in processing restart when the active unit (parallel processing unit) 1 fails.
[0039]
In addition, since there is no contradiction with the third data, the change of the first data that allows a temporarily inconsistent state and the reading of the second data triggered by it can be excluded from exclusive control. The exclusive control time in parallel processing can be shortened, and the processing can be resumed immediately upon failure.
[0040]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 2 is a block diagram of an embodiment of the present invention, and FIG. 3 is a process flow diagram of an embodiment of the present invention.
[0041]
In FIG. 2, the parallel processing device 1 that is the active machine detects a failure by a parallel processing processor 10 that performs parallel processing, a main memory 11, an exclusive control mechanism 12 that performs exclusive control, a transfer mechanism 13 that performs transfer, And a failure detection mechanism 14 for notification.
[0042]
The parallel processor 10 is composed of, for example, a plurality of processors and executes a plurality of processes in parallel. The main memory 11 holds data necessary for the execution of the parallel processor 10.
[0043]
The main memory 11 includes a free block list (bitmap) 11a for managing blocks not used in the file, a block list 11b for managing blocks on the disk storing the file contents, and the number of free blocks. The number of free blocks 11c shown and the shadow object 11d with the number of free blocks showing the number of free blocks are held.
[0044]
That is, the resource indicating the number of free blocks is duplicated in the number 11c of free blocks and the shadow object 11d of the number of free blocks.
[0045]
The exclusive control mechanism 12 performs exclusive control of the resources of the main memory 11. The transfer mechanism 13 is for exchanging data with the standby processing device 2. The failure detection mechanism 14 detects a failure of the active machine 1.
[0046]
The processing device 2 that is a standby machine is also configured by a parallel processing device. The parallel processing device 2 includes a parallel processing processor 20 that performs parallel processing, a main memory 21, an exclusive control mechanism 22 that performs exclusive control, a transfer mechanism 23 that performs transfer, and a failure detection mechanism 24 that detects and notifies a failure. Have
[0047]
The parallel processor 20 is also composed of a plurality of processors, for example, and executes a plurality of processes in parallel. The main memory 21 holds data necessary for the execution of the parallel processing processor 20.
[0048]
The main memory 21 includes a free block list (bitmap) copy 21a for managing blocks not used in the file, a block list copy 21b for managing blocks on the disk storing the file contents, A shadow object copy 21c having the number of empty blocks indicating the number of empty blocks is held.
[0049]
The exclusive control mechanism 22 performs exclusive control of the resources of the main memory 21. The transfer mechanism 23 is for exchanging data with the active processing device 1. The failure detection mechanism 24 detects a failure of the standby device 2.
[0050]
The file device 3 includes a free block list (bitmap) for managing blocks not used in the file, a block list for managing blocks on the disk storing the file contents, and a free space indicating the number of free blocks. Holds the number of blocks. The file device 3 is composed of a disk storage device, and is connected to both the processing devices 1 and 2.
[0051]
The operation will be described with reference to the processing flowchart of FIG. FIG. 3 shows a process at the time of file extension.
[0052]
(S1) The parallel processor 10 changes the number of free blocks 11c in the main memory 11. For example, when the file is expanded, the number of free blocks 11c is decreased by “1”.
[0053]
(S2) When the number of free blocks 11c is changed, the parallel processor 10 reads the free block bitmap (list) and the block list from the file device 3 and expands them in the main memory 11. This reading process takes time.
[0054]
(S3) The parallel processor 10 requests the exclusive control mechanism 12 to perform exclusive control of the empty block bitmap (list) 11a, the block list 11b, and the shadow object 11d having the number of empty blocks in the main memory 11. Thereby, the free block bitmap (list) 11a and the block list 11b of the main memory 11 are exclusively controlled. Next, the shadow objects 11d corresponding to the number of empty blocks in the main memory 11 are exclusively controlled.
[0055]
(S4) Next, the parallel processor 10 finds an empty block from the empty block bitmap 11a. Then, after finding an empty block and changing the empty block list 11a, the block is inserted into the block list 11b. Further, the shadow object 11d having the number of free blocks is reduced by “1”. Then, the free block list, the changed portion of the block list, and the shadow object of the number of free blocks are transferred to the standby processing device 2 via the transfer mechanism 13.
[0056]
In this manner, the processing device 2 updates the free block list copy 21a, the block list copy 21b, and the shadow object copy 21c of the number of free blocks in the main memory 21.
[0057]
(S5) After the transfer, the parallel processor 10 requests the exclusive control mechanism 12 to release exclusive control of the free block bitmap (list) 11a, the block list 11b, and the shadow object 11d having the number of free blocks in the main memory 11. . As a result, the exclusive control of the free block bitmap (list) 11a, the block list 11b, and the shadow object 11d with the number of free blocks is released.
[0058]
As described above, the standby processing apparatus 2 has the updated free block list copy, block list copy, and shadow object copy of the number of free blocks in the main memory 21, so that the active processing is performed. When a failure detection notification is received from the failure detection mechanism 14 of the apparatus 1 via the failure detection mechanism 24, the processing can be resumed immediately.
[0059]
In this parallel processing device 1, when two file expansion processes run simultaneously, the number of free blocks is reduced by two at first, but the shadow object of the number of free blocks changes the free block list and adds the block list. It is changed at the same time. That is, the shadow object having the number of free blocks maintains consistency with the change of the free block list and the addition of the block list. Then, the data is sent to the standby processing device 2 during exclusive control.
[0060]
When the active processing device 1 fails, the standby processing device 2 uses the contents of the shadow object of the number of free blocks as the initial value of the number of free blocks.
[0061]
In this way, in file processing, the number of empty blocks that temporarily allow contradiction is doubled, and after updating the number of one empty block, the empty block list and block list are read out. Then, the other free block count (shadow object of the free block count) is exclusively controlled and updated together with the update of the free block list and the block list. There are no free blocks.
[0062]
Since the number of free blocks without contradiction is transferred to the standby system, the standby system can immediately resume the file processing when the active system fails. Therefore, it can be executed without increasing the time required for exclusive control in parallel processing. In addition, when the process is resumed in the standby system, the process can be resumed without running the consistency check program, and a delay in the process resume can be prevented.
[0063]
In addition, during exclusive control, resources that are exclusively controlled are transferred to the standby system, so that even if parallel processing is performed, the standby system can obtain resources with consistent contents.
[0064]
Furthermore, since it can be realized only by duplicating data and exclusive control of one data, it can be realized easily.
[0065]
In addition to the above-described embodiments, the present invention can be modified as follows.
[0066]
(1) In the above-described embodiment, the number of empty blocks has been described as an example of a resource (data) that temporarily permits contradiction, but it can also be applied to a file link count value or the like.
[0067]
{Circle around (2)} The parallel processing apparatus has been described with the configuration of the example of FIG. 2, but various apparatuses can be applied.
[0068]
As mentioned above, although this invention was demonstrated by embodiment, a various deformation | transformation is possible within the range of the main point of this invention, and these are not excluded from the scope of the present invention.
[0069]
【The invention's effect】
As described above, the present invention has the following effects.
[0070]
(1) Hold double resources that allow temporary contradiction, update one without exclusive control, and perform exclusive control on the other, change along with other related resources, and transfer to the standby system Therefore, it is possible to give a fault tolerance function to the parallel processing system without reducing the operation speed in the parallel processing.
[0071]
(2) In addition, when the active system fails, data can be taken over without running the consistency check program in the standby system, so that it is possible to prevent delays in processing resumption.
[Brief description of the drawings]
FIG. 1 is a principle diagram of the present invention.
FIG. 2 is a configuration diagram of an embodiment of the present invention.
FIG. 3 is a process flow diagram of one embodiment of the present invention.
FIG. 4 is an explanatory diagram of parallel processing.
FIG. 5 is an explanatory diagram of exclusive control.
FIG. 6 is an explanatory diagram of an alternative system.
FIG. 7 is an explanatory diagram of a first prior art.
FIG. 8 is a processing flowchart of the first prior art.
FIG. 9 is an explanatory diagram of a second prior art.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Parallel processing apparatus 2 Alternative processing apparatus 3 File apparatus 10, 20 Parallel processor 11, 21 Main memory 12, 22 Exclusive control mechanism 13, 23 Transfer mechanism 14, 24 Failure detection mechanism

Claims (3)

A parallel processing device that executes a plurality of processes in parallel, a processing device that executes processing by pulling operations when a failure occurs in the parallel processing device, and the parallel processing device and the processing device that are accessed to perform file processing Data storage means for storing the data and file contents of
A parallel processing system in which the parallel processing device changes the second data related to the first data after changing the first data for the file processing, and performs the file processing of the data storage means In the process takeover method for
The parallel processing device uses the first data and the first data that are the number of free blocks so as to allow a state inconsistent with the second data that is temporarily a free block list and a block list . Reading the second data from the data storage means to the main memory after changing the first data in the main memory without exclusively controlling file processing;
The parallel processing device exclusively controlling the second data and the third data which is a shadow object of the number of empty blocks of the first data for indicating the same content as the first data;
After the parallel processing device changes the second data and the third data in the main memory, transferring at least the third data to the processing device;
The parallel processing device releasing exclusive control of the second data and the third data;
The processing apparatus has a step of taking over and executing the process using the transferred third data as an initial value when a failure occurs in the parallel processing apparatus. The processing takeover method of the parallel processing system according to claim 1 ,
The parallel processing device includes:
The parallel processing characterized by changing the shadow object of the number of free blocks so as to maintain consistency with the change of the free block list and the addition of the block list when changing the shadow object of the number of free blocks System processing takeover method. In the processing takeover method of the parallel processing system according to claim 1 or 2 ,
The parallel processing apparatus transfers the third data to the processing apparatus and transfers the second data to the processing apparatus.

Images