JP4773715B2 - How to get checkpoint - Google Patents

Description

The present invention relates to a checkpoint acquisition method by an information processing apparatus.

  A parallel program executed on a cluster system constructed using a large number of host computers stops during execution, even if some of the host computer hardware resources fail. Is wasted.

  A checkpoint / restart function has been devised and incorporated in many cluster systems in order to facilitate recovery in the event of such an accident and not to waste calculation results until the accident occurs. .

  Here, the checkpoint / restart function means that the execution status at any point during the execution of the program is saved on the local disk of the host computer, and the host computer executing the program unexpectedly fails. In this case, the function restores the state of the program from the saved information without restarting the program from the beginning and restarts the execution. Conventionally, checkpoints are collected by the following method.

  First, a checkpoint collection request is sent to each process being executed by a timer function at regular intervals or an instruction from an operator or the like.

  Next, the process that has received the checkpoint collection request suspends execution so far and calls the tickpoint collection process.

  Next, in the checkpoint collection process, the execution state of the process (for example, the memory image, the descriptor of the resource being used, etc.) is written on the local disk of the host computer.

Next, when the checkpoint collection process ends, the process resumes execution that was temporarily suspended.
Japanese Patent Laid-Open No. 11-45229

  However, in the processing as described above, the checkpoint collection processing is interrupted regardless of the state of the parallel program. Therefore, if the process cannot call the checkpoint collection processing normally, for example, the process is a thread. There was a problem that checkpoints could not be collected.

  Also, since the timing at which the checkpoint collection request arrives at each process of the parallel program varies, depending on the timing, the recorded execution state is restored, and the parallel program is executed from the recorded location. When restarting (restarting), there was a problem of collecting checkpoints in a state where a deadlock occurred.

  FIG. 1 is a diagram illustrating a conventional problem. As shown in FIG. 1, for example, the process 1 running on the host computer 1 receives a checkpoint collection request after barrier synchronization, collects the checkpoint, and operates on the host computer 2. The process 2 that has received the checkpoint collection request before barrier synchronization has collected the checkpoint. In such a case, there is a high possibility that a deadlock will occur when the recorded execution state is restored and the execution of the parallel program is restarted (restarted) from the recorded position.

  The present invention has been made in view of the above points, and an object of the present invention is to safely collect checkpoints by capturing timing at which inconvenience such as deadlock does not occur.

  Another object of the present invention is to enable checkpoint collection even in a parallel program in which each process has a thread.

Therefore, in order to solve the above-described problem, the present invention detects a checkpoint acquisition request, a child process operating as a parallel program sets a flag in an information processing apparatus in which a parent process operates, and the checkpoint acquisition request When establishment of barrier synchronization that is performed independently of the process is detected, it is determined whether or not a flag is set for all processes that are targets of establishment of barrier synchronization in the information processing apparatus in which the parent process operates. Indicates a positive determination result, checkpoint acquisition is performed for all processes for which the current barrier synchronization is established, and when the determination indicates a negative determination result, the next barrier synchronization establishment is performed. If it is determined that the flag is set for all processes for which barrier synchronization is to be established, A synchronization establishment to perform acquisition of checkpoint for all processes that have been made, characterized in that to perform the processing to the information processing apparatus.

According to the present invention, when a checkpoint acquisition request is detected, a child process operating as a parallel program sets a flag in the information processing apparatus in which the parent process operates, and is performed independently of the checkpoint acquisition request. When the establishment of barrier synchronization is detected, the information processing apparatus in which the parent process operates determines whether or not a flag is set for all processes targeted for establishment of barrier synchronization, and the determination is positive. If all the processes that have been established for barrier synchronization this time are checked, checkpoint acquisition is performed. When it is determined that the flag is set for all processes, all the next establishment of barrier synchronization is performed. Process to perform acquisition of checkpoint for, by executing the processing to the information processing apparatus can be collected safely checkpoint captures timing disadvantage of deadlock and the like does not occur.

  According to the present invention, it is possible to safely collect checkpoints by capturing a timing at which inconvenience such as deadlock does not occur.

  Furthermore, according to the present invention, it is possible to collect checkpoints even in a parallel program in which each process has a thread.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 2 is a diagram illustrating an example of a cluster system. As shown in FIG. 2, in the cluster system according to the present invention, a plurality of host computers 1 that execute parallel programs and a management host 2 are connected via a network.

  A hardware configuration of an example of the host computer 1 is shown in FIG. FIG. 3 is a hardware configuration diagram of an example of a host computer.

  The hardware configuration of the host computer 1 shown in FIG. 3 includes a drive device 13, a ROM (Read Only Memory) 15, a RAM (Random Access Memory) 16, and a CPU (Central Processing) that are mutually connected by a bus. (Unit) 17, interface device 18, and HDD (Hard Disk Drive) 19.

  The interface device 18 is an interface for connecting the host computer 1 to a network or the like.

  A program of the host computer 1 (hereinafter simply referred to as a parallel computer program) is transferred from the management host 2 to the HDD 19 via the network, for example. Alternatively, the parallel computer program may be provided to the host computer 1 by the recording medium 14 such as a CD-ROM or downloaded through a network. The recording medium 14 is set in the drive device 13, and the parallel calculation program is installed from the recording medium 14 to the HDD 19 via the drive device 13.

  The ROM 15 stores data and the like. The RAM 16 reads and stores a parallel calculation program or the like from the HDD 19 when the host computer 1 is started, for example. The CPU 17 executes processing according to a parallel calculation program or the like read and stored in the RAM 16.

  The functional configuration of an example of the host computer 1 is shown in FIG. FIG. 4 is a functional configuration diagram of an example of the host computer.

  As shown in FIG. 4, the host computer 1 includes an OS (Operating System) 20, PC cluster system software 30, a language library 50, and a language object 60. The OS 20 includes a thread library 21, and the PC cluster system software 30 includes a checkpoint collection processing unit 30. The checkpoint collection processing unit 30 includes a signal handler 41, a flag area 42, and a checkpoint collection procedure. Part 43. The language library 50 includes a barrier synchronization processing unit 51 and a thread procedure unit 52.

  Here, the signal handler 41 registers itself with the OS 20 in the initialization of the PC cluster system software 30 and the like, and secures and initializes a flag in the flag area 42 of the RAM 16 or the like. Further, when the signal handler 41 receives a checkpoint collection request from the operator via the management host 2 or the like, or receives a checkpoint collection request from the OS 20 due to an interval time interruption, the signal handler 41 turns ON (ON). ) Is set.

  The checkpoint collection procedure unit 43 saves the execution state of the process at the requested time on the HDD 19 of the host computer 1 or the like.

  For example, the barrier synchronization processing unit 51 performs barrier synchronization with processes included in other host computers 1, and when the barrier synchronization is performed, the flag in the flag area 42 is set to ON, and all of the synchronization should be performed. If it is determined that the flag in the flag area 42 of each host computer 1 in which another process operates is also set to ON, the checkpoint collection procedure unit 43 is requested to collect the checkpoint.

  Here, an example of the timing for executing the checkpoint collection process is shown in FIG. FIG. 5 is a diagram illustrating an example of timing for executing the checkpoint collection process.

  As shown in FIG. 5, when the barrier synchronization processing unit 51 establishes barrier synchronization with another process, the barrier synchronization processing unit 51 performs a checkpoint even in one of the other processes (process 3 in the example of FIG. 5). If it is determined that the request has not been received, that is, the flag in the flag area 42 of the host computer 1 on which the process is being executed is not set to ON, checkpoint collection is not requested and barrier synchronization with another process is performed again. Is established, each process determines whether or not a checkpoint request has been received, and the checkpoint request has been received, that is, the flag in the flag area 42 of the host computer 1 on which each process is being executed is ON. If it is determined that is set, checkpoint collection procedure unit 43 is requested to collect checkpoints.

  Returning to FIG. 4, when the process has a plurality of threads, the barrier synchronization processing unit 51 instructs the thread procedure unit 52 to stop sub-threads other than the main thread before requesting the collection of a checkpoint. Request. The barrier synchronization processing unit 51 requests the thread procedure unit 52 to resume the stopped sub thread after the check point collection procedure unit 43 has collected the check points.

  The thread procedure unit 52 performs a procedure related to a thread such as calling a library included in the thread library 21 in response to a request.

  An example of the signal handler registration process in the signal handler 41 is shown in FIG. FIG. 6 is a flowchart illustrating an example of the signal handler registration process. The signal handler 41 executes the following processing in initialization of the PC cluster system software.

  In step S <b> 10, the signal handler 41 registers itself with the OS 20. Progressing to step S11 following step S10, the signal handler 41 secures and initializes a flag in the flag area 42 of the RAM 16 or the like.

  An example of flag registration processing in the signal handler 41 is shown in FIG. FIG. 7 is a flowchart illustrating an example of the flag registration process. When the signal handler 41 receives a checkpoint collection request from the operator via the management host 2 or the like, or receives a checkpoint collection request by an interval time interrupt from the OS 20, the signal handler 41 executes the following processing. To do.

  In step S <b> 20, the signal handler 41 sets ON to the flag in the flag area 42.

  Hereinafter, an example of the barrier synchronization processing in the barrier synchronization processing unit 51 and the like is shown in FIG. FIG. 8 is a flowchart illustrating an example of the barrier synchronization process. The barrier synchronization processing unit 51 executes the following processing in response to a synchronization processing request from the language object 60 or the like.

  In step S30, the barrier synchronization processing unit 51 performs barrier synchronization with, for example, processes included in other host computers 1. Progressing to step S31 following step S30, the barrier synchronization processing unit 51 determines whether or not the number of synchronized processes is the same as the number of all processes to be synchronized. If the barrier synchronization processing unit 51 determines that the number of synchronized processes is the same as the number of all processes to be synchronized (YES in step S31), the process proceeds to step S32, where the number of synchronized processes is the number of all processes to be synchronized. If it is determined that they are not the same (NO in step S31), the barrier synchronization process is terminated.

  In step S32, the barrier synchronization processing unit 51 determines whether or not the flag in the flag area 42 is set to ON. If the barrier synchronization processing unit 51 determines that the flag in the flag area 42 is set to ON (YES in step S32), the process proceeds to step S33 and determines that the flag in the flag area 42 is not set to ON ( In step S32, NO), the barrier synchronization processing is terminated.

  In step S33, the barrier synchronization processing unit 51 calls the barrier synchronization processing unit 51 or determines whether the process including the barrier synchronization processing unit 51 being executed is a parent process. If the barrier synchronization processing unit 51 determines that the process that includes the barrier synchronization processing unit 51 that has called or is executing the barrier synchronization processing unit 51 is a parent process (the parent process in step S33), the process proceeds to step S34. If it is determined that the process including the barrier synchronization processing unit 51 that has called or is executing the barrier synchronization processing unit 51 is a child process (child process in step S33), the process proceeds to step S39.

  In step S34, the barrier synchronization processing unit 51 waits for a certain period of time to write flag information indicating that the flag is set ON from all child processes to be synchronized. Progressing to step S35 following step S34, the barrier synchronization processing unit 51 determines whether flag information has been written from all the child processes to be synchronized. If the barrier synchronization processing unit 51 determines that flag information has been written from all child processes to be synchronized (YES in step S35), the process proceeds to step S37, and flag information is written from all child processes to be synchronized. If it is determined that there is no (NO in step S35), the process proceeds to step S36.

  In step S36, the barrier synchronization processing unit 51 returns a checkpoint request refusal that means that checkpoint collection is not permitted to the child process for which flag information has been written. On the other hand, in step S37, the barrier synchronization processing unit 51 replies with a checkpoint request reception that means that checkpoint collection is permitted to all child processes to be synchronized.

  Progressing to step S38 following step S37, the barrier synchronization processing unit 51 executes processing related to the collection of checkpoints.

  On the other hand, in step S39, the barrier synchronization processing unit 51 writes flag information to the parent process. Progressing to step S40 following step S39, the barrier synchronization processing unit 51 waits for a predetermined time for a response (reply) from the parent process related to the writing of flag information.

  Progressing to step S41 following step S40, the barrier synchronization processing unit 51 determines whether or not the process of waiting for a response has timed out. When the barrier synchronization processing unit 51 determines that the process of waiting for a response has timed out (YES in step S41), the barrier synchronization processing unit 51 ends the barrier synchronization process, and determines that there is a response from the parent process before the process of waiting for a response times out. (NO in step S41), the process proceeds to step S42.

  In step S42, the barrier synchronization processing unit 51 determines whether the response from the parent process is a checkpoint request receipt. When the barrier synchronization processing unit 51 determines that the response from the parent process is a checkpoint request reception (YES in step S42), the barrier synchronization processing unit 51 proceeds to step S43 and determines that the response from the parent process is not a checkpoint request reception (step S42). NO in S42), the barrier synchronization process is terminated.

  In step S43, the barrier synchronization processing unit 51 executes processing related to checkpoint collection.

  As shown in FIG. 6 to FIG. 8, when a checkpoint collection request is received, a checkpoint is not collected immediately, but a flag is set to synchronize with other processes to be synchronized. It is determined whether or not the memory in which all other processes to be synchronized are running has a flag, and if the flag is raised, a checkpoint is taken to avoid inconvenience such as deadlock. It is possible to take checkpoints safely by capturing the timing that does not occur.

  An example of checkpoint collection processing in the barrier synchronization processing unit 51 and the checkpoint collection procedure unit 43 is shown in FIG. FIG. 9 is a flowchart illustrating an example of the checkpoint collection process. The barrier synchronization processing unit 51 executes the following process in step S38 and step S43 in FIG.

  In step S50, the barrier synchronization processing unit 51 requests, for example, the thread procedure unit 52 and the like to stop sub threads other than the main thread. Progressing to step S51 following step S50, the barrier synchronization processing unit 51 and / or the thread procedure unit 52 wait for the sub thread to stop.

  Progressing to step S52 following step S51, when the sub-thread is stopped, the barrier synchronization processing unit 51 requests the checkpoint collection procedure unit 43 to collect a checkpoint, and the checkpoint collection procedure unit 43 Execute point collection processing.

  Progressing to step S53 following step S52, when the barrier synchronization processing unit 51 receives a response from the checkpoint collection procedure unit 43 indicating that the checkpoint collection processing is completed, the barrier synchronization processing unit 51 sends a sub-thread to the thread procedure unit 52, for example. Request to resume. Note that the barrier synchronization processing unit 51 may set OFF in the flag in the flag area 42 after resuming the sub thread in step S53.

  As shown in FIG. 9, the sub-thread other than the main thread is stopped, the checkpoint collection process is executed, and after the execution is completed, the stopped sub-thread is restarted so that the process Even in a parallel program, checkpoints can be collected.

  As described above, according to the present invention, it is possible to safely collect checkpoints by capturing the timing at which inconvenience such as deadlock does not occur. Furthermore, according to the present invention, it is possible to collect checkpoints even in a parallel program in which each process has a thread.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims.・ Change is possible.

In addition, the structure like the additional remarks described below can be considered for this invention.
(Appendix 1)
An identification information setting means for setting identification information in the memory in response to a checkpoint collection request;
Synchronization execution means for synchronizing with other processes included in the information processing apparatus or other information processing apparatus connected to the information processing apparatus via a network;
Checkpoint collection means for collecting checkpoints;
Have
The synchronization execution means has identification information set in the memory at the time of synchronization, and identification information is also stored in the memory of each information processing apparatus in which all the other processes to be synchronized operate. If it is determined that the check point is set, the information processing apparatus requests the check point collecting means to collect a check point.
(Appendix 2)
When the process has a plurality of threads, the synchronous execution unit requests the sub thread other than the main thread to stop before requesting the check point collection unit to collect the check point. The information processing apparatus according to attachment 1.
(Appendix 3)
The information processing apparatus according to claim 2, wherein the synchronization execution unit requests the sub thread to resume after the check point collection unit has collected the check point.
(Appendix 4)
If the process that called itself is a parent process, the synchronization execution means waits for a certain period of time on information related to identification information from all child processes to be synchronized, and relates to identification information from all the child processes. Appendices 1 to 3, wherein when the information is received, permission information related to the collection of checkpoints is sent to all child processes, and the checkpoint collection means in the same apparatus as itself is requested to collect the checkpoints. The information processing apparatus according to any one of claims.
(Appendix 5)
If the process that called itself is a child process, the synchronous execution means sends information related to the identification information to the parent process, waits for a reply from the parent process for a certain period of time, and collects checkpoints from the parent process. 5. The information processing apparatus according to any one of appendices 1 to 4, wherein when the permission information is received, the checkpoint collection unit in the same apparatus as itself is requested to collect the checkpoint.
(Appendix 6)
Computer
An identification information setting means for setting identification information in the memory in response to a checkpoint collection request;
Synchronization execution means for synchronizing with other processes included in the information processing apparatus or other information processing apparatus connected to the information processing apparatus via a network;
Checkpoint collection means for collecting checkpoints;
And operate
The synchronization execution means has identification information set in the memory at the time of synchronization, and identification information is also stored in the memory of each information processing apparatus in which all the other processes to be synchronized operate. An information processing program that requests the checkpoint collecting means to collect a checkpoint when it is determined that the checkpoint is set.
(Appendix 7)
Computer
An identification information setting means for setting identification information in the memory in response to a checkpoint collection request;
Synchronization execution means for synchronizing with other processes included in the information processing apparatus or other information processing apparatus connected to the information processing apparatus via a network;
Checkpoint collection means for collecting checkpoints;
And operate
The synchronization execution means has identification information set in the memory at the time of synchronization, and identification information is also stored in the memory of each information processing apparatus in which all the other processes to be synchronized operate. A computer-readable recording medium on which an information processing program is recorded, wherein when it is determined that a checkpoint is set, the checkpoint collection means is requested to collect a checkpoint.
(Appendix 8)
A checkpoint collection method in an information processing device, wherein the information processing device includes:
In response to a checkpoint collection request, an identification information setting stage for setting identification information in a memory;
A synchronization execution stage that synchronizes with other processes included in the information processing apparatus or another information processing apparatus connected to the information processing apparatus via a network;
When it is determined that the identification information is set in the memory after the synchronization execution stage and the identification information is also set in the memory of each information processing apparatus in which all the other processes to be synchronized operate. A checkpoint collection request stage for requesting checkpoint collection to the checkpoint collection means for collecting checkpoints;
A checkpoint collection method, characterized in that:

It is a figure which shows the conventional problem. It is a figure which shows an example of a cluster system. It is a hardware block diagram of an example of a host computer. It is a functional block diagram of an example of a host computer. It is a figure which shows an example of the timing which performs a checkpoint collection process. It is a flowchart which shows an example of a signal handler registration process. It is a flowchart which shows an example of a flag registration process. It is a flowchart which shows an example of a barrier synchronous process. It is a flowchart which shows an example of a checkpoint collection process.

Explanation of symbols

1 Host computer 2 Management host 13 Drive device 14 Recording medium 15 ROM
16 RAM
17 CPU
18 Interface device 19 HDD
20 OS
21 Thread Library 30 PC Cluster System Software 40 Checkpoint Collection Processing Unit 41 Signal Handler 42 Flag Area 43 Checkpoint Collection Procedure Unit 50 Language Library 51 Barrier Synchronization Processing Unit 52 Thread Procedure Unit 60 Language Object

Claims (3)

  When a checkpoint acquisition request is detected, a child process that operates as a parallel program sets a flag in the information processing device in which the parent process operates,
  When the establishment of barrier synchronization that is performed independently of the checkpoint acquisition request is detected, whether or not flags are set for all processes that are targets of establishment of barrier synchronization in the information processing apparatus in which the parent process operates Make a decision,
  If the determination shows a positive determination result, checkpoint acquisition is performed for all processes for which the current barrier synchronization has been established, and if the determination indicates a negative determination result, the next barrier synchronization When it is determined at the time of establishment that the flag is set for all the processes for which barrier synchronization establishment is to be performed, checkpoint acquisition is performed for all the processes for which the next barrier synchronization establishment is performed.
  A checkpoint acquisition method characterized by causing an information processing apparatus to execute processing. If the previous SL process has multiple threads, the before requesting acquisition of checkpoint, the checkpoint acquisition method according to claim 1, wherein the request to stop the sub-thread other than the main thread. After obtaining the pre Symbol checkpoint, the checkpoint acquisition method according to claim 2, wherein the request to resume the sub thread.

Images