JP2630239B2 - Distributed processing system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distributed processing system for processing jobs in a distributed system using a distributed file system.

[0002]

2. Description of the Related Art A distributed file system is a system for accessing a file in a remote node by means simple for a user. In general, a technology that realizes position transparency that does not require the user to be aware of the position of the file from the user's convenience. USL Syste
Examples of technologies implemented on the UNIX operating system, which is a trademark of m Laboratories, include Sun Microsystems' Network File System (N.
FS)) and Andrew File System (AFS) of Carnegie Mellon University.

[0003] The contents of NFS are based on the Design and Implementation of the Sun Net Network System (“Design and Implementation of t
he Sun Net Network Fill System "), a publication entitled" Conference Proceedings, "Usenix 1985, p.
119-130. ) And "V Nose: An Architecture for Multiple File System Types
In Sun UNIX "(“ Vnodes: An Architectu
re for Multiple File System Types in Sun U
NIX ”), a publication titled“ Conference Procedure Dilegs ”
gs, Usenix 1986, pp. 238-247. ). AF
The content of S is "Andrew: A Distributed Personal Computing Environment"("Andrew: A Distributed Personal Comb"
puting Environment ”) was published in“ Communication of the AC M ”(Commmu
nications of the ACM29 (3): 184-201, March, 1986
. ) And "The IT Distributed File System: Principles and
Design ”(“ The ITC Distributed File Sys
tem: Principles and Design "), a publication entitled" In Proceedings of the Tenth ACM Symposium on Operating Systems Principles "(In Proceed).
ingsof the Tenth ACM Symposium on Operatin
g Systems Principles, 33-50. December, 1985. )
Can be referred to.

[0004] Both approaches to a location transparent distributed file system are essentially different.

In the NFS, a stateless machine in which each server does not have client information is realized. This technology simplifies server design, such as easy recovery in the event of a server crash, but is not suitable for large-scale configurations. On the other hand, AFS is a distributed file system for a large-scale configuration using a small number of highly reliable server machines and a large number of other unreliable client machines as a configuration model, and is realized by a stateful server machine. AFS has a local cache on each client's disk in order to reduce the load on the server and achieve high speed. An example of using this technology to enhance the portability of each client is the Carnegie Mellon University Coda File System (Coda Fil).
e System). Coda uses a local cache on disk, and if the network is disconnected,
Or even if you intentionally disconnect from the network,
Operation is possible as if you are connected to the network. The operation on the client side in a state disconnected from the network is realized by a function called a disconnected operation. Coda says "Code: A
Highly Available File System for a Distributed Workstation
Embalment ”(“ Coda: A highly Available
Fill System for a Distributed Workstation E
nvironment ”), a publication titled“ IE Triple Transactions on Computers ”.
(4): 447-459, April, 1990) and "Scalable,
Secure and Bailey Available Distributed File Access ”(“ Scalable,
Secure and Highly Available Distributed F
ile Access "), a publication titled" I Ripley Computer "(IEEE Computer 9-2).
1, May, 1990). Such a distributed file location transparent file system played a role in enhancing the mobility of the user because the file looks the same from any computer. In addition, using technology to improve the portability of computers, such as Coda, develop computers that can be used in a network environment where computers are separated from high-speed local area networks (LANs) and connected by relatively low-speed communication paths such as wireless communication. Has been made. However, even when executing jobs on a distributed file system, if the required file data is not available on the computer that uses it, it is necessary to load the data via a network. Therefore, a computer connected to a file server easily suffers from a communication bottleneck, and cannot obtain sufficient processing performance even when a computer having a high-speed CPU performance is used. In addition, regarding the distributed processing system, JP-A-60-1313
In Japanese Patent Publication No. 51, the following invention is described. In other words, the present invention is a load distribution method in a distributed system. When a computer system in an overloaded state requests processing of a job generated in the system to another computer, files necessary for the processing are added together with the job. To the computer system. However, the present invention is a method in which a job request is triggered when the system becomes overloaded, and does not deal with a communication bottleneck in executing a job. In fact, in this technique, when a job requiring a large file capacity is requested to be processed by another computer, a great deal of time may be consumed in communication for transfer.

The invention disclosed in Japanese Patent Application Laid-Open No. 1-139590 is a processing request method for a distributed computer system, and deals with a technique for transmitting a processing request to a processing receiving computer. This realizes a fast response process by selectively extracting only necessary information and returning it to the requester. This is a method in which all processing is performed by a node called a processing receiving computer.
It is an invention that focuses on how to select necessary information for communication, and how to request other computers for processing or how to select such a job request destination computer Not touching. In addition, a network queuing system (Network
A distributed processing system such as the Queuing System (NQS) is a mechanism that integrally manages the delegation of job processing from a client computer to a computer acting as a server on a network. Was processed as a batch, and the user had to explicitly request the distributed processing system to perform job processing.

[0007]

As described above, according to the prior art, in a distributed system connected by a network, particularly when an attempt is made to use a distributed file system from a computer in a weak connection state, the computer is required to use the distributed file system. The transfer time for transferring files is greatly delayed due to the narrow communication width, and recent advances in hardware technology have enabled the desired performance to be obtained despite the fact that individual computers have sufficient power. Did not. Also, when a job is in a sufficiently strong connection state with the computer that has the files necessary for job execution, that is, the communication transfer width between these computers is sufficiently wide, the job is explicitly requested to request the computer to execute the job. It is only possible to specify a computer or explicitly request a distributed job processing management to select a computer and execute a job in batches. Here, the proposed invention aims to solve these problems. That is, in the present invention, the user does not need to be aware of the computer on which the job is being executed, and a computer that executes the job transparently is selected,
It is intended to be processed by a job. Another object of the present invention is to allow a user to enjoy the usability of executing a job on a computer at hand even when performing the job processing, and to omit the procedure of batch processing request. . Another object of the present invention is to allow these job processes to proceed interactively, and to stop or terminate the processes as necessary. In addition, according to the present invention, if the own computer has the processing performance necessary for job execution, and if a communication bottleneck does not occur, the job is not transferred, and the process is performed by itself. Another object is to distribute the load of the entire system and to effectively use a high-speed computer.

[0008]

SUMMARY OF THE INVENTION A first distributed processing system according to the present invention is a system including a first computer and a remote second computer connected to the first computer. the communication process on the first computer, the job start condition acquiring means for acquiring a job start conditions, the job management process generating means for generating a job management process on the second computer, generating the job management process Job start condition transmitting means for transmitting the job start condition obtained by the job start condition obtaining means to the job management process generated by the means, and reception of output data from the job management process, reception of an end code, or own communication A communication process waiting means for waiting for input from the standard input file of the process, and a standard input file When receiving a et of input includes an input data transmission means for transmitting the input data to the job management process, a job management process on the second computer
But a job start condition receiving means for receiving the job start condition sent from the communication process, the job process generating means for generating start a process of job execution based on the job start condition received by the job start condition receiving means Receiving input data from the input data transmitting means,
A job management process waiting unit that waits for reception of an end code of the job process or output data to the standard / error output file of the job process, and receives the input data from the input data transmission unit by the job management process waiting unit. In this case, when the input data is input to a standard input file of a job process, and when an output to a standard / error output file is received by the job management process queuing unit, the received data is transferred to the communication process queuing unit. Output data transmitting means for transmitting the end code of the job process to the communication process waiting means when the job management process waiting means receives the end code of the job process .
You .

A second distributed processing system of the present invention, the communication process, the job control signal acquisition unit configured to receive a job control signal generated by the first computing machine to job, this job control signal acquisition means a job control signal transfer means for <br/> transferring acquired job control signals to the job management process in a signal reception state receiving means for receiving a signal reception state of the job process in the signal reception state receiving means received signal reception state and a signal reception state display means for displaying the error output file of its own communication process, the job management process receives a job control signals transferred by the job control signal transfer means of the communication process Job control signal receiving means and job control signal receiving means Signal sending means for sending the job control signal received in the step to the job process, signal receiving state detecting means for detecting the signal receiving state for detecting the signal receiving state received by the job process, and signal receiving state detecting means include a signal reception state transmitting means for transmitting the signal reception state of the job process of detecting the signal reception state receiving means by
It is characterized by .

[0010]

Next, an embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 1, one embodiment of the present invention is:
The present invention is applied to a distributed processing system having a computer 1, a computer 2, and a communication path 3 connecting the computer 1 and the computer 2.

Referring to FIG. 2, in the first embodiment of the present invention, the computer 1 has a job leading process 101 and a communication process 102, and the computer 2 has a job management process 1
03 and a job process 104. The job leading process 101 includes a job start request receiving unit 111 that receives a job start request, a communication process generating unit 112 that generates and starts a communication process 102, and a job end code receiving unit 113 that receives a job end code passed from the communication process. And

The communication process 102 includes a job start condition obtaining means 121 for obtaining a job start condition, and a computer 2
A job management process generating means 122 for generating the job management process 103, a job start condition transmission means 123 for transmitting the job start condition to the job management process 103, and receiving or ending the output data from the job management process 103 A communication process waiting means 124 for simultaneously receiving a code or input from the standard input file of the communication process 102; and when the communication process waiting means 124 receives output data from the job management process 103, the output data is An output unit 125 for outputting to a standard / error output file of the communication process; and an input data transmitting unit 126 for transmitting the input data to the job management process 103 when the communication process waiting unit 124 receives an input from the standard input file. When receiving the termination code the communication process waiting unit 124, and an end code notification unit 127 for notifying the termination code in the job leading process 101.

The job management process 103 includes a job start condition receiving unit 131 that receives a job start condition transmitted from the communication process 102, and a job process that generates and starts a process for executing a job based on the received job start condition. Generating means 132 and the communication process 10
Receiving input data from 2 or job process 10
The job management process waiting means 133 waits for the reception of the end code of No. 4 or the output data of the job process 104 to the standard / error output file at the same time, and the job management process waiting means 133 transmits the data by the input data transmitting means 126. When the input is received, the input data is input to the standard input file of the job process 104. When the output to the standard / error output file is received by the job management process queuing unit 133, the received data is output. Output data transmitting means 135 for transmitting to the communication process 102
And an end code transfer unit 136 for transferring the end code to the communication process 102 when the job management process waiting unit 133 receives the end code of the job process.

Next, the operation of the process transfer system according to the first embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIGS. 1 and 2, the job leading process 101 reads the job request from the user from the command line terminal or from a general file by the job start request receiving means 111. Further, the communication process generation means 112 stores the communication process 10 on the computer 1.
2 is generated and activated. If the job is to be executed in the foreground, the job completion code receiving unit 113 waits for the completion of the job until receiving the notification of the job completion code from the communication process 102. When the job is executed in the background, the process does not wait for a job end code from the communication process 102. The standard input / output and error output file of the communication process 102 are rewritten by the job leading process 101 according to the job start request.

The communication process 102 generated by the communication process generating means 112 is converted into a job starting condition obtaining means 121.
Obtains the conditions for starting the job. This includes the command line needed to generate the command,
Information on environment variables, standard input / output files, and error output files. The attributes of the computer to be executed and the file information to be used from now on can be obtained by this means by giving them as environment variables. The standard input / output file and the error output file of the communication process 102 are set to be the same as those of the job when the job is executed on the computer 1.

Further, the communication process 102 generates and activates the job management process 103 on the computer 2 by using the job management process generation means 122. The job management process 103 manages a later-described job process 104 while communicating with the communication process 102.

The communication process 102 then outputs the output request transmitted by the output data transmission unit 135 of the job management process 103 described later by the communication process waiting unit 124, or the end code from the job process 104 described later, or Communication process 1
02, and waits for input from the standard input file at the same time.

In this case, when an output request is received from the job management process 103 by the output data transmitting means 135, the received data is transmitted to the standard /
Output to error output file. The output data transmitting unit 135 indicates whether to output to the standard output file or the error output file.

Since the standard output file or the error output file is the same as the file expected for the job requested by the user, the output from the job process 104 executed on the computer 2 is as if the user Recognizable as if done in In the case of output to a regular file in addition to output to the terminal, or when using pipes and redirection, this can be achieved by setting the file on a distributed file system that realizes location transparency.

When the input from the standard input file is received by the communication process waiting means 124, the input data transmitting means 126 is activated and this input data is sent to the job management process 103. By this means, the user can perform input to the job process 104 executed on the computer 2 as in the case of output without needing to know the computer position to execute.

When an end code of a job is received from the job management process 103 by an end code transfer unit 136 described below, the end code is processed by an end code notification unit 127. At this time, the end code notification unit 12
7, when the job leading process 101 is waiting for the end of the job, that is, when the job is started in the foreground, the end code notifying unit 127 notifies the job leading process 101 of the end code. On the other hand, when the job leading process 101 does not wait for the end of the job, that is, when the job is started in the background, the end code is output to the error output file.

The job management process 103 receives, by the job activation condition receiving means 131, the activation condition requested by the job, that is, the command line, environment variables, and the like. The job process generating means 132 starts the job process 104 according to the job starting conditions obtained by the job starting condition receiving means 131. If the standard input / output file or error output file is a device terminal or if the standard input file is a regular file, the job is forcibly rewritten to a virtual terminal.

Next, job management process waiting means 1
33, and simultaneously waits for an end request from the generated job process 104, output to the virtual terminal, or an input request to the job process 104 transmitted by the input data transmitting unit 126 performed by the communication process 102.

This job management process waiting means 1
33, when the input data output from the communication process 102 is received,
Data is input to the job process 104. Job process 104 receives this input data from the standard input file.

Job management process waiting means 133
When the job process 104 outputs to the standard / error output file, the output data is received via the virtual terminal and transmitted to the communication process 102 using the output data transmission means 135.

Job management process waiting means 133
When the end code of the job process 104 is received, the end code is transferred to the communication process 102 using the end code transfer unit 136.

Next, the operation of the remote process control system according to the second embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIGS. 1, 2 and 3, in the second embodiment of the present invention, the communication process 102 is a job control signal acquiring means for receiving a job control signal generated on the computer 1 for a job. 211, a job control signal transfer unit 212 that transfers the job control signal to the job management process 103, a signal reception state reception unit 213 that receives a signal reception state of the job process 104, and a signal reception received by the reception unit 213. The job management process 103 receives the job control signal transferred by the job control signal transfer unit 212 of the communication process 102. The signal reception status display unit 214 displays the status in an error output file of the communication process 102. Job control signal receiving means 22 A signal transmitting unit 222 for transmitting the job control signal received by the job control signal receiving unit 221 to the job process 104; a signal receiving state detecting unit 223 for detecting a reception state of the signal received by the job process 104; Job process 104 detected by signal reception state detection means 223
And a signal reception status transmitting unit 224 for transmitting the signal reception status of the communication process 102 to the communication process 102.

Next, a remote process control system according to a second embodiment of the present invention will be described in detail with reference to the drawings.

The communication process 102 on the computer 1 is a job control signal acquisition means 2 for receiving a job control signal for controlling the job process 104 on the computer 2.
Has 11 This includes a signal for forcibly terminating the process, a signal for stopping the process, a signal for continuing the process, and the like. On the computer 1, the communication process 102 renames its own process to the job process name, and makes it appear as if the job process 104 exists on the computer 1, and the user on the computer 1 sends the signal to the renamed communication process 102. Can be raised.

In the job control signal transfer means 212,
The job control signal acquired by the job control signal acquisition means 211 is transferred to the job management process 103 on the computer 2. At the time of the transfer, additional information such as a time stamp is simultaneously transmitted in order to log on the job management process 103 side in addition to the signal type.

The job management process 103 on the computer 2 has a job control signal receiving means 221 for receiving the job control signal transmitted from the communication process 102.
By this means, the additional information such as the time stamp is obtained at the same time when the signal type is received. Further, the signal sending means 222 sends the signal code received by the job control signal receiving means 221 to the job process 104 to the job process 104.

The job process 104 generated by the job management process 103 ends abnormally when a signal is received,
Alternatively, if there is a state change due to processing interruption or the like, a signal is sent to the parent process, the job management process 103. The job management process 103 receives this signal by the signal reception state detection means 223, and receives the received information. The signal is transmitted to the communication process 102 by the signal reception state transmission unit 224.

In the communication process 102, the signal reception status transmitted by the signal reception status transmission unit 223 of the job management process 103 is received by the signal reception status reception unit 213. Further, it is displayed on the error output file of the communication process 102 by the signal reception state display means 214.

Next, a configuration common to the other embodiments except the first and second embodiments of the present invention will be described with reference to FIG.

Referring to FIG. 4, N computers are represented by computers 1-M (1.ltoreq.M.ltoreq.N), each of which is connected by a high-speed communication path 5. Each computer 1-M has a distributed file system 7. The distributed file system 7 is characterized in that a secondary storage medium has an area for caching files. A computer pool 4 is composed of the computer group and the high-speed communication path 5. The computer 20 also has a distributed file system 7, and is connected to the computer pool 4 via the low-speed communication path 6.

Next, a third embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 4, FIG. 5, and FIG. 6, in the third embodiment of the present invention, the computer pool 4 includes N computers from a plurality of computers 1-1 to 1-N, and A computer 20 is connected to the computer pool 4 by a low-speed communication path 6, and the computers in the computer pool 4 and the computers 20 are distributed processing having a distributed file system 7 that is transparent in file position. In the system, the computer 20 has a communication process 102, and the communication process 102 is a computer 1-M (0
≦ M ≦ N), a decision request transmitting unit 321 for transmitting a job server determination request accompanied by requirements for job processing;
Decision receiving means 3 for receiving the result determined by computer 1-M
22. The computer 1-M receives the computer status from each computer in the computer pool 4 at regular intervals, and transmits it by the status monitoring unit 311 that records it in the computer status table 301 and the determination request transmission unit 321. A decision request receiving unit 312 for receiving the job server determination request, and a job server determining unit 313 for determining a computer to execute a job from within the computer pool 4 based on requirements necessary for job processing.
And a notifying unit 314 for notifying the communication process 102 of the determined result.

Next, the operation of the third embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 4, FIG. 5 and FIG. 6, the third embodiment of the present invention relates to the state monitoring means 3 of the computer 1-M.
11 receives a status notification signal sent from each computer,
The result is recorded in the computer status table 301. The contents notified by each computer are a load state and a file cache state.

FIG. 6 shows an example of the computer status table 301.

In this example, the load state indicates the operating rate of the processor in five stages. Occupancy rate is 0% to less than 20%, 1%, 20% to less than 40% is 2, 40% to less than 60% is 3, 60% to less than 80% is 4, 80% to 100%
Is given in a value such as 5 and recorded in the computer status table 301. Alternatively, the processor operation rate may be recorded as a percentage or may be recorded as a pair with the disk operation rate. There is also a method of taking an arithmetic average of the load state up to that time.

There is also a method of recording the processor operation rate in detail, such as the kernel execution rate and the user execution rate. However, the size of the computer status table 301, the network load when notifying such information, and the job server described later The decision is made in consideration of the complexity of the decision algorithm in the decision means 313 and the like. Also, the entry of the cache file that each computer has cached on the secondary storage medium is recorded. Since notifying or recording all the cached file entries causes a large overhead, in this example, the top five entries with the largest cached capacity are reported for each computer and recorded. Therefore, this management method is effective not only when caching the entire file but also when partially caching the file. Here, the computer attribute records the type of the processor, the memory capacity, and the disk capacity for the cache file at the time of the first communication, and is updated when there is a change thereafter.

The communication process 102 in the computer 20 entrusts the determination of the computer to execute the job to the computer 1-M by using the determination request transmitting means 321. Communication process 102
If it knows that the computer 1-M is managing the computer state in the computer pool 4, it makes a request directly to the computer 1-M via the communication station in the computer pool 4. In the determination notification means 314 described later, the computer 1-M communicates via the communication station. There is also a method in which the computer 20 simply transmits a job server determination request to the communication station, and finds or knows the computer 1-M in which the communication station manages the computer state. When finding the computer 1-M, there is a method of sending a broadcast in the computer pool 1 to make the computer respond or to request another computer to search. They are all included in this means.

When activating the decision request transmitting means 321, the optimum job server differs depending on the type of job. Therefore, the decision request transmitting means 321 is accompanied by computer attributes and file information which are necessary for the job. For example,
In a job for generating an object to be executed by the own computer, it is conceivable that the type of processor of the computer 20 or a file path required for generation is transmitted as hint information. In addition, a job that performs a scientific and technical operation wants to find a computer having a high-speed processor, so it is conceivable to specify the type of the high-speed processor and the path name of the data file as a hint. In the description here, it is assumed that a maximum of five files can be specified, but this is for convenience of description and does not limit the scope of the present invention.

The computer 1-M receives the job server determination request transmitted from the computer 20 by the determination request receiving means 312. The determination request receiving unit 312 passes the received computer attributes and file information to the job server determining unit 313.

The job server determining means 313 uses the information passed from the determination request receiving means 312 as the
1 is used to determine the job server. For example, the decision algorithm includes the following.

First, the score is calculated for each of the three items of the computer attribute, the load, and the file cache status. Score 0
From 1 to 1. If a processor is specified as a computer attribute, and there is no corresponding processor, 0 is given. If the processors match or the processor is not specified, the larger the memory capacity and the file cache disk capacity, the larger the value.
Give a value close to. The load uses the reciprocal of the value indicated in five stages. If there are all the files specified in the entry of the file cache state of the last item, it is 1, 0.9 if it is multiplied by 1, and 0 if it is multiplied.
8, 3 times 0.7, 4 times 0.
6. If there is no one, give 0.5. The geometric mean of these scores is calculated, and the computer that has the highest score is determined as the job server. Alternatively, there is a method of calculating a geometric mean of values obtained by multiplying these items by a ratio representing a weight. In this case, there are a method in which the ratio is uniform and a method in which the computer 20 specifies the ratio.

The identifier of the computer thus determined is notified to the computer 20 by the determination notifying means 314. An example of the identifier is a network address. The computer 20 receives the identifier of the computer to be used as the job server by the decision receiving means 322.

Next, a fourth embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 7, an embodiment of the present invention is shown in FIG.
In comparison with the third embodiment shown in FIG.
A computer status inquiry means 411 for inquiring each computer in the computer pool 4 about the computer status is provided.

Next, the characteristic operation of the fourth embodiment of the present invention will be described with reference to the drawings.

Referring to FIGS. 6 and 7, the computer status inquiry means 411 of the computer 1-M according to the fourth embodiment of the present invention actively queries the status of the computers in the computer pool 4. Do. For this purpose, a broadcast is performed to notify the computer attributes, file cache status, and load status of each computer. This result is put into, for example, a computer status table 301 and passed to the next job server determining unit 314 as a determination factor for determining a job server.

Next, a fifth embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIGS. 2 and 8, the fifth embodiment of the present invention is different from the second embodiment in that the job leading process 101 stores the job process to be transferred as information on a job leading process unit basis. It has a transfer job registration unit 511 and an individual transfer job check unit 512 for checking whether the received job request is a job process to be transferred.

Referring to FIG. 8, in the first transfer job process selection method according to the fifth embodiment of the present invention, the job leading process 101 includes an individual transfer job registration unit 511.
, Any one of the computers 1 -M in the computer pool 4
Is stored as information for each job leading process. An example of this is a method of storing as environment variables. As an environment variable name, for example, MIGR
There is a method of registering a job by connecting a plurality of jobs to a variable name of “ATIONJOB” by using “:”. Alternatively, there is a method of specifying a path to a file in which a job entry to be transferred is registered in an environment variable. In any case, these are handled as information for each job leading process or for each session, and do not affect other users in the computer 20.

Next, when a job request is input by the individual transfer job check unit 512, the job leading process 101 registers the transfer job candidate entry for each job leading process registered by the individual transfer job registering unit 511. Check and compare with the just entered job request entry. For example, if the job is registered as an environment variable, the job entry registered in the environment variable is compared with the requested job. If they match, the job is transferred to the computer 1-M in the computer pool 4 by the process transfer method according to the first embodiment of the present invention, where the job is processed.
In selecting the computer 1-M, the job server selection method of the third embodiment or the fourth embodiment is used.
In addition, these processes are signal-controlled at any time by the remote process control method of the second embodiment. The same applies to the above example where the path to the file storing the list of job entries to be transferred is specified as an environment variable.

Next, a sixth embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 9, the sixth embodiment of the present invention is different from the second embodiment in that the computer 20 has a common transfer job registering means for registering a job process to be transferred as computer unit information. 611, the job leading process 101 checks whether the received job request is a job process to be transferred.
1 is provided.

Next, the operation of the sixth embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 9, the computer 20 according to the sixth embodiment of the present invention uses the common transfer job registration unit 611 to store a job entry to be executed by any one of the computers 1-M in the computer pool 4. Remember in
The storage location includes a location that can be commonly referred to by a plurality of job launchers, such as a file, a shared memory, and a kernel. The registration information can be accessed through a system call in the kernel, and can be read and written after attachment in a shared memory. In the case of a file, the information can be updated and referenced using a normal file access interface. The job entry is stored in this storage location. This information can be referred to by all processes in the computer 20. When a job request is input, the job leading process 101 activates the common transfer job check unit 621, and the input job request is first input to the common transfer job registration unit 611.
Compare with the job entry registered by. If they match, the job is transferred to the computer 1-M in the computer pool 4 by the process transfer method of the first embodiment, and the job is processed. In addition, these processes are signal-controlled at any time by the remote process control method of the second embodiment.

Next, a seventh embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 10, according to a seventh embodiment of the present invention, based on the second embodiment, the computer 20 includes a job process 702 and a plurality of job child processes 703-L activated by the job process 702. (L is a natural number), and the job leading process 101 sends a job request defined as a script described by a combination of a plurality of commands, an end notification from the job process 702, or a job transfer instruction from the user. Job leading process waiting means 711 for waiting at the same time, and job process starting means 7 for generating a job process for executing a job
12 and an interruption instructing unit 713 for sending an interruption instruction to the job process 702.

The job process 702 reads a script and sequentially starts job child processes. The job process 702 is a job process that simultaneously waits for an instruction to suspend the job process 702 or a completion notification from the job child process 703-L. When receiving the completion notification from the waiting means 722 and the job child process 703-L, the processing continuation determining means 723 for determining whether to continue the next script line, and the processing continuation determining means 723 When a mark for interrupting the job process 702 is detected, a job process interrupting unit 724 for interrupting the job process, and an interrupt completion notifying unit 725 for notifying that the job process 702 has been interrupted by the job process interrupting unit 724. And the processing continuation determination 7 If it is determined to exit the script at 3, job leading process 101 a job end code
And a job end code transmitting unit 726 for transmitting the job completion code to the job process waiting unit 722.
7, and the job child process 703-L includes a completion notifying unit 731 for reporting that its own process has been completed.

Hereinafter, what is necessary as a premise of the operation of the seventh to tenth embodiments will be described.

In the following description, the job leading process 1
An example of a job request for 01 is a command line. The job requested by the user may be given in the form of a single command or a script in which a plurality of commands are combined. Frequently used scripts are created in a file, and are executed by giving the execution right to the file and directly starting the script, or by passing a script interpreting command (a shell command in UNIX).

As another example, there is a method of assigning an identifier to a job in advance and placing the identifier in a job queue. However, since a job script corresponds to a job identifier in the near future, the description will be given here using an example using a command for interpreting the script.

Next, the operation of the seventh embodiment of the present invention will be described in detail with reference to the drawings.

The job leading process 101 is transmitted by the job leading process waiting means 711 by a job request defined as a script described by a combination of a plurality of commands, a job interruption instruction, or a job end code transmitting means 726. Waiting for the end notification at the same time. When a job request is accepted, the job process starting unit 712 is started to
02 is generated and started in the computer 20 to cause it to perform job request processing. If the job request is an execution request in the foreground, the next job request is suppressed by the job leading process waiting means 711. The change from the foreground execution request to the background execution request during the execution of the job, and vice versa, can also be controlled by changing the job request acceptance suppression by the job leading process queuing means 711. The end of the job process 702 is received by an end code notification transmitted by a job end code transmission unit 726 described later.

When the job leading process 101 receives an interruption instruction from the job initiator or the system administrator by the job leading process waiting unit 711, the interruption instruction unit 713 gives an interruption instruction to the job process 702.

When the job leading process 101 receives the interruption completion notice by the job leading process waiting means 711, the execution of the script part that has not been executed is executed in the computer pool 4 by using the process transfer method of the first embodiment. And execution is controlled by the remote process control method of the second embodiment. In this case, the line number at which the execution of the script has been completed is given as additional information to the job start request receiving means 111 of the process transfer method. It can be realized by doing. The selection of a computer for executing a job can be determined by the job server selection method according to the third embodiment or the fourth embodiment.

On the other hand, the job process 702 generates and starts the job child process 703-L when it is necessary to sequentially read the scripts by the job child process starting means 721, analyze the lexical data, and generate a process. In the script, execute the built-in command line by itself.

When the job child process 703 -L is activated, the completion notification notified by the completion notifying means 731 from the job child process 703 -L and the job leading process 1
In order to simultaneously wait for the interruption instruction from the interruption instruction unit 713 from the job process queuing unit 722,
Start When the completion notice is received, the processing continuation determining means 723 is activated, and it is confirmed whether or not the interruption instruction is marked. If the interruption instruction has not been issued and the next script line exists, the job child process activation unit 7
21 is executed to execute the script line. Also,
If there is no next script line, or if the next script line instructs the end of the script such as exit even if the interruption instruction is given, the job end code transmission unit 726 is activated and the job request is completed. This is transmitted and transmitted together with the job end code.

If the processing continuation judging means 723 has instructed the interruption and there is a script line to be executed next, the job process interrupting means 724 interrupts the processing of the job process 702. When the processing related to the interruption is completed, the interruption completion notifying unit 725 is activated to notify the job leading process 101 that the interruption of the job process 702 has been completed, and at the same time, the number of the interrupted script lines and the internally held variable value are transferred. .

This job process waiting means 722
When the interruption notification is received from the job leading process 101, the interruption instruction marking means 727 marks the interruption instruction.

When the job process 702 itself executes the internal built-in command in the job child process starting means 721, the processing continuation determining means 723 is directly called without passing through the job process waiting means 722.

The job child process 703-L uses the completion notifying means 731 when the processing is completed,
The job process 702 is notified with an end code.

Next, an eighth embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 11, according to an eighth embodiment of the present invention, based on the seventh embodiment, the computer 20 manages cache misses for the file cache on the secondary storage for each process group. Job leading process 10 having cache miss count means 801
When a job request is received by the job leading process waiting unit 711, a transfer job checking unit 811 that checks a job registered as a job to be transferred, and an interruption completion notification is sent to the job leading process waiting unit 711. And a transfer job registration unit 812 for registering the script name when receiving the script name.
The job child process 703 includes a mishit count checking unit 821 that checks whether the accumulated mishit count of a process including the job process 702 activated by the job process 702 by itself has not exceeded a threshold value. -L is a threshold value check unit 831 for checking whether the accumulated cache miss count of the process including the job process 702 started by the job process 702 has reached the threshold value, and an interruption instruction to the job process 702. Suspension instruction means 7 to send
Next, the operation of the transfer job process selection method according to the eighth embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 11, when the job leading process 101 receives a job request by the job leading process waiting means 711, the job request should be transferred by the transfer job checking means 811 by the transfer job registering means 812 described later. Check whether the job has been registered. If registered, the first
The job is transferred by activating the claimed process transfer method, and the signal is controlled by the remote process control method of the second embodiment. The transfer destination computer is determined using the job server determination method of the third embodiment or the fourth embodiment. If not registered, the job process activation unit 712 generates a job process 702 for processing the job request.

When the job leading process 101 receives the interruption completion notification by the job leading process waiting unit 711, the transfer job registering unit 812 registers that it is more effective to transfer and process the script. This registration will be performed automatically by the system. Thereafter, using the process transfer method of the first embodiment, the script portion that has not been executed is executed by the computer 1-M in the computer pool 1, and the remote process control of the second embodiment is executed. The execution is controlled by the method. In this case, the fact that the execution of the script has been completed is given to the job transfer request receiving means 111 of the process transfer method with the line number as additional information, and the computer 1-M
When performing the job continuation execution above, this can be realized by executing from the line following the script end line.

On the other hand, the job process 702 generates and starts a job child process 703-L when it is necessary to sequentially read the script by the job child process starting means 721 and analyze the lexical data to generate a process. In the script, execute the built-in command line by itself. In this case, the mishit count checking unit 821 checks whether the cache miss count during job execution by a process including the job process 702 generated and started by the job process 702 during processing is not greater than a threshold value. If it exceeds, the interruption instruction marking means 727 is used to mark interruption when the next script line is executed.

When a cache miss occurs during execution, the job child process 703-L checks the accumulated cache miss hit count of the process generated and started by the job process 702 including the job process 702, and does not exceed the threshold value. Is checked by the miss hit count check means 831. As a result, if the threshold value is exceeded, the interruption instruction means 713 instructs the job process 702 to mark the interruption, and the processing itself continues. Then, when the processing of the job child process 703-L is completed, the job notification unit 731 notifies the job process 702 of the completion by using the completion notification unit 731 together with an end code.

Finally, the cache miss count means 801 will be described. The secondary storage medium on the computer 20 has a file cache managed by the distributed file system 7. Since this file cache capacity is limited, it is not possible to cache and hold all necessary files. Therefore, when a process on the computer 20 accesses a file that is not cached or a portion thereof during operation, As a cache miss, an attempt is made to transfer a necessary file or a part thereof from a file server existing in the computer pool 4.
The operation related to the cache miss is solved by means for managing the cache, but here, the number of cache misses is also held at the same time. In addition, the number of cache misses can be held individually for each process, or the process can be managed as a group, and the number of cache misses can be counted for each group. Here, paying attention to the number of cache misses occurring in the process of processing a job, the number of cache misses is managed for each group.

Next, a ninth embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 12, according to a ninth embodiment of the present invention, a data transfer amount count for managing a data transfer amount transferred from a file server when a cache mishit occurs in a file cache on a secondary storage for each process group. Means 901, the job process 70
Reference numeral 2 denotes a transfer amount count check unit 921 that checks whether the integrated data transfer amount of a process including the job process 702 activated by the job process 702 itself has reached a threshold value.
The job child process 703-L includes a transfer amount count check unit 931 for checking whether the integrated data transfer amount of a process including the job process 702 started by the job process 702 has reached a threshold value.

Next, a ninth embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 1 shows the operation of the ninth embodiment of the present invention.
The operation of the eighth embodiment will be described in detail with reference to FIG.

Referring to FIG. 12, in the ninth embodiment of the present invention, the data transfer amount counting means 901 will be described first. As described above, the cache management means, which is a component for resolving a cache miss, attempts to transfer a necessary file or a part thereof from a file server existing in the computer pool 4 at the time of a cache miss. Here, the data transfer amount for each process is held at the same time as the number of cache misses, or the process is managed as a group, and the number of cache misses is counted for each group. Here, paying attention to the data transfer amount generated in the process of processing the job, the data transfer amount is managed on a group basis.

The other features are a transfer amount count checking means 921 and a transfer amount count checking means 931. These are respectively changed threshold value check targets to data transfer amounts.

Next, a tenth embodiment will be described in detail with reference to the drawings.

Referring to FIG. 13, a tenth embodiment of the present invention is based on the
Is a resource monitoring process generation and activation unit 1011 that generates and activates the resource monitoring process 1001, and the resource monitoring process 10 that is generated by the resource monitoring process generation and activation unit 1011.
01, the resource monitoring process 1001 monitors resource consumption of a process group having the job process 702 as a parent, and the resource monitoring process 1013 terminates resource consumption. If the job process 702 is determined to be remarkable,
And an interruption instruction means 713 for transmitting an interruption instruction to the CPU.

Next, the operation of the tenth embodiment of the present invention will be described in detail with reference to FIG.

The features of the operation of the tenth embodiment of the present invention will be described below with reference to FIG. 13 as parts different from the operation of the seventh embodiment.

Referring to FIG. 13, in the characteristic portion of the tenth embodiment of the present invention, the job process 702 generates and starts the resource monitoring process 1001 by the resource monitoring process generation and start means 1011.

If the processing continuation unit 723 determines that the job needs to be interrupted, the resource monitoring process 1001 is terminated by the resource management process termination unit 1012. Then, immediately, the job process suspending means 7
24 to start the job process interruption processing.

When the processing continuation unit 723 determines that the job has been completed, the resource monitoring process ending unit 1012 ends the resource monitoring process 1001. Then, immediately, the job end code transmitting unit 726
To notify the job leading process 101 of the job end code.

The resource monitoring process 1001 activates the resource monitoring means 1013 at regular intervals. The resource monitoring unit 1013 plays a role of monitoring the consumed resources of the process group having the job process 702 as a parent. Information on each process, such as CPU consumption time, memory consumption, disk usage rate, etc., is accumulated for each process group as consumption resources, and data is statistically held. Resource monitoring means 1013
Then, the information collected in this way is analyzed, and if the long-term resource utilization is extremely high, the execution on the computer 20 is inefficient, and the computer pool 4 having higher computational power and abundant computer specifications is provided. Select one of the calculators 1-M,
Job process 7 to execute the rest of the script
In step 02, a job interruption request is transmitted using the interruption instruction unit 713. The interruption instructing unit 713 is notified with a reason for determining that it should be interrupted as additional information.
25 can pass the information to the job leading process 101.
When the job leading process 101 receives the reason for interruption, it becomes possible to select a server according to the reason for interruption in the job server selection method used in the process transfer method. For example, if the memory overload is a reason for interruption, a condition for selecting a computer with a large memory implementation may be used.

[0101]

As an effect of the present invention, a job execution request from a computer in a weak connection state to a computer having a file necessary for executing a job is automatically sent to a computer in a stronger connection state. To be requested and executed,
The communication bottleneck of a computer in a weakly connected state is alleviated, and job execution is performed at high speed. Further, it is transparent to the user on which computer the job is being executed, and it is possible to provide a user view as if the distributed job service is executed on a local machine.
That is, the user does not need to be aware of the computer on which the job is being executed, and the computer that executes the job transparently is selected and the job is processed. Also, in the job processing, the user can enjoy the usability as if the job is being executed on the computer at hand, and does not need to perform the procedure of batch processing request. In addition, these job processes can proceed interactively, and the processes can be stopped or terminated as necessary. Furthermore, if this technique is used, a conventional command group for job processing can be used without modification, and effective use of assets can be achieved. In addition, if the own computer has the processing performance necessary for job execution and there is no communication bottleneck, the job is not transferred and processing is performed by itself. It is possible to effectively utilize a high-speed computer which leads to load distribution.

[Brief description of the drawings]

FIG. 1 is a diagram showing a system applied to the present invention;

FIG. 2 is a diagram showing a first embodiment of the present invention.

FIG. 3 shows an example of a computer status table.

FIG. 4 is a diagram showing a second embodiment of the present invention.

FIG. 5 is a diagram showing a third embodiment of the present invention.

FIG. 6 is a diagram showing a configuration applied to the fourth to tenth embodiments of the present invention.

FIG. 7 is a diagram showing a fourth embodiment of the present invention.

FIG. 8 is a diagram showing a fifth embodiment of the present invention.

FIG. 9 is a diagram showing a sixth embodiment of the present invention.

FIG. 10 is a diagram showing a seventh embodiment of the present invention.

FIG. 11 is a diagram showing an eighth embodiment of the present invention.

FIG. 12 is a diagram showing a ninth embodiment of the present invention.

FIG. 13 is a diagram showing a tenth embodiment of the present invention.

[Explanation of symbols]

 Reference Signs List 1 computer 2 computer 3 communication path 4 computer pool 5 high-speed communication path 6 low-speed communication path 7 distributed file system 1-1 computer 1-2 computer 1-M computer (1 ≦ M ≦ N) 1-N computer 20 computer 101 job leading Process 102 Communication process 103 Job management process 104 Job process 111 Job start request receiving means 112 Communication process generating means 113 Job end code receiving means 121 Job starting condition obtaining means 122 Job management process generating means 123 Job starting condition transmitting means 124 Communication process waiting Means 125 Output means 126 Input data transmission means 127 End code communication means 131 Job start condition receiving means 132 Job process generation means 133 Job management process waiting means 134 Data input means 13 Output data transmission means 136 End code transfer means 211 Job control signal acquisition means 212 Job control signal transfer means 213 Signal reception status reception means 214 Signal reception status display means 221 Job control signal reception means 222 Signal transmission means 223 Signal reception status detection means 224 Signal reception status transmission means 301 Computer status table 311 Status monitoring means 312 Decision request reception means 313 Job server determination means 314 Decision notification means 321 Decision request transmission means 322 Decision reception means 411 Computer status inquiry means 511 Individual transfer job registration means 512 Individual transfer Job check means 611 Common transfer job registration means 612 Common transfer job check means 702 Job process 703-L Job child process 711 Job destination Process waiting means 712 Job process starting means 713 Interruption instructing means 721 Job child process starting means 722 Job process waiting means 723 Processing continuation judging means 724 Job process interrupting means 725 Interruption completion notifying means 726 Job end code transmitting means 727 Interruption instructing mark means 731 Completion Notification means 801 Cache mishit count means 811 Transfer job check means 812 Transfer job registration means 821 Mishit count check means 831 Mishit count check means 901 Data transfer quantity count means 921 Transfer quantity count check means 931 Transfer quantity count check means 1001 Resources Monitoring process 1011 resource monitoring process generation and activation means 1012 resource monitoring process ending means 1013 resource Monitoring means

Claims (2)

(57) [Claims] 1. A first computer and the job be the first system including a computer connected to a second computer for remote communication process on the first computer, to obtain a job start conditions Start condition obtaining means; a job management process generating means for generating a job management process on the second computer; and a job management process generated by the job management process generating means obtained by the job start condition obtaining means. A job start condition transmitting unit for transmitting a job start condition; a communication process waiting unit for waiting for reception of output data from the job management process, reception of an end code, or input from a standard input file of the own communication process; When the input from the standard input file is received by the waiting means, the input data is Wherein the input data transmission means for transmitting to the process, the job management process on the second computer, and the job start condition receiving means for receiving the job start condition sent from the communication process, the job start condition receiving means A job process generating means for generating and starting a job execution process based on the job starting condition received in step 1, receiving input data from the input data transmitting means, receiving an end code of the job process, or standard / error of the job process A job management process queuing unit that waits for reception of output data to an output file; and when the job management process queuing unit receives input data from the input data transmission unit, inputs the input data to a standard input file of the job process. Data input means for executing the job management process. When the output to the standard / error output file is received by the job queuing means, the output data transmitting means for transmitting the received data to the communication process queuing means; and the job management process queuing means receives the end code of the job process. In a case, the distributed processing system includes an end code transfer unit that transfers the end code to the communication process waiting unit. 2. The communication process according to claim 1, wherein the communication process receives a job control signal generated on the first computer for the job, the job control signal acquiring means receiving the job control signal acquired by the job control signal acquiring means. job management and job control signals transfer means for transferring the process, and the signal reception state receiving means for receiving a signal reception state of the job process, error output file of its own communication process the signal reception state received in the signal receiving state receiving means and a signal receiving state display means for displaying the job management process, a job control signal receiving means for receiving the transferred job control signals by the job control signal transfer means of the communication process, the job control signal receiving means The job control signal received by A signal sending means for sending to the job process; a signal receiving state detecting means for detecting a signal receiving state for detecting a signal receiving state received by the job process; and a signal receiving state of the job process detected by the signal receiving state detecting means. 2. The distributed processing system according to claim 1, further comprising: a signal receiving state transmitting unit that transmits the signal receiving state to the signal receiving state receiving unit.