JPH0997203A - Distributed processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a distributed processing system with which the descriptions of programs by systematically unified logical file titles are permitted and a uniform job execution history can be outputted. SOLUTION: This system is provided with a job managing server 3 for managing entire distributed processing and job executing servers 4a...4n for executing the jobs of programs and the programs to be distributedly processed are described by processing programs, the execution order of processing programs and logical title files. Then, the job managing server 3 is provided with a node table 5 recording the names or the like of job executing servers, a job execution history file 6 and a job execution managing means 7 for managing the execution of jobs, and the job executing servers 4a∼4n are provided with logical/physical resource converting tables 8a...8n for converting the logical title files into physical resource files in reading and job executing means 9a...9n for executing jobs.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distributed processing system in which a plurality of computers are connected to process a program in a distributed manner, and in particular, for a program to be processed, a physical storage device name of the computer actually connected. The present invention relates to a distributed processing system that allows description by systematically unified logical file names without being conscious of the above, and can output a job execution history of a processing result of a program.

[0002]

2. Description of the Related Art Conventionally, processing of large programs has been carried out uniformly by a large computer called a mainframe. But system flexibility, economy,
Due to demands for system reliability, there has recently been a growing demand for processing equivalent to that of large computers by connecting minicomputers, workstations, personal computers, etc. via communication lines and distributing the processing of programs. .

In response to this demand, various distributed operating systems (referred to as distributed OS) have recently been proposed. With these distributed OSs, a program can be written with a unified grammar and files in other computers can be accessed. This distributed O
S makes it possible technically to perform a distributed processing of a large program by a plurality of computers. UNIX is an example of such a distributed OS.

FIG. 7 shows a hardware system configuration in the case of distributed processing of a program by a conventional distributed OS.

As shown in FIG. 7, a conventional distributed processing system 11 comprises a plurality of computers 12a, ..., 12n connected by a communication line 13.

Each of the computers 12a, ..., 12n has a different physical resource configuration, for example, a storage device for storing a data file, and the like, but the same job executing means 14a ,. They are common in that they are parallel to each other.

The program 15 processed by the distributed processing system 11 having the above configuration must be described based on the physical resources and environment. That is, the program 15 specifies the job (a series of processes in the program) and the execution order of the computers 12a, ..., 12n, and the specific storage device names of the computers 12a ,.
The I / O file must be described by specifying the directory name.

In the example of FIG. 7, as indicated by the arrow, the program 15 is first processed by the computer 12a, then the processing result is transferred to the computer 12b and processed, and further processed by the computer 12n. The final processing result is returned to the computer 12a.

[0009]

However, in the above-mentioned conventional distributed processing system, the work for writing the program is complicated, and it is difficult to flexibly deal with the expansion and modification of the system configuration. .

That is, although the description of the program by the distributed OS can be described by a unified grammar,
The program writer must explicitly describe the location of data, the processing at the time of error occurrence, etc. by those physical devices while keeping in mind the physical device name of each computer that configures the system. did not become. As described above, describing the physical location of each input / output file in a program is a heavy burden on the program creator. In particular, in a large-scale system, the burden on the program creator mentioned above has increased significantly.

Further, the program created in this way is effective only for a unique system configuration and cannot be used for a distributed processing system having a different configuration. Therefore, in such a distributed processing system, it is necessary to review the contents of the program again when adding or changing the computer that configures the system or when changing the device configuration of each computer. With this, the flexibility, which is an advantage of the distributed processing system, could not be fully utilized.

Further, as is apparent from the example of FIG. 7, in the conventional distributed processing system, since the processing result is exchanged between the computers of the system, for example, the processing of the computer 12b is hindered and the processing result is If it was not transferred to the computer 12n, it was not possible to specify in which part of the program the trouble occurred.

Further, even if all jobs were processed satisfactorily, it was not possible to obtain data on which computer was operating and to what extent.

In particular, when processing a plurality of programs simultaneously in parallel, the load on each computer could not be taken into consideration.

Therefore, the problem to be solved by the present invention is to allow a distributed system to allow a systematic unified description of a program by a logical file name and to obtain a processing result equivalent to or larger than that of a large computer. To provide a system.

[0016]

In order to solve the above problems, a distributed processing system according to claim 1 of the present application is a distributed processing system in which a plurality of computers are connected to perform distributed processing of programs. One of them is a job management server that manages the entire distributed processing of the program, and another computer is a job execution server that executes a job consisting of a series of processing in the program, and the program to be distributed-processed. To
Each is a program with a logical job description consisting of a processing program for performing a certain process, an execution order of the processing programs, and an input / output file with a logical name, and the name and machine address of the job execution server are recorded in the job management server. Node table, job execution history file for recording job execution results by each job execution server, job execution for instructing distribution of job execution to each job execution server, and recording those processing results in the job execution history file Management means, and a logical / physical resource conversion table for reading the input / output file with the logical name into a file on the physical resource of each job execution server, and a job execution means for executing the job in the job execution server. , Is provided.

Further, in the distributed processing system according to claim 2 of the present application, in the distributed processing system according to claim 1, the job management server refers to the job execution history file to execute execution start time and end time of each job. And a job list including the input / output file and the execution result of the job are output.

In the above distributed processing system, the job execution management means of the job management server reads the program in which the logical job is described, and distributes and gives instructions to the job execution servers to execute the jobs constituting the program.

The job execution server instructed to execute the job by the job management server is a physical device in which the job execution server has the logical name file described in the program by the logical / physical resource conversion table. The job is actually executed by the job executing means after the file is read into the name file.

The job execution server which has finished executing the job reports the execution result of the job to the job management server.
Upon receiving the report of the job execution result, the job management server registers the execution result of each job in the job execution history file.

Further, the job management server, upon request, refers to the job execution history file and includes the job execution history, that is, the execution start time and end time of each job, the input / output file, the execution result of each job, and the like. Job list can be output.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A distributed processing system according to an embodiment of the present invention will be described below with reference to the drawings attached to the application.

FIG. 1 shows the system configuration of this distributed processing system and the flow of its processing. As shown in FIG.
The distributed processing system 1 is composed of a plurality of computers connected by a communication line 2. One of the connected computers is a job management server 3 that manages the entire distributed processing of programs, and the other computers are , 4n which execute jobs each consisting of a series of processes in the program.

The job management server 3 stores a node table 5 in which the names and machine addresses of the job execution servers 4a ... 4n are recorded, a job execution history file 6 in which job execution results by each job execution server are recorded, and job execution. Job execution management means 7 is provided for instructing and managing distribution to each job execution server and recording the processing results thereof in the job execution history file 6.

The job execution servers 4a ... 4n actually read the input / output files with the logical names into logical / physical resource conversion tables 8a ... 8n for reading the files on the physical resources of the respective job execution servers. Job execution means 9a for reporting the execution result to the job management server 3 ...
9n and.

In addition to this, the distributed processing system 1 has a job definition file 10 for supplying a program to be distributed-processed to the job management server 3. The job definition file 10 is stored in a predetermined storage device inside or outside the system, and is described by a processing program name that performs a certain processing, an execution order of the processing programs, and an input / output file with a logical name. A large number of programs P1 ... Pn (this description method is referred to as a logical job description in this specification) are stored.

In the distributed processing system 1, the programs P1 ... Pn described in the above logical job are sent from the job definition file 10 to the job execution management means 7 of the job management server 3, and the job execution management means 7 executes the program. Jobs (a series of processes in the program) that compose the programs P1 ... Pn according to the description of P1 ... Pn are distributed to the job execution servers 4a ... 4n, and their execution is instructed.

4n of the job execution servers 4a ... 4n are connected to the logical / physical resource conversion tables 8a.
8n, the logical name file in the program is read into a file on each physical resource, and the job is executed.

When the job processing is completed, the job execution means 9a ... 9n of each job execution server 4a ... 4n execute the job execution result (in this specification, the job execution result is whether the job was processed without any trouble). Signal, the time spent executing the job, the input / output file, etc.) to the job execution management means 7 of the job management server 3. In response to this, the job execution management means 7 registers the execution result of each job in the job execution history file 6. The data obtained by executing the job (this is referred to as "processing data" in this specification in distinction from the "job execution result") is sent from each job execution server 4a ... 4n to the job management server 3 and output not shown. Output through the means.

Next, the distributed processing system 1 simplifies the description of the programs P1 ... Pn to be distributed and realizes the same processing as a large-sized mainframe with expansion and flexibility. Or program P1 ... Pn
A specific example of the above will be described below.

FIG. 2 shows an example of a program described in the logical job used in the distributed processing system 1 and an explanation of each sentence. FIG. 3 is a flow diagram conceptually showing the processing flow of the program of FIG.

As shown in FIG. 2, this logically described program includes a.jcl (lines 1 to 16) and b.jcl.
The three jobs (line 17 to line 41) and c.jcl (line 42 to line 50) are described. Each job is composed of a combination of processing units called "steps" separated by "/".

Now, paying attention to the job "a.jcl", first, the job name (first line), the search order of the storage location of the load module of PGM (abbreviation of program) (second line), and the job Execution priority (3rd line).

Next, the processing of steps (4th to 10th lines) will be described. In the processing of this step, first describe the step name (5th line), then instruct execution of the processing program "pgm1" that performs certain processing (6th line), and then in the 2nd line search order. After instructing the storage location of the subordinate program (7th line), the input storage file "SYS10 se.ial" and (8th line) and the temporary file "SYS20 &ial" to be output are instructed. Where "SYS1
"0" and "SYS20" are general conventions for input and output files, and "se.ial" and "&ial" are unique names for each file.

The meaning of the above steps is to input the storage file "se.ial", process the storage file "se.ial" by the processing program "pgm1", and output the processing result to the temporary file "&ial". (See FIG. 3).

In the next step (lines 10 to 15), the execution of the program "pgm2" is instructed (line 12),
Input the temporary file "&ial" that was output earlier (line 13), and use the program "pgm2" to save the temporary file "&ia".
The result of processing "l" is the fixed file "/ fi
x / se.iam ”(line 14). The processing of the job “a.jcl” is completed by the above two steps.

Similarly, "b.jcl" inputs the fixed file "/fix/se.iam" and the storage file "se.ia2" as shown in FIG. It processes and passes the processing result for each data to the program “pgm4” on memory (this is called pipe function), processes by the program “pgm4”, and writes the processing result to the temporary file “& ia3”. Next, the temporary file "&ia3" and the fixed file "/fix/se.iam" are read, processed by the program "pgm5", and the processing result is written to the storage file "se.ia3". Further, the storage file “se.ia3” is transferred to the storage file “se.ia3” of the node B. Here, the node B indicates a specific computer connected to the distributed processing system, and the name of a specific job execution server is actually entered.

The processing of the job "b.jcl" is completed as described above. The above jobs "a.jcl" and "b.jcl" are unconditionally processed by the node A because they are written at the beginning of the program, but from the next job "c.jcl" the specified Executed in the node B.

The job "c.jcl" inputs the storage file "se.ia3" and uses this file as the program "pgm."
6 ”and the processing result is stored in the storage file“ s
e.ia4 ”is instructed to be written.

The description features of the above program will be described below. The programs processed by the distributed processing system 1 are "se.ia1", ... "se.ia4" (storage file), "&ia1", ..., "&ia3" (temporary file), "#i".
A logical file name systematically determined in advance is used, such as "a2" (pipe file).

This makes a big difference from the description of the conventional program. That is, conventionally, the above-mentioned input / output files, particularly “temporary file”, “pipe file”, etc., are considered in consideration of the physical resource configuration of the computer (here, node A and node B) where the program creator actually executes the job. A specific directory (file storage) name of a specific storage device name of each computer had to be specifically specified.

Therefore, it is necessary to check a storage device name or a directory name which is different for each job execution server for each file, which imposes a heavy burden on the program creation. The same applies to the node names (job execution server name, machine address).

On the other hand, in the program processed by the distributed processing system 1, first of all the files, "storage file", "temporary file", "pipe file"
By defining a system of file names with a predetermined name and a serial number for each type of file, the program creator can create a program without being aware of the physical environment of the job execution server.

When the above program is read into the job execution management means 7 of the job management server 3, the job execution management means 7 refers to the node table 5 and executes each job according to the node name described in the program. 4n are distributed to the execution servers 4a ... 4n, and execution is instructed.

An example of the node table 5 is shown in FIG. As shown in FIG. 4, the node table 5 lists the machine addresses and device names of the job execution servers 4a ... 4n. With this node table 5, each job execution server can be specified by searching the node name described in the program.

The job execution servers 4a ... 4n which have received the job execution command from the job execution management means 7 refer to the logical / physical resource conversion tables 8a ... 8n and copy the logical name file to the physical name of each job execution server. Read the file on the device and execute the job.

Here, the logical / physical resource conversion table 8a ...
An example of 8n is shown in FIG. Lines 6 to 8 in FIG. 5 indicate the physical device name or directory name for storing the storage file. Similarly, the 9th and 10th lines in FIG. 5 show the physical device name / directory name for storing the temporary file, and the 11th line shows the physical device name / directory name for storing the pipe file.

The logical / physical resource conversion table 8a ... 8
4n allows the job execution servers 4a ... 4n to handle files on their respective physical devices and execute jobs without any trouble.

The job execution servers 4a ... 4n that have executed the jobs in this way report the job execution results to the job execution management means 7, and the job execution management means 7 reports the execution results of these jobs to the job execution history file. Register at 6.

By referring to the job execution history file 6, the distributed processing system 1 according to the present invention can output a job list similar to a large computer such as a mainframe.

The job list includes the execution start time and end time of each job, the input / output file, and the job execution result, and can be very useful for program verification.

FIG. 6 shows an example of the job list. As shown in Fig. 6, by referring to the job list, each job (SEIA000, SEIA100 etc. in this list)
Execution start time, execution end time, input / output file, CP
Information such as the usage time of U can be obtained. If the processing is interrupted due to a defect in the program, it is possible to identify which job was executed to interrupt the processing, and it is possible to help find an error in the program.

According to the distributed processing system of the present invention, the program shows only the concept of processing (abstracted processing flow), and is not affected by the physical environment of the computer connected to the system. That is, for example, when a job execution server is added to the distributed processing system or the configuration of the storage device of a predetermined job execution server is changed, the node table and the logical / physical resource conversion table need not be rewritten without rewriting the program. It can be dealt with by some changes.

Further, in the above description, the case of processing one program has been described, but the distributed processing system of the present invention is not limited to the processing of one program, but a plurality of programs can be simultaneously executed in parallel like a large computer. Can be processed.

In this case, the job execution management means 7 assigns the job execution order to the job execution servers 4a ... 4n one after another, regardless of which program the job belongs to, according to the programs P1 ... Pn. To execute.

In each of the job execution servers 4a ... 4n, unprocessed jobs are placed in a waiting state, and the jobs are sequentially processed in the execution order. The processed job is reported to the job management server 3, and the job management server 3 outputs the processing result and processing data for each program.

[0057]

As is apparent from the above description, according to the distributed processing system of the present invention, a program can be described by a logical file name without being aware of the physical resources of the connected computer, and the program can be created. The burden on the can be greatly reduced.

Further, by locally modifying the node table and the logical / physical resource conversion table, the system configuration can be expanded / changed.

As described above, according to the present invention, it is easy to create a program, and it is possible to flexibly deal with system expansion / change, and further, in an environment in which a small computer is connected, processing similar to that of a large computer is performed. It is possible to provide a distributed processing system that realizes.

[Brief description of drawings]

FIG. 1 is a block diagram showing a system configuration and a processing flow of a distributed processing system according to the present invention.

FIG. 2 is an explanatory diagram in which an example of a program to be processed by the distributed processing system according to the present invention and its description are shown together.

FIG. 3 is a flowchart conceptually showing a processing flow of the program shown in FIG.

FIG. 4 is an explanatory diagram showing an example of a node table.

FIG. 5 is an explanatory diagram showing an example of a logical / physical resource conversion table.

FIG. 6 is an explanatory diagram showing an example of a job list.

FIG. 7 is a block diagram showing a system configuration and a processing flow of a conventional distributed processing system.

[Explanation of symbols]

 1 distributed processing system 2 communication line 3 job management server 4 job execution server 5 node table 6 job execution history file 7 job execution management means 8 logical / physical resource conversion table 9 job execution means 10 job definition file P program

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Nobuyoshi Yoshida 134 Kobe-cho, Hodogaya-ku, Yokohama-shi, Kanagawa Nomura Research Institute, Ltd. (72) Yoichi Inada 134, Kobe-cho, Hodogaya-ku, Yokohama-shi, Kanagawa Address Nomura Research Institute, Ltd. (72) Inventor Masanori Yamawaki 134 Kobe-cho, Hodogaya-ku, Yokohama-shi, Kanagawa Address Nomura Research Institute, Ltd. (72) Inventor Miho Fujishima 134, Kobe-cho, Hodogaya-ku, Yokohama-shi, Kanagawa Address Nomura Research Institute, Ltd.

Claims (2)

[Claims] 1. In a distributed processing system for connecting a plurality of computers for distributed processing of a program, one of the connected computers is a job management server for managing the entire distributed processing of the program, and another computer. Each of which is a job execution server for executing a job consisting of a series of processes in the program, and each of the programs to be distributed-processed is a processing program that performs a certain process, an execution order of the processing programs, and a logical name. A program based on a logical job description consisting of input / output files, a node table in which the name and machine address of the job execution server are recorded in the job management server, and a job execution history file in which job execution results of each job execution server are recorded. , Job execution to each job execution server And a job execution management unit that records the processing results in the job execution history file, and outputs the input / output file with the logical name to the job execution server on the physical resource of each job execution server. A distributed processing system, comprising: a logical / physical resource conversion table to be read as a job, and a job executing means for executing a job. 2. The job management server refers to the job execution history file, and starts and ends the execution time and end time of each job, an input / output file, and a job execution result.
The distributed processing system according to claim 1, wherein a job list including the job list is output.