KR20060121074A - Hybrid peer to peer network intelligent distributed compiler system and method thereof, and computer readable recording medium on which a program therefor is recorded - Google Patents

Abstract

A hybrid P2P(Peer-to-Peer) network intelligent distributed compiler system and a method thereof, and a computer-readable recording medium storing a program thereof are provided to perform a compiling process in the shorted time with maximum efficiency by concurrently advancing the compiling process after intelligently allocating large compile work to multiple remote hosts and distributing source files in a P2P mode. A management server(104) manages an idle host connection information list by connecting to the remote hosts(101,103) through the network. A local host(102) generates a dependency map of compiled files according to a distributed compile working request of the user, generates a compile working set according to the dependency map, and allocates/transmits the compile working set in the P2P mode based on the idle host connection information list. Remote hosts perform compiling for the compile working set received from the local host and transmit a corresponding result to the local host. The local host allocates the compile working set to each idle host by making similar 'CWSB(U)(Compile Working Set Bunch Union).size' allocated to each idle host.

Description

Hybrid Peer To Peer Network Intelligent Distributed Compiler System and Method about, and Computer Readable Recording Medium on which a Program therefor is recorded}

1 is a block diagram of an embodiment of a distributed compiler system according to the present invention;

FIG. 2 is a diagram for explaining a compilation working set analyzing process in a local host of FIG. 1;

3 is a flowchart illustrating a hybrid P2P network intelligent distributed compilation method according to the present invention;

4 is another exemplary flowchart illustrating a hybrid P2P network intelligent distributed compilation method according to the present invention.

* Explanation of symbols for the main parts of the drawings

101, 103: remote host 102: local host

104: management server

The present invention relates to a distributed compilation system, and more particularly, hybrid P2P which intelligently allocates compilation tasks to idle remote hosts accessible on a network and distributes source files in a peer to peer (P2P) manner to simultaneously perform a compilation process. A network intelligent distributed compiler system and a method thereof, and a computer-readable recording medium having recorded thereon a program for realizing the method.

The size of the software is getting bigger and more complicated, and accordingly, the program source file, which is a component constituting the software, is also gradually enlarged. Therefore, the time taken to compile the program source files to create an object file, and then generate the binary through a link process to produce a software product is getting longer. This time lag is a major obstacle to software product development.

The conventional compilation method is to install a native or cross compiler that creates an object that runs on the platform on which the produced software is to be run on each local host, and composes a source file and compiles it. This method can compile quickly and simply when the amount of source files is small. However, since the compilation should be done only on the local host, the time required to compile is proportionately increased when the amount of source files becomes large. Since the compilation process is a mechanical process, the knowledge worker who performs the compilation process wastes excessive time, and there is a problem that a large cost is required.

In the above-described method, a more advanced method is to preprocess a source file and distribute the preprocessed file to a remote host for compilation. This approach is fine for handling simple source files, but it is a problem for compiling complex and intensive source files. Since the preprocessing process is done on the local host, the overhead is quite large, and the preprocessed file size is several times to several tens of the source files, and the network I / O data increases in proportion to the number of remote hosts. The problem is that the time is slower than compiling only on the local host.

The present invention has been proposed to solve the above problems, by intelligently allocating a large amount of compilation work to a plurality of hosts and by distributing the source files in a peer-to-peer (P2P) method at the same time, maximizing efficiency It is an object of the present invention to provide a hybrid P2P network intelligent distributed compiler system capable of performing a compilation operation in the shortest time, and a computer readable recording medium having recorded thereon a program for realizing the method.

Other objects and advantages of the present invention can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. Also, it will be readily appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

According to an aspect of the present invention, there is provided a peer to peer network intelligent distributed compiler system, comprising: a management server for network connection with a plurality of hosts to manage a connection information list of an idle host; A local host for generating a dependency map of files to be compiled according to a user's distributed compilation task request and generating a compilation task set accordingly, and allocating the compilation task set based on the idle host access information list and transmitting the P2P scheme. ; And at least one remote host for performing a compilation on the compilation work set received from the local host and transmitting a result thereof to the local host.

In addition, the present invention is a P2P network intelligent distributed compilation method, in which any host (hereinafter referred to as a "local host") generates a dependency map of files to be compiled according to a user's request for distributed compilation, and compiles accordingly A compilation working set analyzing step of generating a working set; A network setting step of receiving a connection information list of a host (hereinafter referred to as a "remote host") available from a management server and establishing a P2P network connection with remote hosts based on the received connection information list; A compilation task set assignment step of allocating the compilation task set to each host; A P2P transmission step of transmitting the assigned compilation work set to the remote host in a P2P manner; And a compilation result receiving step of receiving a compilation result from a remote host.

The present invention also provides a compilation task set analysis function for generating a dependency map of compilation files according to a user's distributed compilation task request and generating a compilation task set accordingly; A network setting function for receiving a connection information list of available hosts from a management server and establishing a P2P network connection with remote hosts based on the received connection information list; A compilation work set assignment function for allocating the compilation work set to each host; A P2P transmission function for transmitting the allocated compilation work set to the host in a P2P manner; And a computer readable recording medium having recorded thereon a program for realizing a compilation result receiving function for receiving a compilation result from a host.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, whereby those skilled in the art may easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an embodiment of a distributed compiler system in accordance with the present invention.

First, agent software is installed on each of the hosts 101, 102, 103. In addition, specific host 104 is provided with management server software for managing and coordinating agents installed in each host. Each agent establishes a network connection with the management server.

After the network setup, when a user makes a distributed compile request from any host, the arbitrary host 102 (hereinafter referred to as "local host") performs a compile work set analysis. After analyzing the compilation set, the local host 102 requests the appropriate number of hosts (hereinafter referred to as "remote hosts") from the management server 104 according to the number of compilation operations, and the management server 104 makes available remotes. The connection information list of the hosts 101 and 103 is transmitted to the local host. In the present embodiment, only two hosts are shown as available remote hosts, but the present invention is not limited thereto.

That is, the management server 104 has information on CPU usage, memory usage, idle storage size, etc. of each host existing on the network, and uses the information to optimally idle the local host 102. A list of host connection information is created and transmitted to the local host 102.

Upon receiving the connection information list from the management server 104, the local host 102 establishes a network with the remote hosts 101 and 103 based on the connection information list. In addition, the local host 102 allocates a compilation work set to each host (including a remote host and a local host) according to a sophisticated distribution algorithm, and hosts each of the hosts 101 and 103 in a P2P manner. To be sent). Detailed description thereof will be described later.

The remote hosts 101 and 103 compile the compilation work set transmitted from the local host 102 in a P2P manner and transmit the result to the local host 102. At the same time, the local host 102 also compiles against the assigned compilation set. Compile and file send and receive operations are nested on each host (including remote hosts and local hosts), so the file send and receive time is large for the overall distributed compile time, except for the initial time when all of the file sets included in the compile set are not ready. Does not affect When all compilations are complete, the distributed compilation is finished.

FIG. 2 is a diagram for describing a compilation working set analysis process in the local host 102 of FIG. 1.

Analyzing the contents of the source file allows you to analyze the resource files (source files, text files, data files, etc.) needed to generate the object files by compiling the source files normally, and these files depend on each other. Dependency analysis on the entire source file results in a dependency map of the entire file.

Define the bundle of all files that have compile target source file and dependency link as Compile Working Set (CWS), and the number of files included in CWS is called CWS.count. Define the sum of the files included in CWS as CWS.size.

Since the local host 102 has all the source files, there is no problem, but in order for the compilation to be normally performed on the remote hosts 101 and 103, all files included in the CWS must be prepared in the remote hosts 101 and 103. Therefore, the local host 102 must send all files in the CWS to the remote host 101, 103.

Compilation command line options and source files are required to analyze the compilation set. Among the command line options are resource file search path options (/ I, -I, etc.), user-defined value options (-D, / D, etc.) and source specification options. The compiler uses the resource file search path option to locate the source file. The user-defined value option replaces the symbol value defined in the source file. Parse the parts of the specified source file that match the #include <expr> syntax and pattern. The <expr> syntax can be a constant value or a macro symbol. For symbols, it is changed to the value given by the user-defined value option. The <expr> syntax results in a file existing on the resource file search path. These files are part of the resource file needed to compile the source file. Resource files may also need other resource files internally, so all resource files are analyzed in the same process. Recursively identify all resource files needed to compile the source file.

For example, a request to compile files Ac 201, Bc 202, Cc 203, and Dc 204 was requested by the user at the local host 102. Assume that a dependency structure of the form shown in Figure 2 has been analyzed. Therefore, dependency map data is as follows.

A.c (201): E.h (205), F.h (206), G.h (207)

B.c (202): F.h (206), G.h (207)

C. c (203): H. h (208)

D.c (204):

E.h (205): I.h (209)

F.h (206):

G.h (207): J.h (210)

H.h (208):

I.h (209): K.h (211)

J.h (210): K.h (211)

K.h (211):

The Ac 201 file is compiled according to the dependency map to generate the Ao 212. For example, Eh 205, Fh 206, Gh 207, Ih 209, Jh 210, Kh 211 A file is required, and the Ac (201), Eh (205), Fh (206), Gh (207), Ih (209), Jh (210), and Kh (211) files that combine both the source file and the resource file are Ao. It becomes a compilation working set (CWS) to create a. In addition, Fc 206, Gh 207, Jh 210, and Kh 211 files are required for the Bc 202 file to be compiled to generate Bo 213. The files Bc 202, Fh 206, Gh 207, Jh 210, and Kh 211 become a compilation working set (CWS) for making Bo 213. In addition, in order to compile the Cc 203 file and generate the Co 214, the Hh 208 file is required. The Cc 203 and Hh 208 files in which both the source file and the resource file are combined are the Co (214) files. It becomes a compilation working set (CWS) to create a. Also, the compilation set (CWS) for the D.c 204 file to be compiled to create the D.o 215 is D.c 204.

Hereinafter, one embodiment of a compilation working set distribution algorithm in the local host 102 of FIG. 1 will be described.

Assuming that there are N compilations requested by the user of local host 102, N CWSs are created on local host 102. Further, assuming that the local host 102 obtains a connection information list including M-1 remote host information from the management server 104, compilation is performed on a total of M hosts (including local hosts). Therefore, there is a need for an algorithm that properly distributes N CWS to M hosts. However, excessively complex distribution algorithms use a simple and effective algorithm because the distribution process takes too long, reducing the time advantage of distributed compilation.

Since N CWSs differ in file contents, size, and number, based on these conditions, a distribution algorithm is applied to complete all compilations in the shortest time. In addition, each host has a difference in performance, so the host with the best performance finishes the allocated task in advance. Hosts that have finished all their work are redistributed with CWS that has not yet been processed by all hosts. CWS redistribution ensures that no single host is waiting for distributed compilation to finish.

The entire CWS allocated by an individual host is defined as a Compile Working Set Bunch (hereinafter referred to as "CWSB"). Distributing CWS to M hosts results in M CWSBs. CWSBs assigned to M hosts are defined as CWSB-1, CWSB-2, ... CWSB-M, respectively. Thus, the CWSB assigned to any K th host is CWSB-K. In addition, the file set obtained by performing union on the CWS files in the CWSB is a complete list of files that the local host 102 should transfer to the remote host, and the complete list of files is CWSB (U) (Compile Working Set Bunch Union). ). At this time, CWSB (U) .size is defined as the sum of the sizes of files belonging to CWSB (U), and CWSB (U) .count is defined as the number of files belonging to CWSB (U).

Compile working set file size equalization algorithm

This is a method of allocating N CWSs such that CWSB-K (U) .size allocated to each of M hosts is about the same size. This is the preferred method for compiling binary structured files. Since binary files (images, data files, etc.) have compile time proportional to the file size, allocate N CWS to each host so that CWSB (U) .size is similar.

Compile Working Set File Count Sort Algorithm

Intersection between CWS with large CWS.count increases the likelihood of duplicate files. The more files that are duplicated between CWS, the smaller CWSB (U) .count and CWSB (U) .size are, which means fewer files need to be transferred to the remote host. Thus, effective network input and output is possible. N CWSs are sorted based on CWS.count and assigned to M hosts in the sorted order.

Compile Working Set Network I / O Minimization Algorithm

Sort N CWS in descending order by CWS.size to allocate the largest N / M (large) file size CWS to the local host, and assign the remaining CWS to the remaining M-1 remote hosts. Set file number sorting algorithm. Compilation tasks assigned to the local host do not require file transfers, resulting in smaller overall network I / O.

Hereinafter, a P2P file transfer method from the local host 102 of FIG. 1 to the remote hosts 101 and 103 will be described.

In order for the remote host 101 and 103 to compile, the remote host 101 and 103 must receive all CWS files to be compiled. However, if the local host 102 uses the method of directly sending CWS files to all remote hosts 101 and 103, network input / output is concentrated on the local host 102, causing the local host 102 to act as a bottleneck. In addition, as the number of remote hosts increases, the network input / output of the local host 102 increases, and thus the time required for distributed compilation increases.

For example, "A" is a local host, "A" local host must transfer A, B, C files to "I" remote host and A, B, C, D, E to "A" remote host. Assuming you need to transfer a file, you need to transfer files directly from the local host to the remote host, so the "A" host transfers A, B, and C files to "I" and A, B, C, D and E files must be transferred. "A" host transfers a total of eight files A, B, C, A, B, C, D, and E. Although A, B, and C files are the same file, duplicate "I" and "many" hosts To transmit. In other words, the same file is repeatedly transmitted and thus time delay occurs, and the amount of network input / output is increased in the local host, thereby reducing the overall distributed compilation performance.

The present invention solves the above problem by using a file transfer method using P2P.

As described above, when the local host 102 receives a compilation job request for N CWSs from the user, the local host 102 receives M-1 remote host connection information available on the network from the management server 104. Receive the list to establish a P2P network connection with remote hosts. Thereafter, the local host 102 generates M CWSBs by allocating N CWSs for M hosts (including local hosts) using the above-described compilation working set distribution algorithm.

The local host 102 then checks the set of files in the CWSB (U) assigned to each host. The local host 102 creates a host list by mapping a host to which the file should be transmitted for each file in the CWSB (U) to be transmitted to all remote hosts. The local host 102 transmits each file and a list of hosts to which the file should be transmitted to one remote host. The remote host receiving the file stores the received file, deletes its address from the host list, and sends the file and the host list back to one of the remaining hosts. In this way, the file is sent to all remote hosts.

For example, "A" is a local host, "A" local host must transfer A, B, C files to "I" remote host, and A, B, C, D, E files to "da" remote host. Suppose that it is necessary to transmit, and in accordance with the present invention describes a method for transmitting a file in a P2P method as follows.

The local host examines the CWSBs assigned to the "me" and "many" remote hosts, and checks the host A (or, many), B (or, many), C (or, many), D (or many), or E (many) hosts. Make a list. The local host sends the file "A" and the list of "and" hosts to the "I" remote host. Receiving the A file and the "me," host list, the "me" remote host saves the file A and deletes the "me" from the "me," host list.

The "I" remote host then sends the file "A" and the "multi" host list to the "multi" remote host. Receiving the A file and the "many" host list, the "many" remote host saves the A file and deletes "many" from the "many" host list. If you delete "D" from the "D" host list, the "D" remote host terminates the file transfer because there is no more host list to pass.

In this way, P2P transfer is also performed for B, C, D, and E files to transfer the files to all hosts. From the local host's point of view, the local host only transfers a total of five A, B, C, D, and E files, and no duplicate file transfers occur. It is more efficient than the local host sending files directly to each remote host. Depending on the configuration of the CWSB, this effect can be several times higher than the direct file transfer method.

On the other hand, compared with the source file preprocessing method has the following advantages. When the source file preprocessing method is described by the compilation task analysis according to the present invention, in order to compile one CWS, all files in the CWS are made into one file through the preprocessing process, and the preprocessed file is transmitted to the remote host. You will get a compiled result. Therefore, the same file is transferred to individual remote hosts in duplicate. In particular, the larger the CWS.count, the more the preprocessing file transfer is duplicated and the network I / O is limited to the compilation performance. By using the compilation work set analysis and the P2P file transfer method according to the present invention, duplicate file transfer does not occur. Comparing the amount of network I / O generated by the preprocessing method and the compilation method according to the present invention through actual verification, the difference is about 100 times.

Hereinafter, a method of managing a source file cache at the remote host 102 of FIG. 1 will be described.

The compilation process is repeated. Therefore, the distributed compilation process is repeated. Source files necessary for each process should be prepared in the remote host (101, 103). However, duplicate transmission of the same source file in every process adds to network input / output overhead. Therefore, in the present invention, the source file cache management technique prevents the same file from being transmitted repeatedly and transmits only the changed source file or the absent file.

First, the local host 102 to transmit a source file transmits information of the source file to be transmitted to the remote hosts 101 and 103. The remote host (101, 103) receiving this information checks the source file from the source file cache manager in the host and checks whether there is a source file having the same content. If the source file corresponding to the source file information received from the local host 102 does not exist or the contents are different, the remote hosts 101 and 103 request a file transfer to the local host 102. The local host 102 that has received the file transfer request actually transmits the file to the remote host 101, 103.

The name of the file, the size of the file, the last modification time of the file, and the MD4 hash value of the file content are used as information for determining whether the file is the same source file. If all of this information is the same, it is determined as the same source file. There is some overhead in obtaining the MD4 hash value, so the MD4 hash value can be omitted as an option if necessary.

Hereinafter, the object file cache management method in the management server 104 of FIG. 1 will be described.

Input values for the compilation process are the compile command line and CWS. The output values of the compilation process are object files and compiler output messages (compiler errors / warnings, etc.). If you give the compiler the same input, you get the same output. Thus, by managing the relationship between the compiled input value and the output value, the output value can be expected when the input value is given. In addition, by caching output values for input values, you can respond to the user who requested the compilation and the object file and the compiler output message as a result of the compilation even if they do not actually compile.

Each host (including local and remote hosts) compiles the CWS using the given compile command line and produces the resulting object file and compiler output messages. At this time, the remote host 101, 103 returns the object output to the local host 102, and at the same time, the local host 102 and the remote host 101, 103 generate an object cache key and the object cache key and the object. Pass file and compiler output messages to the management server 104. The management server 104 stores the received object cache key, object file, and compiler output message in the object cache manager.

The key value for cache management is generated as follows. You need information about the compiler command line, the compile command line, and CWS. Information about the CWS is the name, size, and last modification time of each file. Each host first writes the compile command line information to a specific memory buffer, followed by the file name, size, and last modified time of each file in the CWS, in order of file name. The data created in this way is made into 64byte key using MD4 hash. CWS compilation requests with the same key value produce the same result.

When the local host 102 requests distributed compilation using the cache retrieval option, it first passes a list of object cache keys of the CWSs to compile to the management server 104. The management server 104, which has received the object cache key list, searches for the object cache manager and checks whether the corresponding object cache key exists. If the corresponding object cache key exists, the management server 104 sends the corresponding object file and compiler output message to the local host 102. The local host 102 receives the cached object file and the compiler output message and stores them as a result of compilation of the corresponding CWS. Processing of remaining CWSs whose results are not obtained through a cache search is handled by a general distributed compilation process.

Hereinafter, a method for establishing a virtual compiler environment in the remote hosts 101 and 103 of FIG. 1 will be described.

For distributed compilation, the compiling process should be performed on the remote host 101 and 103 as well as the local host 102. To do this, the same native compiler must be installed on the local and remote hosts. However, the virtual compiler environment construction technology can compile even remote hosts 101 and 103 that do not have a native compiler installed.

In the case of a remote host that does not have a native compiler installed, the native compiler execution related files are transmitted from the local host 102 before distributed compilation. In order for the native compiler execution related file received from the local host 102 to run normally on the remote host, an external environment (for example, an operating system (OS) environment) referred to when the native compiler execution related file is executed is executed by the remote host. And should be the same on the local host 102.

In general, native compilers should call APIs provided by the operating system (OS) to contact the external environment. By calling the APIs, the native compiler checks whether it is an environment in which it can run. However, since the native compiler is not normally installed in the remote host of the present embodiment, when a normal API call is made, it is determined that the native compiler execution related file of the remote host cannot be executed normally.

Therefore, when a native compiler execution related file of a remote host makes an API call, it should be determined that the environment can execute normally. To this end, hook APIs called by a native compiler execution related file of a remote host, and pass the hooked APIs to an API call delegate of the local host 102 through a network connection. The API call delegate at the local host 102 calls the API call made at the remote host instead and sends the execution result over the network to the remote host. The native compiler execution related file of the remote host determines that it is an environment in which it can be executed normally based on the execution result of the proxy call, and performs normal compilation. In other words, the same virtual compiler environment as that of the local host is established on the remote host.

3 is a flowchart illustrating an embodiment of a hybrid P2P network intelligent distributed compilation method according to the present invention.

As shown in FIG. 3, first, any one host (hereinafter referred to as "local host") receives a compilation request from a user (301) and determines whether the compilation request is a distributed compilation request (302).

If the determination result is not a distributed compilation request, the local host compiles itself (303). If the determination result is a distributed compilation request, the local host attempts a network connection with the management server (304).

If there is no network connection with the management server, the local host compiles itself (303), and if the network connection with the management server is established, the local host performs compilation working set analysis (305). In other words, it creates a dependency map of the entire file by analyzing the dependencies of the entire source file. Define the bundle of all files that have compile target source file and dependency link as Compile Working Set (CWS), and the number of files included in CWS is called CWS.count. Define the sum of the files included in CWS as CWS.size.

After the network connection with the management server, the local host requests the appropriate number of hosts (hereinafter referred to as "remote host") from the management server according to the number of compilation tasks, and obtains a list of connection information of available remote hosts from the management server. (306).

Subsequently, the local host establishes a P2P network connection with remote hosts based on the connection information list (307). The local host then allocates a compilation working set (CWS) to each host (including remote host and local host) according to a sophisticated decomposition algorithm (308). The compile work set distribution may use the above-described compile work set file size equalization algorithm, compile work set file number sorting algorithm, or compile work set network I / O minimization algorithm, but is not limited thereto. M CWSBs are created by allocating N CWSs for M hosts (including local hosts) using the compilation working set distribution algorithm.

After allocating the compilation working set (CWS), the local host transmits the compilation working set to each remote host in a P2P manner (309). That is, the local host examines a set of files in the CWSB (U) assigned to each host. The local host creates a host list by mapping a host to which the file should be transmitted for each file in the CWSB (U) to be transmitted to all remote hosts. The local host sends each file together with a list of hosts that should receive the file. The remote host receiving the file stores the received file, deletes its address from the host list, and transmits the file and the host list to one of the other hosts. In this way, the file is sent to all remote hosts.

Subsequently, the local host and the remote host receiving the compilation work set in the P2P manner perform the compilation 310, and the local host receives the compilation result from the remote host 311.

The local host then determines 312 whether a compilation error has occurred. As a result of the determination, if there is no compilation error, the local host determines whether all compilation tasks are completed (313). If not all of the compilation work is completed, the local host reassigns the compilation work set (308), and if all of the compilation work is completed, the local host terminates compilation.

On the other hand, if a compilation error exists as a result of receiving a compilation result from a remote host and determining whether a compilation error has occurred, the local host terminates compilation.

4 is another exemplary flowchart illustrating a hybrid P2P network intelligent distributed compilation method according to the present invention.

4 includes the following process between the process of performing a compilation work set analysis by the local host 305 and the process of obtaining a list of access information of remote hosts available from the management server 306.

First, it is determined whether the user has requested distributed compilation using the cache search option (401). When a user makes a distributed compile request using the cache retrieval option, the local host sends a list of object cache keys for the CWSs to compile to the management server (402), and the management server retrieves the object cache manager to find the corresponding object cache key. Check for the presence of. If the corresponding object cache key exists, the management server transmits the corresponding object file and compiler output message to the local host (403).

The local host receiving the object file and the compiler output message from the management server determines whether all compilations are completed (404), and if the compilation is not completed, obtains and distributes a list of available remote host connection information from the management server. The compilation process proceeds (306).

Meanwhile, for the above-described process, the local host and the remote host compile the CWS according to the present invention, generate an object cache key, and transmit the generated object cache key, object file, and compiler output message to the management server. The management server stores the received object cache key, object file, and compilation output message in the object cache manager (405).

The object cache key generation uses the following method. You need information about the compiler command line, the compile command line, and CWS. Information about the CWS is the name, size, and last modification time of each file. Each host first writes the compile command line information to a specific memory buffer, followed by the file name, size, and last modified time of each file in the CWS, in order of file name. The data created in this way is made into 64 byte key using MD4 hash.

In addition, FIG. 4 illustrates a process in which a local host allocates a compilation working set (CWS) to each host (including a remote host and a local host) according to a sophisticated decomposition algorithm, and transmits a corresponding file to a remote host in a P2P manner. Between processes 309, the following processes are included.

That is, before transferring the allocated file to the remote host in a P2P manner, the local host transmits information of the source file to the remote host (406). The remote host searches for a source file in the source file cache manager to check whether there is a source file having the same content, and if the file does not exist, requests the transfer of the corresponding file to the local host (407).

The compilation process of a typical compiler proceeds sequentially. Compile one source file to produce the result, then compile the next source file. When compiling a large number of source files, the rest of the source files have to wait for compilation while compiling one source file, so the total time takes a long proportional to the size of the compilation. Using the distributed compilation system according to the present invention, since it is possible to compile a plurality of source files at the same time, when using M hosts (M: the total number of hosts participating in the compilation, including the local host), the compilation latency is 1 /. Decreases to M Hosts participating in distributed compilation will vary in performance, so actual performance may be over 1 / M. As more hosts participate in distributed compilation and more powerful hosts participate, the overall compile time is inversely shorter.

As described above, the method of the present invention may be implemented as a program and stored in a recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.) in a computer-readable form. Since this process can be easily implemented by those skilled in the art will not be described in more detail.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings.

The present invention as described above, by intelligently distributing a large amount of compilation work to a plurality of hosts, there is an effect that can process the compilation work in the shortest time with maximum efficiency.

Claims (18)

Peer-to-peer network intelligent distributed compiler system, A management server for network connection with a plurality of hosts and for managing a list of connection information of idle hosts; A local host for generating a dependency map of files to be compiled according to a user's distributed compilation task request and generating a compilation task set accordingly, and allocating the compilation task set based on the idle host access information list and transmitting the P2P scheme. ; And At least one remote host for performing a compile on the compilation set received from the local host and transmitting the resulting result to the local host Distributed compiler system comprising a. The method of claim 1, The local host, A distributed compiler system that allocates a set of compilation operations to each host so that the CWSB (U) .size allocated to each host is similar. Here, CWSB (U) is a file set obtained by performing a union on compilation work sets allocated to individual hosts, and CWSB (U) .size is the sum of the sizes of files belonging to CWSB (U). The method of claim 1, The local host, A distributed compiler system for allocating a compilation working set to a host based on the number of files included in the compilation working set. The method of claim 1, The local host, Based on the sum of the sizes of the files included in the compilation set, the larger compilation set is distributed to itself, and the remaining compilation sets are allocated to the remote host based on the number of files in the compilation set. Featured distributed compiler system. The method according to any one of claims 2 to 4, The local host, For each file in CWSB (U) to be sent to each remote host, create a host list by mapping the remote host that should receive the file, and send each file and host list for each file to one remote host together. Distributed compiler system, characterized in that. Here, CWSB (U) is a file set obtained by performing a union on compilation work sets allocated to individual hosts. The method of claim 5, The remote host, And receiving information of a source file from the local host and searching for a source file cache manager in the host using the information, but requesting a file transfer to the local host if there is no corresponding source file. The method of claim 5, The management server, Receives the object cache key, the object file generated as a result of the compilation and the compiler output message from the host and stores them in the object cache manager And, upon receiving the object cache key list from the local host, retrieve the corresponding object file and compiler output message from the object cache manager and transmit them to the local host. The method of claim 7, wherein The host, A distributed compiler system that generates an object cache key by applying an MD4 hash to the file name, file size, and last modified time of each file included in the compilation command line and compilation set. The method of claim 5, The remote host, And executing a compiler execution related file received from the local host, hooking API calls to build a virtual compiler environment. P2P network intelligent distributed compilation method, A compilation task set analyzing step in which any one host (hereinafter referred to as a 'local host') generates a dependency map of files to be compiled according to a user's request for distributed compilation tasks, and generates a compilation task set accordingly; A network setting step of receiving a connection information list of a host (hereinafter referred to as a "remote host") available from a management server and establishing a P2P network connection with remote hosts based on the received connection information list; A compilation task set assignment step of allocating the compilation task set to each host; A P2P transmission step of transmitting the assigned compilation work set to the remote host in a P2P manner; And Receive compilation result step to receive compilation result from remote host Distributed compilation method comprising a. The method of claim 10, The compilation task set allocation step, Distributed compilation method, which allocates a set of compilation tasks to each host so that the CWSB (U) .size allocated to each host is similar to each other. Here, CWSB (U) is a file set obtained by unioning the compilation working sets allocated to individual hosts, and CWSB (U) .size is the sum of the sizes of files belonging to CWSB (U). The method of claim 10, The compilation task set allocation step, A distributed compilation method comprising allocating a compilation working set to a host based on the number of files included in the compilation working set. The method of claim 10, The compilation task set allocation step, Based on the sum of the files included in the compilation set, the compilation set with the largest sum is allocated to the local host, and the remaining compilation set is allocated to the remote host based on the number of files in the compilation set. Distributed compilation method, characterized in that. The method according to any one of claims 11 to 13, The P2P transmission step, For each file in the CWSB (U) to be sent to each remote host, create a host list by mapping the host that should receive the file, and send each file together with the host list for each file to one remote host. Featured distributed compilation method. Here, CWSB (U) is a file set obtained by unioning the compilation work sets allocated to individual hosts. The method of claim 14, Receiving a source file information from the local host and searching for a source file cache manager in the host using the information, but requesting a file transfer to the local host if the source file does not exist. Distributed compilation method characterized in that it further comprises. The method of claim 14, Receiving, by the management server, an object cache key, an object file generated as a result of compilation, and a compiler output message from each host and storing the object cache key in an object cache manager; And Receiving, by the management server, an object cache key list from the local host, and retrieving a corresponding object file and a compiler output message from the object cache manager accordingly; Distributed compilation method characterized in that it further comprises. The method of claim 14, The remote host hooking an API call to build a virtual compiler environment Distributed compilation method characterized in that it further comprises. A compilation task set analysis function of generating a dependency map of compilation target files according to a user's distributed compilation task request and generating a compilation task set accordingly; A network setting function for receiving a connection information list of available hosts from a management server and establishing a P2P network connection with remote hosts based on the received connection information list; A compilation work set assignment function for allocating the compilation work set to each host; A P2P transmission function for transmitting the allocated compilation work set to the host in a P2P manner; And Receive compilation result to receive compilation result from host A computer-readable recording medium having recorded thereon a program for realizing this.

Images