KR101522881B1 - System and method for distributed compiling - Google Patents

Abstract

The present invention relates to a distributed compilation system and method based on sharing of source files and header file caching. In the distributed compilation method according to the present invention, a server computer and at least one remote computer connect a directory in which a source file is stored as a shared directory, and change a working directory from a remote computer to a connected directory. Then, distributed files are created in the shared directory by performing distributed compilation on the allocated files among the source files on the remote computer. Accordingly, it is possible to perform distributed compilation while minimizing work load on the server computer and the remote computer, and to maintain a certain level of performance improvement according to the number of remote computers.

Distributed compilation, H-cache, preprocessor

Description

{System and method for distributed compiling}

The present invention relates to a distributed compilation system and method, and more particularly, to a distributed compilation system and method based on source file sharing and header file caching.

In general, a distributed compilation system is a structure in which a server computer that owns a source file performs a compilation by transmitting a preprocessing result file to remote computers connected to the network, or The server computer sends the source files and header files needed for compilation to remote computers for preprocessing and compiling.

For the first structure, there is a chance to get a performance boost in a simple configuration compile. However, in the case of complicated configuration compilation, it takes a lot of preprocessing time and the multitasking overhead becomes large because the server computer handles a lot of preprocessing tasks simultaneously, and the chance of parallelism according to Amdhal's law is decreased It is difficult to obtain a performance improvement.

Also, in the case of the second structure, there is an advantage that the disadvantage of the first structure is minimized and the performance increases as the number of remote computers participating in the work increases. However, the network overhead is large in order to transmit the source and header files. In order to solve this problem, the remote computer may participate in the file transfer.

Accordingly, it is an object of the present invention to provide a distributed compilation system and method that can perform distributed compilation while minimizing workload of computers participating in compilation.

According to an aspect of the present invention, there is provided a distributed compilation method comprising: connecting a server computer and at least one remote computer so that a directory in which a source file is stored can be used as a shared directory; And performing distributed compilation on a file allocated among the source files in the remote computer to generate an object file in the shared directory.

Meanwhile, a distributed compilation system according to the present invention includes a server computer that stores a source file, and a directory that is used to store the source file as a shared directory, changes the shared directory to a working directory, And at least one remote computer for performing distributed compilation on the allocated files to generate an object file in the shared directory.

In order to attain the above object, the present invention provides a computer-readable recording medium having recorded thereon a program for causing a computer to execute the method.

According to the present invention, the server computer and the remote computers are connected to share a directory in which the source files are stored, so that the server computer and remote computers in the upper network layer are not involved in file transfer. In addition, remote computers use header file caching to reduce network overhead for sending header files and the like. Accordingly, distributed compilation can be performed while minimizing the workload of the computers participating in the compilation. In addition, since the performance improvement according to the number of remote computers can be maintained at a certain level, it can be optimized and operated in a distributed compilation environment using a plurality of computers.

Hereinafter, the present invention will be described in detail with reference to the drawings.

In the distributed compilation system according to the present invention, source files owned by a server computer are shared by other remote computers connected to the network, and remote computers execute compile operations in shared directories. With this arrangement, there is no file transfer at the higher network layer, so that the server computer and the connected remote computers do not have to engage in network access based on file transfer, thereby providing more opportunities for each computer .

1 is a block diagram of a distributed compilation system according to an embodiment of the present invention. 1, the distributed distributed compilation system includes a server computer 100 and first through nth remote computers 200a through 200n. The server computer 100 and the first through nth remote computers 200a to 200n are connected through a network.

The server computer 100 selects the first to nth remote computers 200a to 200n to perform the operation and then accesses the source directory in which the source files are stored to the first to nth remote computers 200a to 200n to the shared directory Shared directory access permission information, compile command, and directory information to be compiled. The first to nth remote computers 200a to 200n share the source directory of the server computer 100 based on the information provided from the server computer 100 and set them as a shared directory. You can use the network drive method to set up the shared directory, or you can share the source file using other methods.

The first to nth remote computers 200a to 200n perform pre-processing and compiling operations in the set shared directory. At this time, by using the header file caching including the H-cache for storing the header files requested respectively in the first to n-th remote computers 200a to 200n for quick processing of the preprocessing operation, access to the shared directory is minimized do.

That is, when the source files are shared with the shared directory, the first to nth remote computers 200a to 200n request a header file in the shared directory in the preprocessing process. In the preprocessor, , The searcher searches all the search paths set by the user until the preprocessor finds the desired file, and accesses the shared directory more than necessary. Therefore, even if there is no network overhead in the upper layer, the network is used in the lower layer. As a result, unnecessary and excessive access to the shared directory significantly reduces the preprocessing speed, and degradation of the preprocessing speed is directly related to the degradation of the entire compilation time.

In order to solve such a speed reduction, the first to nth remote computers 200a to 200n are connected to the local hard disks of the first to nth remote computers 200a to 200n by a shared directory of the server computer 100 for compilation And a header file cache (H-cache) structure that minimizes access to the shared directory by copying the header files read from the server. This structure makes use of the characteristic that each process generally requires a common header file in the process of compiling multiple source files in the same project. That is, as the compilation process progresses, the header files requested by the remote computer are stored in the local hard disk drive, and the local hard disk is accessed instead of the shared directory by accessing the stored header files, 200a to 200n) will have the same effect as if each compilation operation was done on a local hard disk, not on a shared directory.

The object files generated through the compiling process of the first to nth remote computers 200a to 200n are stored in a shared directory so that the server computer 100 can perform a link and other tasks when the server computer 100 needs it.

2 is a diagram showing a configuration of a first remote computer. The remaining second to nth remote computers also have the same configuration as the first remote computer. Referring to FIG. 2, the first remote computer 200a includes a modified preprocessor 210a, unlike the general purpose preprocessor. The preprocessor 210a receives the source file (.c) from the shared directory for the preprocessing operation and uses the H-cache during the preprocessing operation in the shared directory. The first remote computer 200a includes an RFHT (Requested Header File Table) 220a, an H-cache 221a, and an H-cache hash for utilizing the H-cache. Although the H-cache hash is shown in FIG. 2 as being provided in the preprocessor 210a, the H-cache hash may be provided separately from the preprocessor 210a.

The RFHT 220a stores the name of the header file requested from the preprocessor 210a, the absolute path to the actual location of the header file, and the absolute path of the location to be stored in the cache. The H-cache hash has an absolute path (including the file name) in which the header file is physically located, so that the data stored in the RHFT 220a can be retrieved at a high speed, and the absolute path of the cache directory in which the header file is stored Data.

With such a configuration, after obtaining information on necessary header files from the source file, if the preprocessor 210a enters the file search routine, it checks whether the header file exists in the RFHT 220a , The header file is provided in the H-cache directory 221a, or if not, the header file is directly provided to the shared directory. When accessing the shared directory and receiving a header file, the information in the header file is simultaneously updated in the RFHT 220a and the H-cache hash. The header file is copied and stored in the H-cache directory 221a, When a request for the same header file is received later, a header file is received from the H-cache directory 221a, not the shared directory.

When the preprocessed file is generated through the above process, the compiler 230a installed in the first remote computer 200a compiles the preprocessed file and generates the object file. Since all of this process is done with the shared directory set to the working directory, the created object file is stored in the shared directory even if the user or the distributed compiler does not make any settings. Accordingly, this position is the same as the position at which the actual server computer 100 compiles and the object file is generated.

3 to 5 are flowcharts provided in the description of an operation method of a distributed compilation system according to an embodiment of the present invention. First, FIG. 3 shows a process performed by the server computer 100. For convenience of explanation, only the operation between the server computer 100 and the first remote computer 200a will be described below, but the same process is performed for the remaining second to nth remote computers.

Referring to FIG. 3, the server computer 100 is connected to the first remote computer 200a through a network at the same time as the compile operation starts (S300). The shared directory access right information, the working directory information, the compile command, and the like are transmitted through the connection (S305). The first remote computer 200a processes this information to connect to the shared directory, changes the work path to the shared directory, and transmits a signal indicating the success to the server computer 100. [ When receiving a network drive request success (NDRS) signal from the first remote computer 200a in step S310, the server computer 100 receives a compile end signal from the first remote computer 200a, And waits for reception (S315). However, when receiving the NDRF (Network Drive Request Failed) signal indicating the failure of the shared directory connection, the server computer 100 performs a failed compile operation by itself (S325).

After the first remote computer 200a has undergone the preprocessing and compiling operations, the first remote computer 200a once again transmits the success or failure of the operation to the server computer 100. [ If the operation is successful, the server computer 100 terminates the current process for the next compilation operation (S320). If the compile operation fails in the remote computer 200a, the server computer 100 performs a failed compile operation The process is terminated and the process is terminated (S325).

4 shows a distributed compilation process performed on a remote computer. 4, the first remote computer 200a is connected to the server computer 100 via a network (S300). From the server computer 100, the first remote computer 200a receives the shared directory access right information, the working directory information, (S405). The first remote computer 200a connects the source directory of the server computer 100 as a shared directory by using the shared directory access right information among the received information, connects it as a network drive, tries to change the work path (S410) And transmits the signal to the server computer 100 (S415). That is, when the connection is completed and the connection directory is moved to the compile directory, the NDRS (Network Drive Request Success) signal is transmitted to the server computer 100. When the connection is failed, the NDRF signal is transmitted to the server computer 100 .

After completing the connection to the shared directory, the first remote computer 200a performs a preprocessing operation (S425). The H-cache is used during the preprocessing operation. When the preprocessing operation is completed, the first remote computer 200a performs the compiling operation (S430). The object file generated by the compilation operation is stored in the shared directory. Then, the server computer 100 transmits the success or failure of the operation to the server computer 100 again (S435). When the operation is successful, the first remote computer 200a sends a DONE signal to the server computer 100 to indicate completion of the operation, and the server computer 100 receives the DONE signal and then performs the next compilation operation.

5 is a flowchart provided in the description of a method of operating the H-cache in a remote computer. 5, if there is a read request for a header file in the preprocessing step (S500), the preprocessor 210a generates an absolute path list by combining the requested header file name and the entire path to be searched (S505) . The preprocessor 210a compares all the data of the generated absolute path list with the key of the H-cache hash to search for the corresponding header file information in the RHFT 220a (S510).

If the search result shows that the path to the requested header file is within the RHFT 220a, the preprocessor 210a switches the file search routine to the H-cache mode (S520). Then, the file search routine is entered (S525). In the process of entering a file search routine and performing a file search, the preprocessor 210a compares the absolute path to search for the header file with the absolute path used as the key value of the H-cache hash (S530). If the two paths are different from each other, the search process for the actual shared directory is skipped without updating the data of the search path list. Therefore, in this case, the shared directory is not accessed during the search routine. However, if the two paths match, the corresponding header file is read from the local H-cache directory 221a, not the shared directory (S540).

In step S515, if the information of the corresponding file does not exist in the RFHT 220a as a result of substituting the absolute path list into the H-cache hash, the general search mode is entered (S545) and the normal file search routine is entered (S550). If the search for the requested file is successful as a result of entering the general file search routine and searching for the requested file, the preprocessor 210a includes the name of the searched header file, the absolute path of the actual location, The RHFT 220a and the H-cache directory 221a store information about the searched header file (S560).

That is, the retrieved header file is copied to the H-cache so that the next preprocessing operation can find the header file from the local hard disk. At this time, information on the searched header file is also stored in the H-cache hash. The key of the header file to be stored uses the absolute path of the actual location of the header file, as described above. If the search of the requested file fails, a search failure and a compile error are generated (S565).

Through this process, the preprocessor 210a can read the header file from the local H-cache directory 221a instead of minimizing access to the shared directory. With this configuration, access to the header files except the source file can be converted to the local cache directory from the second compilation operation, and the disadvantages of the distributed compilation work using the shared directory can be minimized.

At the end of the preprocessing process, the remote computer compiles the preprocessed files. Since this compilation is performed in the same way with any compiler, the preprocessor using the H-cache does not depend on the compiler, so it can be easily applied to the compiler of various purpose machines.

6 to 10 are graphs illustrating simulation results of performance of the distributed compilation method according to an embodiment of the present invention. First, FIG. 6 shows the distributed compilation performance when the H-cache is not used. In FIG. 6, 'Local' denotes a general compile time, and 'remote' denotes a computer connected to perform distributed compilation.

7 and 8 are graphs illustrating performance of the distributed compilation method when the H-cache is used. In FIGS. 7 and 8, the general compile time is set to '1', and the degree of performance improvement according to each configuration is shown. As shown in FIGS. 7 and 8, when the number of computers connected to the distributed compilation is increased, the performance of the H-cache increases.

FIG. 9 is a graph illustrating a preprocessing time rate improvement when the H-cache is used. In FIG. 9, the item denoted by 'NDCD' is normalized to '1' as the speed when there is no H-cache. In addition, if the item indicated by 'NDCD with H-cache' has H-cache, it can be seen that the speed is improved compared to the case without H-cache.

10 is a graph showing a ratio of time for processing the H-cache at the entire compile time. That is, the ratio of the preprocessing process, the pure compilation process, and the caching process in the total compile time is shown. As shown in FIG. 10, the ratio of the time required for the caching process for processing the H-cache at the entire compile time is very small.

As described above, the distributed compilation method according to the present invention increases the compilation performance by using the H-cache. In addition, a general distributed compilation method is a method in which a higher layer transmits a preprocessed file through a socket, or transmits a source and a header file necessary for compilation, thereby causing a heavy load on the network and imposing a burden on the server computer, As the number of remote computers increases, the performance improvement is less. However, in the distributed compiling method according to the present invention, the increase in the performance improvement corresponding to the increase in the number of remote computers used is also constant.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave such as transmission over the Internet. In addition, the computer-readable recording medium may be distributed over network-connected computer systems so that computer readable codes can be stored and executed in a distributed manner.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention.

1 is a block diagram of a distributed compilation system according to an embodiment of the present invention;

2 shows a configuration of a remote computer.

FIG. 3 is a flowchart illustrating a method of operating a distributed compilation system according to an embodiment of the present invention. FIG.

5 is a flow chart provided in the description of how the H-cache operates in a remote computer, and Fig.

6 to 10 are graphs illustrating simulation results of performance of the distributed compilation method according to an embodiment of the present invention.

Description of the Related Art [0002]

100: server computer 200a-200n: remote computer

Claims (12)

Connecting the server computer and at least one remote computer so that the directory where the source files are stored can be used as a shared directory; The remote computer changing the shared directory to a working directory; The remote computer obtaining header file information of an allocated source file among the source files; Confirming whether a header file corresponding to the header file information exists in the header file cache before the remote computer accesses the shared directory; If the header file exists in the header file cache, the remote computer obtains the header file from the header file cache; The remote computer preprocessing the source file using the header file; And The remote computer performs distributed compilation on the preprocessed source file through a compiler to generate an object file in the shared directory; Lt; / RTI > The method according to claim 1, Further comprising the step of performing a link operation on the object file in the server computer. The method according to claim 1, Wherein the server computer and the remote computer are connected in a network drive manner. The method according to claim 1, Further comprising the step of, when an error occurs in the process of performing the distributed compilation, performing a distributed compilation process in which the error occurs in the server computer again. The method according to claim 1, The remote computer comprising: If the header file does not exist in the header file cache, accesses the shared directory to obtain the header file, and copies the obtained header file to the header file cache. . delete 6. The method of claim 5, Further comprising the step of generating a caching hash using the absolute path in the server computer for the header file as a key and with the absolute path of the header file cache as an element. A server computer storing source files; And A header file corresponding to the header file information is stored in a header file cache before accessing the shared directory, and a header file corresponding to the header file information is stored in the header file cache, And if the header file exists in the header file cache as a result of the check, acquires the header file from the header file cache, preprocesses the source file using the header file, At least one remote computer for performing distributed compilation on a preprocessed source file to create an object file in the shared directory; Lt; / RTI > 9. The method of claim 8, And the server computer performs a link operation on the object file. 9. The method of claim 8, Wherein the server computer and the remote computer are connected in a network drive manner. 9. The method of claim 8, Wherein the server computer performs a distributed compilation process in which an error occurs when an error occurs in the distributed compilation process. 9. The method of claim 8, The remote computer comprising:  If the header file does not exist in the header file cache, accesses the shared directory to obtain the header file, and copies the obtained header file to the header file cache. .

Images