KR19990032182A - Buffer management device for variable length files and its method - Google Patents

Abstract

The present invention relates to a buffer management apparatus for a variable length file and a method thereof. The present invention relates to a buffer management apparatus for providing files provided from a server to a client in a network configured as a client / server. A hard disk, a shared memory buffer that classifies and stores the contents stored in the hard disk into two types according to the size of the data, and checks whether the file is stored in the shared memory buffer when the client requests a file from the server. A main memory resource of the computer system by providing a device configured as a buffer manager which delivers the contents of the shared memory buffer to the client if stored, and searches the hard disk and delivers the stored contents to the client if not stored. Not waste The advantage is that it can be managed effectively.

Description

Buffer management device for variable length files and its method

The present invention relates to a buffer management apparatus for a variable length file and a method thereof, and more particularly, to a buffer management method for a file having a variable size distribution, wherein a shared memory buffer includes two data pages and a metadata page. The present invention relates to a buffer management apparatus and a method for controlling a variable length file that can be efficiently managed without wasting main memory resources of a computer system when a read request for a file stored in a hard disk occurs.

The modern computing environment has become an urgent need for effective and rapid processing of intensive search requests for various materials in terms of size and content due to the rapid development of the Internet, popularization of client / server technologies, and multimedia technologies. In order to satisfy these demands, a buffer management technique for loading and processing file data stored in a hard disk into main memory is used.

In this case, in order to perform the above function, the buffer management apparatus may consider loading the contents of the file into the main memory buffer or loading all the files into the buffer before the request for any file occurs. have. However, in the former case, since the contents of the request cannot be recognized in advance, it is impossible to predict in advance the arbitrary file to be requested and load it into the buffer. In the latter case, the size of the hard disk (the size of the entire file) Cannot be larger than the main memory size of the computer system, and also because the price of the main memory is much higher than the price of the disk.

Therefore, most existing systems use a method of loading data that is likely to be requested in a limited main memory buffer. In this method, the performance is improved by applying a buffer replacement method to a unit of a page or block having a fixed size to increase the hit rate for each request.

However, in the case of buffer management for file data of various sizes using uniformly sized pages or blocks, fragmentation occurs largely, which is unsuitable in terms of efficiency of computer resources, and the hit ratio is severely degraded. .

Accordingly, in the present invention, in the buffer management method for a file having various size distributions to compensate for the above disadvantages, the shared memory buffer is composed of two data pages and a metadata page, which are stored on a hard disk. An object of the present invention is to provide a buffer management apparatus and method for a variable-length file that can be efficiently managed without wasting main memory resources of a computer system when a read request for a file occurs.

The present invention provides a buffer management apparatus for providing files provided from a server to a client in a network configured as a client / server, comprising: a hard disk storing and managing files provided by a server, and a size of data stored in the hard disk; The shared memory buffer, which is classified into two types according to the storage, and when the client requests a file from the server, checks whether the file is stored in the shared memory buffer, and if so, delivers the contents of the shared memory buffer to the client. If it is not stored provides a device configured as a buffer manager for searching the hard disk and delivers the contents stored on the hard disk to the client.

In the above-configured apparatus, the shared memory buffer is configured by two types of large pages and small pages, stores files according to the configuration, and receives a file request from a client. A first step of checking whether a file is desired by the client, and if the file desired by the client is found in the shared memory buffer, the contents of the shared memory buffer are read and delivered to the client, The second step of reading a file desired by the client and the size of the file read from the hard disk in the second step is checked and stored in the buffer if the file size is not too large. Otherwise, the contents of the file are bypassed. Consisting of a third step of delivering to the client Provides a buffer management method for variable length files.

1 is a block diagram of a buffer management apparatus for a variable length file according to the present invention;

2 is a block diagram of a buffer manager of the present invention;

3 is a flowchart illustrating a buffer management method of a variable length file according to the present invention.

<Explanation of symbols for main parts of drawing>

10: buffer manager 11: path control module

12: File Management Module 13: Index Management Module

14: page management module 15: concurrency control module

20: shared memory buffer 21: metadata page

22: Big Data Pages 23: Small Data Pages

30: hard disk

Hereinafter, with reference to the accompanying drawings will be described in more detail the present invention.

1 is a block diagram of a buffer management apparatus of a variable length file according to the present invention, FIG. 2 is a block diagram of a buffer manager of the present invention, and FIG. 3 is a buffer management method of a variable length file according to the present invention. Processing flow chart.

Referring to FIG. 1, in order to efficiently provide files provided from a server to a client in a network configured as a client / server, a buffer management apparatus provided by the present invention includes a hard disk 30 storing and managing files provided from a server. The shared memory buffer 20 classifies and stores the contents stored in the hard disk 30 into two types according to the size of the data, and when the client requests a file of the server, the file is stored in the shared memory buffer 20. Check whether it is stored, and if so, deliver the contents of the shared memory buffer 20 to the client. If not, the buffer searches for the hard disk 30 and delivers the contents stored in the hard disk 30 to the client. It consists of a manager 10.

When a plurality of clients request file data stored in a server, the buffer management apparatus having the above configuration searches for the shared memory buffer 20 in the buffer manager 10 and transmits the shared memory buffer 20 to the shared memory buffer 20. Deliver a copy of the loaded file to the client.

In this case, the shared memory buffer 20 uses a multi-stage buffer for file data of variable size. The multi-stage buffer includes two types of pages, that is, a page which is an operation unit constituting the buffer, a large page and a small page. Means a group. In FIG. 1, in the process of loading a file stored in the hard disk 30 into a buffer, a large data page 22 and a small data page 23 are simultaneously loaded into a buffer, thereby ensuring the integrity of the contents of the data. And it also consists of a metadata page 21 for managing the metadata for quick retrieval of data.

As such, the reason why the shared memory buffer 20 is composed of pages having two kinds of sizes is to minimize internal fragmentation of data, and the size of files stored on the disk is very diverse. Likewise, if the shared memory buffer is composed of uniformly sized pages, internal fragmentation may occur largely. However, as in the present invention, maintaining the contents of a file by combining a large page and a small page has an advantage of minimizing fragmentation.

For example, if you set the size of a large data page to 10 bytes and the size of a small data page to 3 bytes, if you load a 15-byte file into the buffer, you will have 1 large data page and 2 small data pages for that file. Can be used. If the buffer configuration is uniformly organized (assuming that each page is 10 bytes in size) as in the conventional method, 5 bytes are wasted because a 20-byte buffer is allocated to a 15-byte file. If the size of each page is minimized in the conventional uniform page system, the waste ratio decreases, but there is a problem in that the size of the metadata for each page increases. Therefore, if the buffer configuration is multi-stage, the problem of memory waste and metadata increase can be solved at the same time.

In addition, in order to guarantee the integrity of the contents of the data and to search for the data quickly in an environment where user requests occur simultaneously, the metadata must be referred to each request. The contents of the meta data are index data, replacement information data (LRU-chain), and page state data, and the processing of these metadata is performed in the buffer manager 10.

FIG. 2 is a block diagram of the buffer manager 10. Referring to FIG. 2, the buffer manager 10 includes a path control module 11, a file management module 12, an index management module 13, and a page. It consists of a management module 14 and the concurrency control module 15.

The path control module 11 determines whether or not a file requested from a client is stored in the shared memory buffers 22 and 23 so that the contents of the hard disk are stored in the shared memory buffer. The path control for the file requested by the path control module 11 first performs a path control on the requested file by comparing the result of calculating the size of the requested file with a path control constant. At this time, the path control constant is set to be greater than 0 and less than 1 so as to be optimized in any environment to which the present invention is applied.

In addition, the file management module 12 buffers the contents of the file using the index management module 13 and the page management module 14 in response to a request for a file whose path is determined from the path control module 11. , 23) or deliver to the client. When the file management module 12 writes the requested file to the buffers 22 and 23, the file management module 12 determines the page allocation according to the size of the file, and if there is no spare of the buffers 22 and 23, the page to be replaced. Make a decision.

The index management module 13 performs a function of searching a file stored in the buffers 22 and 23 and a function of searching a page to which the corresponding file is assigned. To search for a file, we apply a hash with the file name as the primary key. The result of retrieving the index of the hash structure as the file name is the start page address of the file. If the search result is negative, the searched file does not exist in the buffers 22 and 23.

The page management module 14 functions to copy the contents of a corresponding file to be loaded into a buffer among arbitrary files to a page under the control of the file management module 12 and to link the respective copied pages by a link. In charge of. In addition, when any file stored in the buffer is released, the pages belonging to the released file are returned to the metadata page.

On the other hand, the concurrency control module 15 performs a concurrency control function to ensure the integrity of the contents of the file loaded in the buffer, when a plurality of requests for the file from multiple clients occur at the same time If the seriality of the operation of loading and unloading an arbitrary file is not guaranteed, the contents of the buffer will be defective. In this case, the operation of loading and unloading a file that guarantees seriality refers to an operation that ensures that no interruption or interruption occurs between the start and end of each operation. Therefore, concurrency control is performed by using semaphores provided by the operating system of UNIX.

3 is a flowchart illustrating a method for managing a buffer of a variable length file according to the present invention. Referring to FIGS. 1 and 3, first, an arbitrary file request is received from a client (s101), and the shared memory buffer 20 is processed. Check whether the desired file exists (s102), and if the desired file exists, the file is read from the shared memory buffer 20 (s110) and the contents of the file are transmitted to the client (s109) and the next request is made. Wait

However, if the file requested by the client does not exist in the shared memory buffer 20, after reading the desired file from the hard disk 30 (s103), the size of the file is checked (s104) by checking the size of the file (s104). If is not too large, it stores in the buffer, otherwise it performs the path control (s105, s106, s107, s108) bypassing the buffer and passing the contents of the file to the client.

At this time, the path control is to determine whether to load and maintain the requested file in the buffer, and the file determined to be stored in the buffer performs page replacement by applying the LRU method to the buffer and does not store the buffer. The determined file is immediately passed on to the user.

As a specific method for the path control, first, the size of the file read from the hard disk 30 as described above (s103) is obtained (s104), and the file is determined by multiplying the size of the entire buffer by the determination constant determined by the system administrator. The result of subtracting the size of is set as a determination variable (s105). Then, it is determined whether the determination variable is greater than or less than zero (s106). If the determination variable is larger than zero, the read file is stored in the buffer (s108). Otherwise, the buffer is bypassed (s107), and the file contents are transmitted to the user. Deliver (s109).

In this case, the meaning of the path control in the present invention is to reduce the adverse effects caused by loading a file that is too large in the buffer, and the feature that does not occur when performing buffer replacement for data of fixed size is variable size. This happens in page replacements for data, which is the rate of buffer change that occurs in a single page replacement.

Since the buffer change rate is always constant at a fixed size, it has no effect on the buffer replacement and hit rate. However, when a large file is loaded into the buffer at a variable size, the buffer change rate is increased, thus affecting the hit rate for the next request. . That is, if five files in the buffer are replaced by a new request on the assumption that 10 files exist in the limited buffer, six files remain in the buffer. Therefore, if the total number of files is 20, the buffer hit rate due to this request is reduced from 10/20 to 6/20. In this case, the path control principle uses a method that does not store in the buffer.

The apparatus and method of the present invention as described above can achieve a much faster speed improvement than the method of accessing the hard disk every time to read the file data, and keep the contents of the file by combining the big page and the small page to minimize the fragmentation. As a result, it can be effectively managed without wasting the main memory resources of the computer system.

Claims (5)

A buffer management apparatus for providing a client with files provided from a server in a network configured as a client / server, A hard disk that stores and manages files provided by the server, A shared memory buffer for classifying and storing the contents stored in the hard disk into two types according to the size of data; When the client requests a file from the server, the client checks whether the file is stored in the shared memory buffer, and if the file is stored, delivers the contents of the shared memory buffer to the client. Buffer management device for a variable length file, characterized in that the buffer manager for delivering the contents to the client. The method of claim 1, The shared memory buffer For variable sized file data, multi-stage buffer is used, which consists of two groups of pages, which are the unit of operation that constitutes the buffer, of large page and small page. And a metadata page for ensuring the integrity of the contents of the data and for managing the metadata for rapid data retrieval. The method according to claim 1 or 2, The buffer manager A path control module for determining whether to store the contents in the shared memory buffer when a file requested from a client is not stored in the shared memory buffer and the contents of the hard disk are referred to; A file management module for storing the contents of the file in the shared memory buffer or transferring the file to the client with respect to the file whose path is determined from the path control module; An index management module for searching a file stored in the shared memory buffer and searching for a page to which the file is allocated; A page management module for copying the contents of a corresponding file to be loaded into the shared memory buffer among arbitrary files to a page and linking the copied pages to each other by a control of the file management module; And a concurrency control module configured to perform a concurrency control function to ensure the integrity of the contents of the file loaded in the shared memory buffer. In the buffer management method for pre-loading files provided from the server in a network configured as a client / server in a shared memory buffer, and to provide to the client when a request from the client, The shared memory buffer is configured by two types of a large page and a small page to store files according to the configuration, and upon receipt of a file request from a client, a method for checking whether the shared memory buffer includes a desired file. With step one, A second step of reading the contents of the shared memory buffer to the client if the file desired by the client exists in the shared memory buffer as a result of the checking of the first step; And a third step of checking the size of the file read from the hard disk in the second step and storing the file in a buffer if the size of the file is not too large, otherwise transferring the contents of the file to the client by bypassing the buffer. How to manage buffers of variable length files. The method of claim 4, wherein The third step is To determine whether the size of the file read from the hard disk is a size that can be stored in the buffer, the determination variable is a value obtained by subtracting the size of the file from a value obtained by multiplying the total buffer size by a determination constant determined by a system administrator. If is not greater than 0, the contents of the file is stored in a buffer, and if greater than 0 is not stored in the buffer.