KR100753831B1 - Object data storage system and method to enhance IO performance in object storage system - Google Patents

Abstract

The present invention relates to a fast object input and output processing apparatus and method in an object-based storage system, the window generation unit for generating an object prefetch window of a certain area in the memory and the data size on the disk corresponding to the data access request of the client If larger than an object prefetch window, a window controller for expanding an area of the object prefetch window and a portion of data on the disk corresponding to the size of the object prefetch window before expansion are loaded into the object prefetch window before expansion. A data loader which loads the rest of the data on the disk in the extended area of the object prefetch window while providing a portion of the data loaded in the object prefetch window before expansion to the client, thereby providing a high speed in the object-based storage environment. Provide data input and output.

Object Prefetch Window, Client, Fast Data I / O

Description

Object data storage system and method to enhance IO performance in object storage system

1 is an embodiment of a system component constituting an object-based storage environment.

2 is a diagram showing system components constituting the object storage management system according to a preferred embodiment of the present invention.

3 is a diagram illustrating a configuration of an object prefetch window according to a preferred embodiment of the present invention.

4 is a flowchart illustrating a state change of an object prefetch window according to an exemplary embodiment of the present invention.

5 is a diagram illustrating a method for loading blocks of an object prefetch window area into an object cache according to an exemplary embodiment of the present invention.

6 is a flowchart of managing an object data request of a client through an object prefetch window according to a preferred embodiment of the present invention.

7 is a block diagram of an object storage management system according to a preferred embodiment of the present invention.

8 is an example of a flowchart of a method for providing object data to a client through an object prefetch window according to the present invention.

9 is an example of a flowchart of a method of managing an object prefetch window according to the present invention.

10 is an example of a flowchart for a state change of an object prefetch window.

The present invention relates to a method for providing high-speed data input / output in an object-based storage system, and more particularly, to load object data on a disk into a memory by creating, expanding, reducing, merging, and removing an object prefetch window. An object storage management apparatus and method for supporting high-speed data input and output in a network environment.

Most Internet service environments use storage environments such as DAS, NAS, and SAN to efficiently process such a large amount of data.

DAS connects block-based storage devices directly to the host computer's input and output buses via SCSI or IDE, and provides high performance and robust data protection, but there are limitations to the expansion of storage devices.

NAS is a structure that manages all metadata in one server, which describes how files are stored in the storage device in order to provide file services to enable data sharing among hosts with different platforms. Since I / O passes through one server, there is a problem that the file server is a bottleneck.

SANs are designed to address storage device connectivity constraints and storage device sharing issues, and to support fast, scalable interconnects to support more client hosts and storage devices. However, SANs must be able to understand the file system's metadata on different platforms in order for storage applications such as file systems to share data, and are limited by storage scalability because separate expensive devices are required.

Recently, the object-based storage technology, which has been recently issued to solve this problem, suggests a method to solve the problems of the aforementioned storage system. In other words, object-based storage technology integrates existing NAS and SAN concepts through object interfaces, and satisfies high performance, high scalability, and cross-platform data sharing.

However, even in an object-based storage system optimized for an object-based storage environment, since data of an object is stored on a physical disk, performance degradation is inevitable due to a relatively slow disk access compared to memory.

An object of the present invention is to provide an object storage management apparatus and method for providing a high-speed data input and output by effectively prefetching the object data stored in the object storage server in the object-based storage environment.

SUMMARY OF THE INVENTION The present invention has been made in an effort to provide a computer-readable recording medium having recorded thereon a program for executing an object storage management method on a computer.

In one embodiment of the object storage management system according to the present invention for solving the technical problem, a communication manager for receiving a data access request of the client and transmits the data requested by the client to the client; An object storage manager for storing, deleting, and retrieving data requested by the client on a disk; An object cache manager which transmits the loaded data to the client when the data requested by the client is loaded on the memory; An object prefetch manager for allocating an area so that a portion of data requested by the client is loaded into the memory, and expanding the area of the memory to load the remaining portion of the data while providing the data loaded on the memory to the client; And an object input / output manager configured to transmit the data according to the memory area in which the object prefetch manager requests the data stored on the disk.

According to an embodiment of the present invention, there is provided an apparatus for managing object storage, comprising: a window generating unit generating an object prefetch window of a predetermined area in a memory; A window controller configured to expand an area of the object prefetch window when a data size on a disk corresponding to a data access request of a client is larger than the object prefetch window; And loading a portion of data on the disk corresponding to the size of the object prefetch window before expansion into the object prefetch window before expansion and providing the client with a portion of data loaded in the object prefetch window before expansion. And a data loader for loading the rest of the data on the disk in the extended area of the object prefetch window.

According to an embodiment of the present invention, there is provided a method of managing object storage, the method including: generating an object prefetch window of a predetermined area in a memory; Expanding an area of the object prefetch window if the data size on the disk corresponding to the data access request of the client is larger than the object prefetch window; And loading a portion of data on the disk corresponding to the size of the object prefetch window before expansion into the object prefetch window before expansion and providing the client with a portion of data loaded in the object prefetch window before expansion. Loading the remaining portion of data on the disk into an extended area of the object prefetch window.

The present invention also includes a computer-readable recording medium having recorded thereon a program for executing a method of providing a high speed data input / output in a computer in an object-based storage system according to the present invention.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention.

1 is an embodiment of a system component constituting an object-based storage environment. Referring to FIG. 1, an object-based storage system includes a client file system 110, a metadata server 120, and an object storage server 130.

In the object-based storage environment, the client file system 110, the metadata server 120, and the object storage server 130 communicate with each other through a network to provide input / output of data.

The object storage server 130 reads data of the object stored in the storage device and returns the data to the client file system 110.

The object storage server 130 provides a dedicated object-based storage system for efficiently storing and managing objects in a storage space.

2 is a diagram illustrating system components constituting an object storage management system according to a preferred embodiment of the present invention. 2, the object storage management apparatus according to the present invention includes a communication manager 231, an object storage manager 232, an object cache manager 233, an object prefetch manager 234, an object input / output manager 235, and It is configured to include a disk device 240.

The communication manager 231 manages a network channel for accessing an object in the client file system 110 and performs data transmission and reception with each other. That is, when the object access request is requested, the communication manager 231 receives the client's request and delivers it to the object storage manager 232.

The object storage manager 232 manages object input / output operations such as creation / deletion / recording / retrieval / property setting / property change of an object requested by a client. It analyzes client's object access request and analyzes which operation corresponds to and handles each operation.

The object cache manager 233 manages the object's data on memory to reduce the cost of accessing the object data. If the data block of the object requested by the client exists in the cache, the cache data is returned immediately without disk access. If the data block does not exist in the cache, the block on the disk is read, loaded into the cache, and returned to the client.

The object prefetch manager 234 manages high-speed data input / output by minimizing the number of disk input / output. By defining the object prefetch window and loading blocks into the cache in advance, blocks existing in the object prefetch window area can be minimized for disk access to the object data requested by the client.

The object input / output manager 235 is responsible for input / output with a physical disk device. Basically, I / O is done by one block unit on disk. If there is an I / O request as much as object prefetch window area by object prefetch manager, I / O unit is minimized by extending I / O unit to object prefetch window area.

Disk device 240 is a disk device connected with an IDE or SCSI device to provide space for storing objects.

3 is a diagram illustrating a configuration of an object prefetch window according to a preferred embodiment of the present invention. Referring to FIG. 3, the components of the object prefetch window include an object window identifier 311, an object identifier 312, an object size 313, an object window creation time 314, an object window start position 315, An object window end position 316, a data access distribution 317 within the object window, an object window reference count 318, and an object window change count 319.

The object window identifier 311 is a value assigned every time an object prefetch window is generated and used to distinguish each object prefetch window.

The object identifier 312 is an identifier of an object that the object prefetch window is responsible for.

The object size 313 is the total size of the object indicated by the object identifier, which is the maximum value to which the object prefetch window area can be expanded.

The object window creation time 314 is a time at which the object prefetch window is created and is used to remove an object prefetch window that has not been used for a long time.

The object window start position 315 is the start position of the object prefetch window region and indicates a position within the object.

The object window end position 316 is the end position of the object prefetch window region and indicates a position within the object.

The data access distribution 317 within the object window is a value representing the access distribution of the location requested by the client in the object prefetch window area. When the access distribution is frequently made at the beginning of the object prefetch window area, the object prefetch window It is used to adjust the object prefetch window area according to the client's access distribution, such as reducing the end position of and increasing the start position of the object prefetch window if frequently made at the end of the object prefetch window area.

The object window reference count 318 and the object window change count 319 are values that are updated whenever an access to the object prefetch window area occurs, and are used for resizing or removing the object prefetch window.

4 is a flowchart illustrating a state change of an object prefetch window according to an exemplary embodiment of the present invention. Referring to FIG. 4, the object prefetch window has a state of CREAT 410, EXPAND 420, SHRINK 430, MERGE 440, and DESTROY 450.

The CREATE 410 state is a state when the object prefetch window is first created, and creates a new object prefetch window when the data block requested by the client does not belong to the area of existing object prefetch windows.

The EXPAND 420 state transitions at a time when the location of the object that the client accesses is expected to leave the object prefetch window area. That is, when it is determined that the client is out of the object prefetch window area, the client extends the size of the object prefetch window and reads the data of the extended area from the disk and loads the data in the object cache space.

The SHRINK 430 state transitions when the size of the object prefetch window is reduced when the client access frequency is small for the already created object prefetch window area. In other words, the object prefetch window that is not used recently is less frequently accessed by the client, so the object prefetch window size is reduced to avoid wasting unnecessary memory space. Blocks in the cache space corresponding to the reduced object prefetch window area are removed from the cache area when the memory space is insufficient according to the cache memory policy, and left in the cache space to avoid disk access when the client accesses it in the future if there is enough cache space. .

The MERGE 440 state transitions when merging into one object prefetch window to reduce unnecessary computation when existing object prefetch windows are adjacent to each other in the process of expanding or contracting. The MERGE 440 transitions to the EXPAND 420 and SHRINK 430 states according to the change in the position of the object approached by the client. The MERGE 440 transitions to the DESTROY 450 state when the frequency of use is small.

The DESTROY 450 state transitions when deleting the object prefetch window if the client has less access to the object prefetch window. Blocks of cache space corresponding to the deleted object prefetch window area are removed from the cache area when there is insufficient memory space according to the cache memory policy, and if there is enough cache space, it is left in the cache space to avoid disk access for future client access. .

5 is a diagram illustrating a method for loading blocks of an object prefetch window area into an object cache according to an exemplary embodiment of the present invention. Referring to FIG. 5, according to the method of loading blocks of the object prefetch window area according to the present invention, the state of the object prefetch window area 510 has an object prefetch window start position of 80 and a last position of In the state of 102, when the client requests a block of 100 positions, the process of expanding the object prefetch window and the process of loading the data will be described with reference to FIG. 5.

When the client requests a block at position 100, the object storage manager finds a block at position 100 in the object cache space and returns the data on the cache if it exists in the cache space, and reads from the disk if it does not exist in the cache space. After loading, it returns to the client and sends the information of the block read to ③④⑤ object prefetch manager to be reflected in the object prefetch window.

Because the 100 blocks requested by the client do not reflect the object prefetch window and return a block of the object cache space immediately, the performance degradation of the client does not occur due to the cost of managing the object prefetch window.

The object prefetch manager determines whether the window area is expanded by examining the object prefetch window to which the block at position 100 belongs.

Because the last position of the current object prefetch window is 102 and the client's distribution of access to the object prefetch window area is 98, this next client's access position is more likely to leave the object prefetch window area, thus increasing the object prefetch window area. It extends up to 109 and reads the data blocks of the extended area from disk and loads them into the object cache space.

A client can immediately return a block of object cache space without disk access for a request for a block belonging to the extended object prefetch window area.

However, the numerical values shown in FIG. 5 are for illustrative purposes and should be applied in consideration of the application environment or system configuration of the client when applying the present invention.

6 is a flowchart of managing an object data request of a client through an object prefetch window according to a preferred embodiment of the present invention. Referring to FIG. 6, a block of data requested by the client is first requested and returned to the object cache manager (S610), and information about the block requested by the client is transferred to the object prefetch manager.

The object prefetch manager checks whether the received block belongs to existing object prefetch windows (S620).

If it does not belong to any existing object prefetch window, a new object prefetch window is generated (S630) and the object cache manager is requested to load data blocks corresponding to the new object prefetch window area from the disk (S640).

If the data block belongs to an existing object prefetch window, information such as window_pin_offset, reference_counter, and modify_counter is modified (S650), and the value of the object prefetch window table is used to determine whether to change the state of the object prefetch window. (S660).

If the object prefetch window size needs to be increased, data blocks belonging to the increased object prefetch window area are read from the disk, loaded into the object cache space (S671), and the object prefetch window is changed (S672).

If the size of the object prefetch window needs to be reduced, the object cache manager is notified of data blocks of the reduced object prefetch window area (S681).

The object cache manager receiving the data block information of the reduced area can keep or remove the cache space according to the state of the cache space.

7 is a block diagram of an object storage management system according to a preferred embodiment of the present invention. Referring to FIG. 7, the apparatus for managing object storage according to the present invention includes a receiver 700, a window existence determiner 710, a window generator 720, a window size determiner 730, a window controller 740, The data stacker 750 includes a data access frequency determiner 760, a window manager 761, a window remover 762, and a data transmitter 770.

The receiving unit 700 receives a request for access to data of the client, and the existence of a window determines whether the window exists. The determining unit 710 determines whether an object prefetch window corresponding to the data requested by the client exists.

As a result of the determination, if there is no window, the window generation unit 710 generates an object prefetch window for loading the data requested by the client.

The generated object prefetch window is determined by the window size determining unit 730 to have an area of a size enough to load the data requested by the client, and is expanded to an appropriate size through the window control unit 740.

The data loader 750 loads the data requested by the client in the generated object prefetch window, and provides the data requested by the data transmitter 770 to the client.

The generated object prefetch window is determined by the data access frequency determining unit 760, when the access request frequency for the object prefetch window is more than a predetermined number of times, the window manager 761 adjusts the size of the object prefetch window. Manage.

If there is no access request for a certain period of time, the window removing unit 762 removes the object prefetch window.

8 is an example of a flowchart of a method for providing object data to a client through an object prefetch window according to the present invention. Referring to FIG. 1, when an access request for data of a client is received (S810), it is determined whether the client may load the requested data into an already created object prefetch window (S820).

If it can not be loaded as a result of the determination in step S820 generates a new object prefetch window (S830) and loads the data requested by the client in the object prefetch window and provides to the client (S840).

If it is maintained by counting whether the client's access request for the data provided in step S840 is maintained for a predetermined time (S850), it is again determined whether there is an unloaded portion of the data requested by the client in the object prefetch window (S860). ).

 As a result of the determination in step S860, the size of the object prefetch window is expanded (S870) to load the unloaded portion into the expanded object prefetch window (S880) and to provide the loaded data to the client (S890). .

 When it is determined in step S850 that it is not maintained or when it is determined that it does not exist in step S860 it is only necessary to complete the transmission for the data provided to the client in step S840.

9 is an example of a flow diagram for a method for managing an object prefetch window according to the present invention. Referring to FIG. 9, when the access request frequency of the client for the existing object prefetch window is measured (S910), if the access request continues for a predetermined time, the size of the object prefetch window is expanded (S920).

If the access request frequency is less than a certain number of times, the size of the object prefetch window is reduced (S930). If there is no access request for a predetermined period of time, the object prefetch window is removed (S940).

During expansion (S920) or reduction (S930), it is determined whether already created object prefetch windows are adjacent (S950). If the regions of the object prefetch windows are adjacent to each other, the object prefetch windows are merged (S960).

10 is an example of a flowchart for a state change of an object prefetch window. Referring to FIG. 10, if the client cannot load data requested in the already created object prefetch window (S1000), a new object prefetch window is generated (S1010).

If the client determines that a part of the requested data is out of the object prefetch window area, the size of the object prefetch window is extended (S1030).

If the frequency of the client's access request for the object prefetch window does not exceed a certain number of times (S1040), the size of the object prefetch window is reduced (S1060), unless the client's access request is absent for a certain period of time (S1060).

If there is no access request for a certain period of time, the object prefetch window is removed (S1090).

If the expanded object prefetch window belongs to an area adjacent to each other (S1070), the object prefetch windows are merged (S1080) to manage one object prefetch window.

The invention can also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD_ROM, magnetic tape, floppy disks, and optical data storage, and may also include those implemented in the form of carrier waves (e.g., transmission over the Internet). . The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

As described above, optimal embodiments have been disclosed in the drawings and the specification. Although specific terms have been used herein, they are used only for the purpose of describing the present invention and are not intended to limit the scope of the present invention as defined in the claims or the claims. Therefore, those skilled in the art will understand that various modifications and equivalent other embodiments are possible from this. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

The present invention provides a high-speed data input and output through a method of efficiently prefetching object data stored in an object storage server in an object-based storage environment, thereby providing a large amount of data such as a server, a VoD server, or a storage leasing service running various application software. It can be used as a storage platform such as an operating environment that requires data transfer, clustering computers that require high-performance data input and output, and can provide stable and fast data services.

Claims (14)

A communication manager which receives a data access request of a client and transmits the data requested by the client to the client; An object storage manager for storing, deleting, and retrieving data requested by the client on a disk; An object cache manager which transmits the loaded data to the client when the data requested by the client is loaded on the memory; An object prefetch manager for allocating an area so that a portion of data requested by the client is loaded into the memory, and expanding the area of the memory to load the remaining portion of the data while providing the data loaded on the memory to the client; And And an object input / output manager configured to transmit the data according to the memory area requested by the object prefetch manager to the data stored on the disk. The method of claim 1, wherein the object prefetch manager And extending an area of the memory by a predetermined unit until the entire data requested by the client is loaded in the remaining portion of the data which is not loaded from the disk into the memory. A window generator generating an object prefetch window of a predetermined area in a memory; A window controller configured to expand an area of the object prefetch window when a data size on a disk corresponding to a data access request of a client is larger than the object prefetch window; And Loading a portion of data on the disk corresponding to the size of the object prefetch window before expansion into the object prefetch window before expansion and providing the client with a portion of data loaded in the object prefetch window before expansion And a data loader for loading the rest of the data on the disk in the extended area of the object prefetch window. The method of claim 3, wherein the window generating unit And a new object prefetch window is generated when data corresponding to the access request of the client is not loaded on the already created object prefetch window. The method of claim 3, wherein the window control unit The size of the object prefetch window is extended by a predetermined unit until the entire data requested by the client is loaded in the remaining portion of the data which is not loaded from the disk into the object prefetch window. Device. The method of claim 3, wherein the window control unit And removing the extended area of the object prefetch window when the client access request for the data loaded in the object prefetch window ends. The method of claim 3, wherein When the client access request frequency for the data corresponding to the already created object prefetch window is small, the size of the object prefetch window is reduced, and when the object prefetch windows that are already generated during the reduction process are adjacent to each other. And an window manager for merging into one object prefetch window. The method of claim 3, wherein And a window removing unit which removes the object prefetch window when the client does not access data corresponding to the object prefetch window that has already been created. The method of claim 8, wherein the window removing unit And when the memory space is insufficient for the data corresponding to the object prefetch window to be removed, remove the memory space from the memory area, and if the memory space is sufficient, the object input / output processing device. Generating an object prefetch window of a predetermined area in a memory; Expanding an area of the object prefetch window if the data size on the disk corresponding to the data access request of the client is larger than the object prefetch window; And Loading a portion of data on the disk corresponding to the size of the object prefetch window before expansion into the object prefetch window before expansion and providing the client with a portion of data loaded in the object prefetch window before expansion Loading the remaining portion of data on the disk in an extended area of an object prefetch window. The method of claim 10, wherein generating the window And generating a new object prefetch window when data corresponding to the access request of the client is not loaded on the already created object prefetch window. The method of claim 10, wherein expanding the window area An object input / output processing method characterized in that the size of the object prefetch window is extended by a predetermined unit until the entire data requested by the client is loaded in the remaining portion of the data which is not loaded in the object prefetch window from the disk. . The method of claim 10, Reducing the size of the object prefetch window with respect to an already created object prefetch window when the access request frequency of the client is small; Merging the two object prefetch windows into two object prefetch windows when the two object prefetch windows are adjacent by the reducing; And removing the object prefetch window for an object prefetch window that has not been made by the client. The method according to any one of claims 10 to 13, A computer-readable recording medium having recorded thereon a program for performing the object input / output processing method.

Images