KR101138937B1 - Context-aware prefetching system and method exploiting network buffer as prefetching buffer - Google Patents

Abstract

The present invention relates to a context-based information read-ahead system and method capable of improving transmission efficiency of communication while minimizing load caused by frequent network connection / disconnection in a communication environment between a user and a server. The context state information-based read method includes: acquiring a network connection context list by acquiring an existing network connection or creating a new network connection; transmitting a write request to the server through the obtained network connection; Collecting a predetermined amount of data of a file associated with the read request and transmitting the collected data to the server at once, and transmitting a read request to the server through the obtained network connection and obtaining data of a continuous area of the file associated with the read requestThe method comprising the steps of:

Network buffer, context status information, read ahead, client, server

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a context-aware prefetching system,

The present invention relates to a context-information-based read ahead system and method using a network buffer. More particularly, the present invention relates to a context-information-based read-ahead system and method for minimizing load caused by frequent network connection / And more specifically, when a read / write request and a data transfer are sequentially generated in a continuous area of one context between a user and a server, all network communication And a context-information-based read-ahead system and method capable of maximizing I / O performance by using an input / output buffer of the network connection as a prefetch buffer of data to be sent to a user and a server.

The present invention was derived from research carried out as part of the IT Growth Driving Technology Development Project of the Ministry of Knowledge Economy and the Korea IT Industry Promotion Agency. [Assignment Number: 2007-S-016-02, Title: Development of Low Cost Large-scale Global Internet Service Solution ].

Communication between users and servers is the most basic function of any Internet application. When the user sends a read / write request to the context managing server, the server basically transmits the context data to the user. At this time, allocating a new network connection for each user request and releasing the network connection immediately after the user request processing places a heavy burden on the user and the server. Also, the process of accessing the repository within the server every time the network is connected / disconnected, opening the corresponding context file, and closing the file again after data service also puts a lot of burden on the server. Therefore, communication between the user and the server should minimize access to the storage within the server while minimizing the network connection as much as possible.

A user request can be divided into a sequential request for a contiguous region of the context and a non-contiguous request for an arbitrary region of the context. Therefore, sequential and non-sequential requests are classified according to the nature of the request, sequential requests about the continuous area are collected and processed at once, and discontinuous requests are processed immediately to maximize the response speed of the service and the data transmission efficiency of the network have.

When the context data is retrieved from the repository on the server or the context data is stored in the repository, the data processing unit must be larger than a certain size to maximize the data processing efficiency. Therefore, the performance of the data service can be improved through a method of increasing a data processing unit by simultaneously processing all requests for data physically or logically continuous on the storage space.

U.S. Patent No. 7,216,203 B1, which was invented by Sanjeev Bagewadi and filed on September 23, 2003 by Sun Microsystems and registered May 8, 2007, is a method for maximizing the transmission efficiency of a network through read-ahead in a network-based file system . U.S. Patent No. 7,216,203B1 increases the speed of response to a user application by requesting a larger number of data from the server than data corresponding to a file offset that the user is currently requesting and storing the requested data on the buffer cache. When prefetched data is consumed and reduced to a certain amount or less, US Patent No. 7,216,203B1 continuously provides data service by additionally requesting and reading data as much as the user has consumed.

However, the user must send a request message to the server every time data is added to the request. Thus, U.S. Patent US 7,216,203 B1 reduces the transmission efficiency of the network by the time it takes to send a request message and receive a response message. Although U.S. Pat. No. 7,216,203B1 can process user requests at a time in units of buffer space, it increases the load on the server because each user request still needs to open and close the file.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a communication system and a communication method therefor, which can minimize the load caused by frequent network connection / Based read-ahead system and method.

Another object of the present invention is to provide a context-information-based read-ahead system capable of reducing time taken to respond to a request message and improving disk I / O efficiency on a server by collecting transmission data on a network buffer at the time of communication between a user and a server, Method.

It is still another object of the present invention to provide a method and system for providing a context-based information service capable of minimizing load on a server-side network connection management load and file information management load by servicing consecutive requests of a user for a context using an existing network connection Reading system and method.

According to another aspect of the present invention, there is provided a method of reading context-based information using a network buffer, the method comprising: retrieving a network connection context list to acquire an existing network connection or create a new network connection; Transmitting a write request to a server through the obtained network connection, collecting a predetermined amount of data of a file related to the write request, and transmitting the collected data to the server at once; And transmitting a read request to the server through the obtained network connection and obtaining data of a continuous area of the file related to the read request from the server.

According to an embodiment of the present invention, there is provided a method for reading a context-based information, comprising: generating a write thread or a read thread for each write / read request for a context in response to a write / read request from a client; The writing thread sequentially recording data received in the first network buffer in the file until the obtained network connection is completed; And reading the data of the stored file until the obtained network connection is completed and transmitting the read data to the second network buffer.

A client of a context-based information read-ahead system using a network buffer, the client comprising: a first network buffer for storing write data; A second network buffer for storing read data; And a repository for storing a list of network connection contexts, wherein the client transmits a write request to a server through an existing network connection included in the network connection context list, collects data of a file related to the write request to a predetermined amount And transmits the read request to the server through the existing network connection and obtains data of a continuous area of the file related to the read request from the server.

The context state information based read ahead system and method according to the present invention having the above configuration can improve the transmission efficiency of communication while minimizing load caused by frequent network connection / release in a communication environment between a user and a server.

The system and method for reading context-based information according to the present invention collects transmission data on a network buffer at a time in a communication between a user and a server, thereby reducing a time required for a response to a request message and improving disk I / .

The system and method for reading context-based information according to the present invention can minimize the burden of server-side network connection management load and file information management by servicing consecutive requests of a user for one context using existing network connection have.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a configuration diagram of a context state information based read ahead system using a network buffer according to an embodiment of the present invention.

In an environment assumed by the context state information-based read-ahead system according to the present invention, clients and servers are connected in a bidirectional network, buffers are set for each input direction and output direction, It sends a read or write request to an arbitrary offset to the server, and the server provides the appropriate service for the read / write request.

Referring to FIG. 1, the context state information-based preview system according to the present invention includes a client 110 having a file 111, a network input buffer 112, and a network connection context list 113, A client 120 having a network output buffer 122 and a network connection context list 123 and a disk 130 in which a file 131, a network input buffer 132, a network output buffer 133, (130) having a server (134). Here, the client 110 further comprises a repository for storing the network connection context list 113, and the client 120 further comprises a repository for storing the network connection context list 123. [

When the client 110 desires to store the updated data of the file 111 in the server 130, the client 110 checks the network connection context list 113, And judges whether or not there is a connection.

As a result of the determination, if there is a network connection already established, the client 110 transmits a write request to the server 130 storing the related file 131 through the already established network connection, And transmits the data to the server 130. The client 110 transmits the transmission data to the server 130 using the already established network connection and collects a predetermined amount of the transmission data to the network output buffer 112 and transmits the transmission data to the server 130 at one time. The continuous data is collected in the network input buffer 132 and written to the disk 134 at once.

When the client 120 reads the data of the file 131 stored in the server 130, the client 120 checks the network connection context list 123 to determine whether there is a network connection already established by the previous read / .

As a result of the determination, if there is a network connection already established, the client 120 sends a read request for the file 131 to the server 130 using the already established network connection. In response to the read request, the server 130 reads the data of the requested file 131 and fills it in the network output buffer 133, and the data that is filled in the network output buffer 133 is sent to the network input buffer 133 of the client 120 (122). At this time, the server 130 reads the next continuous data of the file 131 immediately after the network output buffer 133 is empty regardless of the additional read request of the client 120, and transmits the network output buffer 133 to the network output buffer 133). Thus, the client 120 can only obtain data in the contiguous area of the file 131 by reading the network input buffer 122.

In FIG. 1, the clients 110 and 120 are separately shown and are for convenience of description. That is, the clients 110 and 120 may be one and the same client.

FIG. 2 shows the structure of the network connection context between the client and the server in FIG.

2, the network connection context 200 has status information 210-1 to 220-N (N is a natural number of 2 or more) by the number of currently active network connections. Each of the status information 210-1 to 210-N includes a file descriptor 211 associated with the context, a most recent input / output offset 212 of the file associated with the context, A socket descriptor 213 for the client 110 and an input / output mode flag 214 for indicating whether the current operation of the client 110 or 120 is a file read or write, .

3 is an exemplary view showing a management state of a network connection between a client and a server in FIG.

Referring to FIG. 3, when a network connection for reading is established between the client 120 and the server 130, one connection related data 310 is stored in the connection management structure. In the connection related data 310, the network connection is operating in the read mode R 314 using the file descriptor value 3 311 and socket descriptor value 4 313, and the current offset is 0x00000100 in hexadecimal Respectively. At this time, when an additional 256 byte read request for the existing network connection and a write request for the new file are input at the same time, the network connection management structure is changed as shown in FIG. That is, the existing offset value 312A is changed to the new offset value 312B, the value is changed to 0x00000200 where 256 is added, and a structure 320 for a new write connection is created. At this time, the file descriptor 321 has the corresponding value 5, the offset 322 has the corresponding value 0x00000000, the socket descriptor 323 has the corresponding value 6, and the writing mode 324 has the W value.

The method by which the clients 110 and 120 perform an input / output operation for a given file descriptor includes the following two processes, for example, the search and generation process of the network connection data structure of FIG. 4, And performs the input / output operation process using the connection.

 4 is a flowchart illustrating a process of a client selecting a network connection corresponding to a descriptor in a context state information-based read system according to an exemplary embodiment of the present invention. Hereinafter, the client is the client 110 and / or the client 120 of FIG.

Referring to FIG. 4, the client searches for a network connection context 200 (S401) and determines whether there is a network connection corresponding to a given descriptor (S402).

If the network connection exists, the client determines whether the offset of the connection context of the previous request (write / read request) matches the offset of the connection context of the current request (write / read request) (S403). If it is determined that the offsets match, it is determined whether or not the read mode and the write mode of the previous request match the mode of at least one of the read mode and the write mode of the current request (S404). If it is determined that the mode matches at least one of the read mode and the write mode, the client obtains an existing network connection matching the current request (S405). Thus, the client can continue to use the existing network connection without further release and connection of the network in case of continuous operation.

If it is determined in step S403 that the offsets do not match, the client cancels the existing network connection (S406), creates a new network connection, and initializes the created network connection (S407). Then, the client obtains a newly created network connection (S408).

If it is determined in step S402 that there is no network connection, the client creates and initializes a new network connection (S407) and acquires a newly created network connection (S408) without releasing the existing network connection Proceed sequentially.

As a result of the determination in step S404, if the read mode and the write mode do not coincide with each other, the client performs a process of releasing an existing network connection (S406), creating and initializing a new network connection (S407) (S408). ≪ / RTI >

5 is a flowchart illustrating a process of a client writing data using a given network connection in a context-based information-based read-ahead system according to an embodiment of the present invention. Hereinafter, the client is the client 110 and / or the client 120 of FIG.

Referring to FIG. 5, the client writes a desired amount of data to a buffer of a given network connection (S501). The client determines whether or not the data write operation is normally completed (S502). If the write operation is normally completed, the client ends the data write operation.

If it is determined that the write operation is not normally completed, the client is disconnected from the network connection and the network connection context is modified (S503) because the network connection is broken due to the load and state of the server or the state of the network. The client informs the manager that it is no longer able to proceed with the write operation due to a network error (S504).

FIG. 6 is a flowchart illustrating a process of a client reading data using a given network connection in a context-based information read-ahead system according to an exemplary embodiment of the present invention. Hereinafter, the client is the client 110 and / or the client 120 of FIG.

Referring to FIG. 6, the client reads a desired amount of data from a buffer of a given network connection (S601). The client determines whether the data reading operation is normally completed (S602). If the reading operation is normally completed, the client obtains the read data (S603).

If it is determined that the data reading operation is not normally completed, the client determines whether the network connection is normally completed (S604). As a result of the determination, if the network connection is normally terminated, the client informs the administrator that the end of the corresponding context file has been reached (S605), and obtains the data read to the end of the file (S606).

As a result of the determination in step S604, if the network connection is not normally completed, the client can not use the network connection any more. Therefore, the client cancels the network connection and modifies the network connection context (S607) It is notified that the read operation can not be proceeded any more (S608). In this case, the client can perform data reading after performing the network connection acquisition operation again.

A method for the server 130 to perform an input / output operation for a given network connection performs the following two processes, for example, the input / output thread creation process of FIG. 7 and the input / output operation process of the corresponding thread of FIGS. 8 and 9 .

7 is a flowchart illustrating a process of generating an input / output thread in a server in a context state information based read ahead system according to an embodiment of the present invention. Hereinafter, the client is the client 110 and / or the client 120 of FIG.

As described below with reference to FIG. 7, the server 130 creates and processes a dedicated dedicated input / output thread for each consecutive request for one context for an input / output request from the client.

Referring to FIG. 7, the server 130 waits for an input / output request from the client (S701). At this time, the server 130 determines whether an input / output request has been received from the client (S702), and if there is no input / output request as a result of the determination, the server 130 proceeds to the waiting process of the input / output request (S701).

As a result of the determination, if there is an input / output request, the server 130 checks whether a sufficient amount of resources for generating a task execution thread is secured (S703). If the resource is insufficient as a result of the check, the server 130 informs the client that it is impossible to perform data input / output due to lack of resources (S704).

If it is determined that a necessary amount of resources is secured, the server 130 determines whether a data write request has been received from the client (S705).

If the data write request is received, the server 130 generates a data write thread according to the type of the request of the client (S706), and transmits the generated data write thread the information of the network connection To allow the data writing thread to perform data input / output (S707).

If the data write request is not received, the server 130 creates a data read thread according to the type of the request of the client (S708), and transmits the read data to the read data read thread through the network connection To allow the data reading thread to perform data input / output (S709).

FIG. 8 is a flowchart illustrating an operation procedure of a write thread in a context-based information read-ahead system according to an embodiment of the present invention. Hereinafter, the client is the client 110 and / or the client 120 of FIG.

The write thread sequentially writes data received in the network buffer to the file until the given network connection is disconnected. At this time, the recording start position is transmitted by the client at the time of requesting the first network connection creation. The detailed procedure is as follows.

Referring to FIG. 8, the write thread waits for data to be received from the client in a buffer of a given network connection (S801).

At this time, the writing thread determines whether new data has been received (S802). If the new thread is received as a result of the determination, the writing thread moves the received data to the writing buffer (S802). The reason for collecting data in the buffer is to collect the amount of data in the storage so as to maximize the efficiency of the write operation.

The writing thread determines whether the amount of the writing buffer exceeds a predetermined reference amount, that is, whether or not the writing buffer can be efficiently stored (S804). If the writing thread exceeds a predetermined reference amount, the writing thread writes The data of the buffer is recorded (S805), and the process goes to the process of waiting for data reception (S801). If it is determined that the amount of the write buffer does not exceed the predetermined reference amount, the writing thread proceeds to step S801 for waiting for data reception.

If it is determined in step S802 that no new data is received, the writing thread determines whether the network connection is completed (step S806). If the determination result in step S806 indicates that the network connection is not completed, The process proceeds to the waiting process (S801).

As a result of the determination in step S806, if the network connection is completed, that is, if a network status change message is received, the write thread writes data of the write buffer in the storage (S807) and ends the operation.

9 is a flowchart illustrating an operation procedure of a read thread in a context state information based read ahead system according to an embodiment of the present invention. Hereinafter, the client is the client 110 and / or the client 120 of FIG.

The read thread reads the data of the file stored in the storage and transfers it to the network buffer until the given network connection is disconnected. At this time, the start position of the read is transmitted by the user at the time of requesting the first network connection creation. The detailed procedure is as follows.

Referring to FIG. 9, the read thread sequentially reads a predetermined amount of data from a file (S901). At this time, the read thread breaks whether the read operation is successful (S902). If the read operation fails as a result of the determination, the read thread informs the client that an error has occurred in the storage (S903), and ends the operation. Here, there is a case where an error of the storage can occur, a case where the end of the file is reached, and data access is no longer possible due to an error in the storage.

As a result of the determination, if the read operation is successful, the read thread transmits the read data to the buffer of the network connection (S904).

At this time, the writing thread determines whether the network connection is completed during the transmission of the read data (S905), and if the network connection is completed, the writing thread ends the operation.

As a result of the determination in step S905, if the network connection is not completed, the write thread determines whether the buffer is full due to the data that has not yet been transmitted (S906) , The writing thread waits until there is space in the buffer (S907), and then the process goes to the process of reading data (S901). If it is determined that the buffer is not full, the process goes to the step of reading data (S901).

As described above, embodiments of the present invention collect transmission data on a network buffer at a time during communication between the clients and the server, thereby reducing the time taken to respond to a request message response and improving disk I / O efficiency on the server have. In addition, embodiments of the present invention can minimize the load on the server-side network connection management load and the file information management load by servicing consecutive requests of the user for one context using the existing network connection.

It is to be noted that the technical idea of the present invention has been specifically described in accordance with the above preferred embodiment, but the above-mentioned embodiments are intended to be illustrative and not restrictive. In addition, it will be understood by those of ordinary skill in the art that various embodiments are possible within the scope of the technical idea of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a configuration diagram of a context state information based read ahead system using a network buffer according to an embodiment of the present invention; FIG.

2 illustrates a structure of a network connection context between a client and a server in FIG. 1;

FIG. 3 is an exemplary view showing a management state of a network connection between a client and a server in FIG. 1. FIG.

FIG. 4 is a flowchart illustrating a process of a client selecting a network connection corresponding to a descriptor in a context-based information-based read system according to an exemplary embodiment of the present invention; FIG.

FIG. 5 is a flowchart illustrating a process in which a client writes data using a given network connection in a context-based information read-ahead system according to an embodiment of the present invention;

FIG. 6 is a flowchart illustrating a process of a client reading data using a given network connection in a context-based information read-ahead system according to an embodiment of the present invention; FIG.

7 is a flowchart illustrating a process of generating an input / output thread in a server in a context state information based read ahead system according to an embodiment of the present invention.

FIG. 8 is a flowchart illustrating an operation process of a write thread in a context-based information read-ahead system according to an embodiment of the present invention; FIG.

9 is a flowchart illustrating a process of a read thread in a context state information based read ahead system according to an embodiment of the present invention.

Claims (11)

Retrieving a network connection context list to acquire an existing network connection or create a new network connection; Transmitting a write request to a server through the obtained network connection, collecting a predetermined amount of data of a file related to the write request, and transmitting the collected data to the server at once; And Sending a read request to the server via the obtained network connection and obtaining data of a continuous area of the file associated with the read request from the server, Wherein the obtaining of the network connection comprises: Retrieving a network connection context and checking whether there is a network connection corresponding to the given descriptor; If it is determined that the existing network connection does not exist, creating and initializing a new network connection and acquiring the created network connection; Determining whether the offset of the connection context of the previous request matches the offset of the connection context of the current request if the existing network connection exists; If the offset does not match, releasing the existing network connection, creating and initializing a new network connection, and acquiring the created network connection; Determining whether a mode of at least one of a read mode and a write mode of the previous request matches a mode of at least one of a read mode and a write mode of the current request if the offsets match; Acquiring an existing network connection that matches the current request if at least one of the reading mode and the writing mode is matched; And If it is determined that the read mode and the write mode do not coincide with each other, the control unit releases the existing network connection, creates and initializes a new network connection, and acquires the created network connection Wherein the context information includes at least one of context information and context information. delete The method of claim 1, wherein the step of transmitting data of a file associated with the write request to the server includes a step of the client using the obtained network connection to write data, Wherein the writing of the data comprises: Writing a desired amount of data to a buffer of the obtained network connection; If the write operation of the data is not normally completed, notifying the administrator that the network connection is canceled, the network connection context is modified, and the write operation can no longer be proceeded due to a network error; And If the data write operation is normally completed, terminating the data writing Wherein the context information includes at least one of context information and context information. The method of claim 1, wherein the step of obtaining data of a continuous area of a file associated with the read request from the server includes a step of reading data using the obtained network connection, Wherein the reading of the data comprises: Reading a desired amount of data from the buffer of the obtained network connection; Acquiring read data if the data read operation is normally completed; Determining whether the acquired network connection is normally completed if the data reading operation is not normally completed; When the acquired network connection is normally completed, notifying the administrator that the end of the corresponding context file has been reached, and acquiring data read to the end of the file; And If it is determined that the acquired network connection is not completed normally, the network connection is released, the network connection context is modified, and then the management server notifies the manager that the network connection error can not be proceeded due to a network error Wherein the context information includes at least one of context information and context information. Generating a write thread or a read thread for each write / read request for a context in response to a write / read request from a client; The writing thread sequentially recording data received in the first network buffer in the file until the obtained network connection is completed; And Reading the data of the stored file until the obtained network connection is completed, and transmitting the read data to the second network buffer, Wherein the step of generating the write / Checking whether an amount of resources required to create a task thread of the task is secured when an input / output request is received from the client; If the resource is insufficient, informing the client that data input / output can not be performed due to the lack of resources; Confirming whether a data write request has been received from the client if a necessary amount of the resource is secured as a result of the checking; Generating a data writing thread according to a type of the request of the client when the data writing request is received, and transmitting information on the received network connection to the generated data writing thread; And If it is determined that the data write request is not received, generating a data read thread according to the type of the request of the client, and transmitting the received network connection information to the generated data read thread, Wherein the context information includes at least one of context information and context information. delete 6. The method of claim 5, wherein the step of writing the data to a corresponding file comprises: Determining whether data is received from the client in a buffer of a given network connection; When the data is received, moving the received data to the first network buffer; Writing data in a write buffer to a storage if the amount of the first network buffer exceeds a predetermined reference amount; And If the data is not received as a result of the determination, writing the data of the first network buffer to the storage when the network connection is completed Wherein the context information includes at least one of context information and context information. 6. The method of claim 5, wherein reading the data of the file and transmitting the data to the second network buffer comprises: Sequentially reading a predetermined amount of data from the file; Determining whether the read operation is successful; If the read operation fails, notifying the client that an error has occurred in the storage; Transmitting the read data to the second network buffer if the read operation is successful; When the network connection is completed, ending the operation of the writing thread; And Waiting until a storage space is created in the second network buffer if the storage space of the second network buffer is insufficient in the state where the network connection is not completed Wherein the context information includes at least one of context information and context information. A system for reading a context based information using a network buffer, A client including a first network buffer for storing write data, a second network buffer for storing read data, and a repository for storing a network connection context list; And And a server connected to the network according to the network connection context list, The client transmits a write request to the server through an existing network connection included in the network connection context list, collects a predetermined amount of data of a file related to the write request, and transmits the collected data to the server at once, A read request is transmitted to the server and data of a continuous area of the file associated with the read request is obtained from the server, The client collects a certain amount of data in the first network buffer and transmits the data at a time, The server reads the data of the read requested file to fill the third network buffer, the data filled in the third network buffer fills the second network buffer of the client, and the server notifies the client of the read request Characterized in that data of a continuous area of the file is obtained by reading the next consecutive data of the file immediately after the third network buffer is emptied to fill the third network buffer and the client reads the second network buffer A client of a context-aware information-based read-ahead system. delete 10. The method according to claim 9, wherein the network connection context list includes status information of the number of currently active network connections, The status information includes at least one of a file descriptor associated with the context, a latest input / output offset of the file associated with the context, a socket descriptor for network connection identification, and an input / output mode flag indicating whether the current operation of the client is reading or writing the file. Wherein the context-information-based client reads the context-based information.

Images