KR20150048861A - Network service system and method with off-heap caching - Google Patents

Abstract

A method for providing data on a network using an application server with off-heap caching includes receiving a request for requested data from an application server connected to the network, Using a key index stored in the application server to find a location stored in an off-heap memory, and recovering the requested data from an off-heap memory of the application server, And resolving the request.

Description

[0001] NETWORK SERVICE SYSTEM AND METHOD WITH OFF-HEAP CACHING [0002]

The present invention generally relates to a computerized system for managing physical memory in a real-time environment.

Electronic commerce, often known as "e-commerce", involves the purchase and sale of products or services through electronic systems such as the Internet. The amount of electronic trade performed has grown tremendously with the widespread adoption of Internet technologies. One particularly explosive area of growth of e-commerce is the field of advertising in the field of advertising and especially on the Internet.

Many network service systems, including the transmission of images over the Internet and supporting the Internet functionality ("Web network service system"), are available from vendors such as the Java software platform (TM) available from Oracle Corporation It is implemented as an object-oriented or pseudo object-oriented software platform. One of the characteristics of Java is portability, which means that computer programs written in the Java language work similarly on any hardware / run-system platform. This is accomplished by accumulating the Java language code in an intermediate representation called Java bytecode, rather than being accumulated directly in platform-specific machine code. Java byte-code instructions are similar to machine code, but are intended to be interpreted by a virtual machine, especially written for host hardware. Standardized libraries provide a common way to access host-specific features such as graphics, threading, and networking.

One of the most important performance problems for web service applications written in Java is garbage data collected in a portion of the virtual memory space known as the "heap." When the heap is filled with obsolete objects, To avoid this, a process known as "garbage collection" can be used to remove obsolete objects. The performance of garbage collection is related to "overhead" Java objects that reside in the heap for a very short amount of time contribute slightly to the overhead during garbage collection. Objects that are in the heap long enough to be moved to the "old generation space" And has a high overhead during garbage collection.

Java uses an automatic garbage collector to manage memory in the object lifecycle. The programmer decides when an object is created, and once the object is no longer usable, the Java runtime is responsible for memory recovery. Once an unreachable memory is associated with an object that has been left untouched, it is automatically freed by the garbage collector. If the programmer code is still associated with an object that is no longer needed, then something similar to a memory leak can occur if objects that are no longer needed are typically stored in a container that is still in use. When a method of an object that does not exist is called, a "null pointer exception" is discarded.

Java includes various types of garbage collectors. Common garbage collections are optionally known as "Concurrent Mask Sweep Collectors" and "CMS Garbage Collectors". However, there are many other garbage collectors that can be used to manage Java heap memory.

One of the ideas behind Java's automatic memory management model is that programmers can avoid the burden of having to perform passive memory management. Unfortunately, with Java, garbage collection can happen at any time. Ideally, this will happen when the program is not running. However, if there is insufficient free memory to place a new object in the heap, garbage collection will be triggered and the program can be temporarily stopped. Clear memory management is not possible in Java.

One of the most important performance issues for web services applications written in Java, such as video delivery systems, is the garbage collection of data on the heap. Java objects that exist for very short periods of time have little overhead during garbage collection. However, objects that are long enough to be moved into the old generation space have high overhead during garbage collection. Since garbage collection is automatic, critical application programs can be held and prevented from running application programs on time.

For example, in an ad system that transmits video ads based on the Internet, the ad (" ad ") performance system needs a decision window within 10 to 20 milliseconds and all the data needed to make a decision Must be cached in the virtual memory of the application process. When the garbage collection is started in heap memory, the application program is suspended and the decision window is lost.

These and other limitations of the prior art will become apparent to those skilled in the art by reading the following description and studying various features of the drawings.

In an illustrative and non-limiting embodiment, a network service system with off-heap caching includes a plurality of application servers, a plurality of cache servers, and a local area network router network routers. Each of the plurality of application servers operates an application server process (ASP) having an ASP virtual memory space with ASP memory and ASP off-heap memory. Each of the plurality of cache servers includes a cache server process (CSP) having a cache configuration process (CCP) for communicating the CSP index and the CSP process, a CCP (CCP) process having the CCP heap memory and the CCP off- Activates a CCP process with a virtual memory space, which provides access to the CCP buffer copy and the CSP exponent stored in the CCP off-heap memory.

A local area network (LAN) router combines a plurality of application servers into a plurality of cache servers, an ASP process stores a copy of the CCP buffer in ASP off-heap memory, and a key index ) Is used.

In an embodiment, which is disclosed by way of example and not limitation, a method of providing data using an application server with off-heap caching on the Internet includes:

Receiving a request for the requested data from an application server connected to the network; Using an important index stored in the application server to find out where the requested data is located in the off-heap memory of the application server; Recovering the requested data from the off-heap memory of the application server; And resolving the request.

In an embodiment disclosed in a non-limiting and illustrative manner, a method of providing off-heap caching to a network service system includes:

(a) Providing a plurality of application servers each running an application server process (ASP) having an ASP virtual memory space with ASP heap memory and ASP off-heap memory ; (b) a cache server process (CSP) having a CCP configuration and communicating with a CSP index and a CSP process, respectively, the CCP process comprising a CCP heap memory and a CCP off- off-heap memory, wherein the CCP process provides access to CCP buffers data and CSP exponents stored in the CCP off-heap memory, Providing a plurality of cache servers; (c) combining the plurality of application servers and the plurality of cache servers so that the ASP process stores a copy of the data in the CCP buffers of the ASP off-heap memory, and accessing a copy of the data from the CCP buffers using a key index. < Desc / Clms Page number 6 >

An advantage of certain embodiments is that the need for garbage collection due to off-heap storage is reduced, which results from the use of off-heap storage, which reduces system overhead and ASP service interruption. These and other advantages will become apparent to those skilled in the art from the following description and study of the various drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Now, with reference to the drawings in which like components have like reference numerals, various illustrative embodiments are described. The illustrative embodiments are illustrative of the concepts disclosed herein and are not intended to be limiting. The drawings include the figures below.

1 is an exemplary block diagram of a network service system with off-heap caching.
2 is an exemplary block diagram illustrating a hardware configuration for a server, a computer, and a microprocessor control apparatus forming part of the exemplary network service system of FIG.
Figure 3 shows an exemplary specific interaction between an application server (AS) and a cache server (CS) of the network service system of Figure 1.
4 is an exemplary flow chart of a Cache Configuration Process (CCP) installation method.
FIG. 4A helps illustrate the exemplary cache configuration process (CCP) installation method of FIG.
5 is a flow diagram of an exemplary update process of a cache configuration process (CCP).
6 is a flow diagram of an exemplary process for initiating an application server.
7 is a flowchart of an exemplary update process of an application server process (ASP).
8 is a flow diagram of an exemplary application server process (ASP).

1 is a block diagram of an exemplary network service system 10 with off-heap caching. An exemplary network service system 10 (sometimes referred to as a " data center ") comprises a plurality of application servers 12 and a LAN router 18 to configure a local area network And a plurality of cache servers 14 that are connected to each other. Exemplary network service system 10 also includes a database server 20 coupled to a plurality of cache servers 14 and a remote network (WAN) router 22 coupled to a plurality of application servers 12.

In a non-limiting example, a plurality of application servers 12 may include N application servers AS1, AS2, AS3, AS4 ... ASN. In a non-limiting example, a plurality of cache servers 14 may include M cache servers CS1, CS2, ..., CSM. As will be appreciated by those skilled in the art, servers such as application server 12 and cache server 14 are typically microprocessor-based computer systems intended for network use.

The LAN router 18 preferably includes a load balancer to enhance the enhancement of the LAN 16. For example, the LAN router 18 connects a plurality of application servers 12 to a plurality of cache servers 18 so that a specific application server is associated with a specific cache server. For example, if the application server AS2 has not yet been associated with one of the plurality of cache servers 14, the load balancer of the router 18 may have a plurality of cache servers 14 ). For example, AS2 is assigned to the cache server CS1 to support its cache server. Once the relationship is established, (AS2) and (CS1) communicate directly with the router (18).

The wide area network (WAN) router 22 also preferably includes a load balancer (LB) to enhance the efficiency of communication between the network service system 10 and the remote network 24. For example, the distance network router 22 is connected to the remote network 24 by a connection 26 and to an Internet Service Provider (ISP) of the remote network 24 and to each of the plurality of application servers 12.

When the data request is received from the new requestor via the connection 26, the load balancer LB of the router 22 allocates one of the plurality of application servers 12 and, based on load balancing criteria well known to those skilled in the art Process the request. For example , a new request is assigned to the application server AS4 (e.g., a computer or mobile device that communicates over the Internet). Once the relationship is formed, the new requestor, via the router 22, communicates directly with the application AS4.

2 is a simplified block diagram of a computer and / or server 28 suitable for use in the system 10. By way of non-limiting example, the computer 28 includes a microprocessor 30 coupled to a memory bus 32 and an input / output (I / O) bus 34. Many memory and / or high speed devices may be coupled to memory bus 32, such as RAM 36, SRAM 38, and VRAM 40. The various input / output devices attached to the input / output bus 34 are mass storage 42, network interface 44 and other input / output devices 46. As will be appreciated by those skilled in the art, there are many non-transient computers that can read the medium and are available to the microprocessor 30, such as RAM 36, SRAM 38, VRAM 40 and mass storage 42. The network interface 44 and other input / output devices 46 also include a computer readable medium, such as a register, cache, buffer, or the like. The mass storage 42 may be of a variety of types including a hard disk drive, an optical drive, and a flash drive, for example, two or three.

Figure 3 illustrates an example of specific transactions between an application server (AS) 12 'of a plurality of application servers 12 and a cache server (CS) 14' of a plurality of cache servers 14. As will be appreciated by those skilled in the art, the application server 12 ' can operate many concurrent processes and threads. By way of non-limiting example, application server process ASP 48 as well as processes P1, ... The application server 12 'operating the PN is shown. The amount of virtual memory space allocated is associated with each of the processes. For example, the ASP 48 is associated with a virtual memory space 50 that includes a heap memory 52 and an off-heap memory 54.

As will be appreciated by those skilled in the art, virtual memory space is typically a contiguous range of virtual memory addresses located in physical memory that may or may not be contiguous. For example, as soon as Java initializes And automatically allocates a range of the heap memory address (which is likely to be automatically garbage collected) to a process such as the ASP 48. Some of the virtual memory space that is not in the heap ("off-heap memory") is not automatically garbage-collected by Java processes.

An example of the present invention was performed using a Java software platform that generates heap memory that is susceptible to garbage collection automatically. Other examples can be performed on other Java software platforms that are susceptible to automatic garbage collection. Thus, in another alternative example, the terms " heep memory ", " memory that is likely to be automatically garbage collected ", and " off-heep memory "Quot; memory ".

As will be appreciated by those skilled in the art, the cache server 14 may operate a number of concurrent processes and threads (THREAD). By way of non-limiting example, a cache server process (CSP) 56 having a virtual memory space 58 and a virtual memory space 62 having a heap memory 64 and an off-heap memory 66 A cache server 14 'running a cache configuration process (CCP) is shown. The CSP 56 and the CCP 60 communicate with each other as indicated by (67). The CSP 56 may be performed by a standard software cache server, such as the EHCache Cashe Server, available from Seracotain Corporation of San Francisco, CA.

In the present example, the application server 12 'and the cache server 14 communicate using the HTTP protocol. As will be appreciated by those skilled in the art, HTTP, which means " hypertext transfer protocol ", is a collaborative hypermedia information system that is assigned. For example, HTTP is the foundation for data communication for the World Wide Web (WWW) on the Internet. Using the HTTP protocol, the ASP 48 may send a request 68 to the CSP 56 which resolves the request and provides a response 70 to the ASP 48. As will be appreciated by those skilled in the art, the request is sent by an HTTP path. An independent HTTP path provides access to various data structures, lists, and blocks of data. The HTTP path to the image cache CSP index through the load balancer of the router 18, for example, may look like this.

http: // loadbaladd / playlisticcache / videocache / index

As used herein, a " buffer " is a fragment of virtual memory that is receptive to a block of data, or " data blocks. &Quot; Data blocks are typically associated with specific data types such as image data, audio data, and so on. A buffer can be of insufficient size to accommodate a particular block of data and in this case various buffers may be used to store the block of data. A buffer may contain one or more data blocks, and may even include other types of blocks. By way of non-limiting example, various types of data may be stored in blocks for a particular data organization, such as a key index. Also, by way of non-limiting example, the data block may be the same size as the data buffer, although various data blocks and data buffers may be required to store data for a particular data configuration.

4 is an exemplary flow chart of the Cache Configuration Process (CCP) 4A is intended to assist in the description of the method 72 of FIG. For example, the method 72 begins at 74 and is executed at 76 and the N buffer 77 is formed in a memory space where garbage is not collected (e.g., " off-heap "). Next, at run 78, a buffer list 79 is generated in which garbage is collected (e.g., " on heap ") with a pointer to an N buffer 77. Run 80, A free pool buffer list 81 is created with a pointer pointing to the N buffer 77. This should be all at the start. An L data list is generated with a pointer to the allocated buffer of the N buffer 77. An execution 86 is performed on the cache 83. The execution of the cache 86 A data block is created in the server process CSP and an execution 88 creates a data block index in the CSP. Next, at execution 90, a message is sent to the application server (e.g., using the JMS protocol) And informs the application server that the off- The JMS protocol provides a Java Message Service (JMS) service for the transmission of messages between two or more customers (e.g., a computer, a server, etc.), as will be appreciated by those skilled in the art. ). &Quot; The method 72 terminates at (92).

5 is an exemplary flow diagram of a process 94 for updating the cache configuration process (CSP). Process 94 begins at 96 and at run 98 it is determined whether to update the data in the data block of the buffer stored in the off-heap memory. If there is an update, execute (100) creates a new data configuration list. Optionally At run 102, it is determined if the number of buffers in the free pool falls below the threshold " t ". For example, if the number of buffers N = 1000 and the threshold t = 200, a message is sent to the operator indicating that the number of pre-buffers will gradually decrease if the number of buffers in the free pool is less than 200. Next, at run 104, a buffer from the free pool is requested and a pointer to the buffer is put into the new list. At run 106, the buffer is populated with a new data block and the old buffers are deallocated to free buffer at run 108 (along with outdated dates). The obsolete key list is discarded at run 110 and the index file is read from the CSP at run 112. Next, at run 114, the data block is read by the CSP and the index file is updated at run 116. Finally, at run 118, a message is sent (using JMS, for example) to the particular cache type to update the particular cache type and provide the address of the CSP and CCP. The process 94 ends at 120. [

It will be appreciated that the process 94 is executed by creating a new set of data blocks and a new CSP index before destroying the obsolete data blocks and the obsolete indexes. For example, if the outdated data is in buffer set " A ", a new buffer set " B " is provided with the new data block and a new CSP index is made in the data block of buffer " B ". In the example of the present invention, the creation of the buffer set " B " and the outdated CSP with the new CSP index are automatically switched to new data. Once completed, the A buffer is returned to the free buffer pool and the CSP index is allowed to be garbage collected.

6 is a flow diagram of an exemplary process 122 for starting the application server 12. Preferably, the application server 12 is started only after the cache server 14 is started. Process 122 begins at 124 and at run 126 an N buffer is generated in a non-garbage collected memory (e.g., heap memory). A buffer list is then created with a pointer to the N buffer at run 128 and a free pool buffer list is created at run 130 with a pointer to free buffers This should be all at startup). The counter " i " is then started at run 132, and the counter i is incremented by one at run 134. The decision execution 136 determines whether the counter i is less than or equal to " L " and the number of data types. If so, execution 138 requests the index and stores the data block in the heap memory of the ASP from the CSP for a data structure (i) as well as a list generation for the data structure (i). For example, one type of data organization is image data organization. Next, at execution 140, an N block is allocated to the N data types read by execution 138 and the data is copied from the heap and written to the list (i) for the data structure (i). Finally, the heap index completely reads the heap memory for each of the data structures in execution (142). The loop including executions 138, 140 and 142 is repeated L times and execution 122 then ends at 144. [

FIG. 7 is a flow diagram of an exemplary process 146 for updating the application server (ASP). The process 146 begins at 148 to determine whether an update has been received from the CSP at decision execution 150. [ If so, run 152 constitutes a data path from the ASP to the CSP index, and the CSP index is read at run 154. Next, at execution 156, a new listener for the data configuration is created and at 158 a new block of data is stored from the CSP in the heap memory of the ASP. Execution 160 allocates the same number of buffers from the free buffer pool and execution 162 copies the data block into the buffer and writes a new data list. Next, at run 164, the on-heap block is released for garbage collection and at 166 the key index is read into the heap memory. An obsolete buffer is returned to the free pool of execution 168 and the obsolete list is disassociated and prone to garbage collection at run 170. The process 146 ends at 172. < RTI ID = 0.0 >

8 is an exemplary flow diagram of a process 174 for the application server (ASP). Process 174 begins at 176 and a decision 178 determines whether a request for data has been received. If so, run 180 uses the key index to find out where the data was stored in the off-heap buffer (e.g., the key provided by the request). Next, at execution 182, data is recovered from the off-heap buffer and the request is resolved. In the case of a TCP / IP request, the response ends with the requested data.

As stated herein, the key index is in heap memory, which is susceptible to garbage collection. However, if the data block is also in the heap memory, there is less overhead associated with this configuration. In another embodiment, the key index is in the off-heap memory. In such an embodiment, the HTTP request of the key index present in the heap memory is retransmitted to the off-heap process (e.g., one performed in the C programming language) and implements the key index in the off-heap memory.

Although various embodiments have been described using terminology and apparatus, such description is for illustrative purposes only and is not intended to be limited to the terms used and described. Without departing from the spirit and scope of any embodiment described herein,

It will be understood by those skilled in the art that changes and modifications may be made. It is also to be understood that aspects of the various other embodiments may be interchanged in whole or in part. Therefore, the claims hereafter and in the following are intended to be interpreted without limitation and without limitation, in accordance with their true technical thought and scope.

Claims (20)

An ASP heap memory having an ASP heap memory and an ASP virtual memory space having an ASP off-heap memory, an ASP heap memory, And an ASP off-heap memory (ASP off-heap memory) having a key index stored in the ASP off-heap memory. In order to determine the location where data is stored in the ASP off-heap memory, A plurality of application servers using a key index;
(CCP) process, each of which has a CSP index and a CCP configuration process (CCP) while communicating with a CCP process, the CCP process comprising a CCP heap memory and a CCP off-heap memory wherein the CSP process comprises: a plurality of cache servers for providing access to the data of CCP buffers stored in the CCP off-heap memory and the CSP exponent;
Combining the plurality of application servers and the plurality of cache servers so that the ASP process stores a copy of the data in the CCP buffers of the ASP off-heap memory, (CCP) off-heap caching that includes a local area network (LAN) router that accesses a copy of data from the CCP buffers using a local area network (LAN) router. The method according to claim 1,
Wherein the LAN router includes a load balancer that provides an initial introduction between an application server of one of the plurality of application servers and one of the plurality of cache servers, Network service system with off-heap caching. 3. The method of claim 2,
And a database server coupled to the plurality of cache servers and providing data to be stored in the CCP off-heap memory. ≪ Desc / Clms Page number 13 > The method of claim 3,
And a remote network (WAN) router coupling the plurality of application servers to a wide area network (WAN). 5. The method of claim 4,
The WAN router includes off-heap caching that includes a load balancer that provides an initial introduction to one of the plurality of application servers. A network service system. 6. The method of claim 5,
Wherein the wide area network (WAN) has off-heap caching implemented in the HTTP protocol. The method according to claim 6,
Wherein the long-distance network (WAN) is an Internet, with off-heap caching. 8. The method of claim 7,
Wherein the local area network (LAN) has off-heap caching performed with the HTTP protocol. 9. The method of claim 8,
And off-heap caching wherein the ASP and the CSP communicate requests and responses. 10. The method of claim 9,
Wherein the key index has off-heap caching stored in the ASP heap memory. A method for providing data on a network using an application server having off-heap caching,
Providing one or more cache servers for communicating with a CSP process and running a CSP server having a CSP exponent and a cache configuration process (CCP);
Combining at least one cache server with an application server to access data in CCP buffers stored in the CCP off-heap memory and the CSP exponent;
Storing a copy of the data in CCP buffers in an off-heap memory of the application server;
Receiving a request for the requested data from the application server connected to the network;
Using a key index stored in the application server to find out where the requested data is stored in the off-heap memory of the application server;
Recovering the requested data from the off-heap memory of the application;
And resolving the request;
The CCP process has a CCP virtual memory space having a CCP heap memory and a CCP off-heap memory
The application server may store the application's heap memory or off-heap memory in order to find out where the data is stored in the off-heap memory of the application server. Heap caching with a key index stored in the application server. 12. The method of claim 11,
The network is the Internet, and the request includes a TCP / IP protocol request, and the resolution of the request is performed using an application server with off-heap caching including a TCP / IP protocol response, Lt; / RTI > 13. The method of claim 12,
The application server uses an application server with off-heap caching including an application server process (ASP) having an ASP virtual memory space with a heap memory that is likely to be garbage collected automatically Thereby providing data on the network. 14. The method of claim 13,
Wherein the key index is provided by an application server having off-heap caching stored in the heap memory. 14. The method of claim 13,
Wherein the key index is provided by an application server having off-heap caching stored in the off-heap memory. A method of providing a network system with off-heap caching,
The ASP heap memory or the ASP off-heap memory has an ASP virtual memory space having an ASP heap memory and an ASP off-heap memory. providing a plurality of application servers each driving the application server having a key index stored in -heap memory;
Providing a plurality of cache servers each driving a cache server process (CSP) having a CSP index and a cache configuration process (CCP) communicating with the CSP process;
Providing a local area network router coupling the plurality of application servers and the plurality of cache servers
To determine the location where data is stored in the ASP off-heap memory, the ASP process uses the key index;
Wherein the CCP process has a CCP virtual memory space having a CCP heap memory and a CCP off-heap memory, the CSP process having a CCP off-heap memory and a CCP off- Provide access to the data of the stored CCP buffer;
The ASP process stores a copy of the data of the CCP buffers in the ASP off-heap memory and uses the key index to retrieve a copy of the data from the CCP buffers Lt; RTI ID = 0.0 > off-heap caching < / RTI > 17. The method of claim 16,
Providing a database server coupled to the plurality of cache servers, and deblocking the CCP buffer using the database server, the method comprising: providing a network service with off-heap caching / RTI > 18. The method of claim 17,
And sending an update message to the plurality of application servers when there is a change in the CCP buffer. ≪ Desc / Clms Page number 21 > 19. The method of claim 18,
Wherein the ASP updates off-heap memory to reflect the change in the CCP buffer. ≪ Desc / Clms Page number 19 > 20. The method of claim 19,
And wherein the key index is updated to reflect the change in the CCP buffer. ≪ Desc / Clms Page number 19 >

Images