JP2014174966A - Character string data processing method, program and system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve memory use efficiency by allowing guest VMs to share character string objects in a heap memory.SOLUTION: In order to allow objects in the same memory image to be utilized among a plurality of JVMs, a system of the invention performs the following: extracting character string objects commonly used by a plurality of guest VMs to summarize them into a file; mapping the file to a predefined address in a memory upon starting execution; retrieving the character string objects in the mapped file when a character string object is to be generated in execution; and, if the same character string object is found, making it usable. Shared character string data sets are extracted and summarized for each application and component. In order to prepare such shared character string data sets, the system of the invention executes one or more Java(R) programs on a plurality of JVMs on a plurality of guest VMs.

Description

  The present invention relates to a technique for processing character string data in an environment in which a plurality of guest environments (operating system and Java (R) VM) operate on one computer system.

  Conventionally, it has been an important issue to increase the use efficiency of physical memory in an environment where a plurality of guest environments operate as a single computer system (also called a machine). At that time, in an environment where almost the same software group operates in a plurality of guest VMs, each guest VM generates the same character string data, which is wasteful.

  As one conventional technique for solving this problem, there is known a method of extending the class sharing function of JVM to share JVM class data and the like among a plurality of guest VMs. For more information about this, see http://www.ibm.com/developerworks/jp/java/library/j-shared/.

  Also known is a method of creating a cache file of a character string object generated when starting up a JVM and sharing it with each guest VM. However, at this time, only the cache file is shared, and the object itself must be recreated in each JVM, and cannot be shared.

  Japanese Patent Laid-Open No. 2002-229793 discloses a plurality of groups consisting of a variable name group and a method name group for each argument type when assigning characters or character strings as variable names and method names in a Java (R) program. Disclosed is that a unique character or character string is assigned to each individual object within one group, and a common character or character string is appropriately assigned to each individual object among a plurality of groups. However, this prior art does not describe a technique for searching for an assigned character string.

  In US Patent Publication No. 2012/0017204, a character string object generated at the time of starting up a JVM is put in a cache file, and a character string object is generated from the cache file at the next time of starting up. It is disclosed that the number of character string objects generated is reduced. In this technique, only character strings that are duplicated before the JVM is started are targeted. Also, after loading the cache file, the string object must be re-stored in the intern table.

  U.S. Pat. No. 7,707,583 discloses a technique for sharing objects in a runtime system and isolating user sessions in a scalable manager. In this technique, a user context corresponding to a user session is stored in a shared memory area. When a request corresponding to the user session is received, one process is selected from the set of operating system processes, and one runtime system is selected from the set of runtime systems. In this technique, the amount of physical memory used is reduced by sharing classes and objects among a plurality of virtual machines. In addition, it is possible to classify objects that can be shared in class units and to determine what types of objects are shared by the user. However, this prior art does not describe a technique for searching for a shared object at high speed.

  U.S. Patent Publication No. 2004/0049493 discloses a string search technique for routing based on an ASCII string in a bucket payload. In this method, the registered character string is registered in one hash table. At the time of searching, before searching for a search character string from the hash table, a search for a partial character string in the array is performed, and a character string search that may not be registered is immediately terminated. Further, two arrays are prepared and are hierarchically configured with a hash table. Although this technique indicates that the search for a character string that does not apply is terminated at an early stage of the search, it does not suggest a contrivance for storing the character string for speeding up the search.

  US Pat. No. 7,418,505 discloses a technique for performing routing at high speed by dividing a hash table for each prefix length of an IP address and reducing collision of values in the hash table. However, this technique can only make limited provisions to avoid collisions of values in the hash table as much as possible.

  Kiyokuni Kawachiya, Kazunori Ogata, Tamiya Onodera. "Analysis and Reduction of Memory Inefficiencies in Java Strings,", In Proceedings of the 23rd Annual ACM Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA '08), pp. 385-401 (Oct. 2008). Discloses that in the Java (R) heap, memory usage efficiency is improved by focusing on duplicate character strings and unused literals.

JP 2002-229793 A US Patent Publication No. 2012/0017204 US Pat. No. 7,707,583 US Patent Publication No. 2004/0049493 US Pat. No. 7,418,505

http://www.ibm.com/developerworks/jp/java/library/j-shared/ Kiyokuni Kawachiya, Kazunori Ogata, Tamiya Onodera. "Analysis and Reduction of Memory Inefficiencies in Java Strings,", In Proceedings of the 23rd Annual ACM Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA '08), pp. 385-401 (Oct. 2008).

  An object of the present invention is to improve memory use efficiency by enabling a character string object in a heap memory to be shared between guest VMs.

  Another object of the present invention is to provide a technique capable of retrieving a character string object in a shared heap memory at high speed.

  The present invention solves the above-mentioned problem by storing character string objects that can be shared among a plurality of guest VMs in a format that can be directly referred to, and enabling efficient search during execution.

  The present invention is not limited to this, but a preferred implementation is an implementation based on Java (R).

  In the case of implementation using Java (R), the system according to the present invention extracts character string objects that are commonly used by a plurality of guest VMs and collects them into a file in order to use the same memory image object among a plurality of JVMs. When the execution is started, the file is mapped to a predetermined address in the memory, and when trying to generate a string object at the time of execution, the string object in the mapped file is searched. Make it available.

  At that time, a character string data set shared for each application or component is extracted and collected. In order to prepare such a shared character string data set, the system according to the present invention executes one or a plurality of Java (R) programs on a plurality of JVMs on a plurality of guest VMs. The Java (R) program extracts the character string data existing in the Java (R) heap of each JVM, and sets the character string data appearing in common from JVMs on different machines as the sharing target character string data. . At this time, the character string data to be shared can be directly referenced as a Java (R) object simply by mapping to the address space of the JVM process.

  In one aspect of the present invention, in order to enable high-speed search of character string objects, grouping is performed based on characteristics of target data, and a data structure that provides the fastest search for target objects is used for each group. To. The characteristics of the target data here are the length of the character string, the class file in which the character string object is generated, the jar file, and the like. A hash table is created for each of the grouped character strings, and the character strings are later searched using the hash table.

  According to the present invention, by sharing the character string object in the heap memory among the guest VMs, an effect of improving the memory usage efficiency can be obtained.

  In addition, character string objects are preferably grouped based on the characteristics of the target data, and the data structure that makes the search for the target object the fastest is used for each group, thereby improving the efficiency of character string search. To do.

It is a block diagram of an example of the hardware constitutions for carrying out the present invention. It is a figure which shows a multiple virtual machine environment. It is a figure which shows the state by which several JVM is started in one guest VM. It is a figure which shows the flowchart of the process which collects a common object character string from several guest VM. It is a figure which shows the flowchart of the process which determines the hash function for searching a character string from a common object character string, and bit shift amount. It is a figure which shows typically the process which determines a hash function and a bit shift amount in the character string divided into groups. It is a figure which shows the flowchart of the process which calculates | requires the index used with a hash function from the character string data in a group. It is a figure which shows the flowchart of the process which determines the hash function with the fewest hash collisions. It is a figure which shows the flowchart of the process which calculates | requires the shift calculation with the least collision of the hash table index value. It is a figure which shows the relationship between a hash table index and a bit shift. It is a figure which shows the structure for storing the shared character string data in DLL. It is a figure which shows the state which stored the shared character string data in DLL concretely. It is a figure which shows the memory map in the state in which the some JVM is started. It is a figure which shows the flowchart of a String constructor call process. It is a figure which shows the flowchart of a String.intern () call process.

  Embodiments of the present invention will be described below with reference to the drawings. It should be understood that these examples are for the purpose of illustrating preferred embodiments of the invention and are not intended to limit the scope of the invention to what is shown here. Further, throughout the following drawings, the same reference numerals denote the same objects unless otherwise specified.

  Referring to FIG. 1, there is shown a block diagram of computer hardware for realizing a system configuration and processing according to an embodiment of the present invention, generally designated by reference numeral 100. In FIG. 1, a CPU 104, a main memory (RAM) 106, a hard disk drive (HDD) 108, a keyboard 110, a mouse 112, and a display 114 are connected to the system bus 102. The CPU 104 is preferably based on a 32-bit or 64-bit architecture, such as Intel Core (TM) i3, Core (TM) i5, Core (TM) i7, Xeon (R), AMD Athlon ™, Phenom ™, Sempron ™, etc. can be used. The main memory 106 preferably has a capacity of 8 GB or more, more preferably a capacity of 16 GB or more.

  As shown in FIG. 2, a hypervisor 202 for implementing a plurality of virtual machines (VMs) is installed in the hard disk drive 108. Examples of programs that can be used as the hypervisor 202 include, but are not limited to, VMWare® and Xen. Here, explanation will be made assuming that Xen is used.

  On the hypervisor 202, a host VM 204 and a plurality of guest VMs 206a, 206,. The host VM 204 is also called domain 0 in Xen and includes a device driver that interfaces with the hardware 100 via the hypervisor 202. As a result, the guest VMs 206a, 206b,..., 206n interface with the hardware 100 via the host VM 204.

  The hard disk drive 108 stores an operating system (OS). The operating system may be any suitable for the CPU 104, such as Linux (trademark), Microsoft Windows (trademark) 7, Windows (trademark) 2008 server. The operating system (OS) is indicated by reference numeral 302 in FIG.

  The hard disk drive 108 further stores a Java® Runtime Environment program for realizing the Java® virtual machine (JVM) 204 (FIG. 2). The JVM is indicated by reference numerals 306a, 306b,..., 306m in FIG.

  The hard disk drive 108 further stores a Java® application program that runs on the JVM. In this embodiment, the Java® application program includes WebSphere® Application Server provided by International Business Machines Corporation. The Java® application program is indicated by reference numerals 308a, 308b,..., 308m in FIG.

  The hard disk drive 108 also stores a program for operating the system as a Web server, such as Apache.

  The keyboard 110 and the mouse 112 are used to operate graphic objects such as icons, task bars, and text boxes displayed on the display 114 in accordance with a graphic user interface provided by the operating system.

  The display 114 is preferably, but is not limited to, a 32-bit true color LCD monitor with a resolution of 1024 × 768 or higher. The display 114 is used, for example, to display a result of an operation performed by an application program executed on the JVM.

  The communication interface 116 is preferably connected to the network by the Ethernet (R) protocol. The communication interface 116 receives a processing request according to a communication protocol such as TCP / IP using a function provided by Apache from a client computer (not shown), and the host VM 204 sends the processing request to a designated guest VM. The processing result is received from the guest VM and returned to the client computer (not shown).

  Next, the guest VM will be described with reference to FIG. Since the guest VMs 206a, 206b,..., 206n are all functionally identical, here, the guest VMs 206a, 206b,.

  Then, as shown in FIG. 3, the guest VM 206a can include a plurality of virtual machines 304a, 304b,..., 304m including JVMs 306a, 306b,. .

  Under such a premise, the processing of the present invention will be described with reference to the flowcharts in FIG. Since the processing shown in FIG. 4 is executed for each Java (R) program in each guest VM, a program for controlling the entire processing can be arranged in the host VM 204 as processing over a plurality of guest VMs. .

  In FIG. 4, the program of the present invention executes a Java® program in a guest VM in step 402. Next, in step 404, the program of the present invention causes the guest VM to output a core file. The core file referred to here is a system dump file in JVM. The system dump file is preferably written to a predetermined directory on the hard disk drive 108.

  Next, in step 406, the program of the present invention extracts information on the character string object from the core file.

  Thus, when the information of the character string object is extracted from the core file as the execution result of the Java (R) program in all the guest VMs, the program of the present invention collects information from the plurality of guest VMs in step 408, and step 410 Thus, character string data appearing in multiple core files is set as a common target character string.

  As described above, since it takes a long time to search for a character string if it is a set of common target character strings obtained by collecting information from a plurality of guest VMs, the program of the present invention is shared by the process shown in the flowchart of FIG. A search key is attached to the set data of the target character string.

  That is, in step 502, the program of the present invention groups character string target data based on characteristics. The characteristics of the target data here are the length of the character string, the class file in which the character string object is generated, the jar file, and the like. The most typical of these is the length of the string.

  In step 504, the program of the present invention uses all the character strings grouped in this way, checks what number of characters has many types of characters in the group, and determines the top few character indexes with many types. Is used for hash calculation. Details of step 504 will be described later with reference to the flowchart of FIG.

  FIG. 6 is a diagram schematically showing processing for character strings grouped by character string length. Shown are the processing of a character string with a length of 9, processing of a character string with a length of 10, and processing of a character string with a length of 12. Here, three to four indexes having high information entropy are selected within a range of strings in which character strings are arranged.

  In step 506, the program of the present invention selects the hash function with the least number of collisions among hash functions prepared in advance. Here, the hash functions prepared in advance are shown as hashFn1, hashFn2,..., HashFn10 in FIG. Examples of hash functions prepared in advance include the following. In the following equation, ch1, ch2, ch3, and ch4 are character string values at the selected index. Note that these hash functions are merely examples, and various other hash functions that can be conceived by those skilled in the art can also be used. Details of step 506 will be described later with reference to the flowchart of FIG.

int hash = ch1 * ch2 + ch3;
return (hash * hash);

int hash = ch1 * ch2 * ch3;
return (hash * hash);

int hash = ch1 * ch2 * ch3 + ch4;
return (hash * hash);

  In step 508, the program of the present invention tries all shift operations when selecting an index, and selects a calculation method with the least collision. This is shown in FIG. 6 as a 1 bit shift,..., N bit shift. Details of step 508 will be described later with reference to the flowchart of FIG.

  Next, the details of step 504 will be described with reference to the flowchart of FIG. The flowchart of FIG. 7 relates to the processing of character strings in a group. In step 702, the program of the present invention counts how many kinds of i-th character. At this time, characters whose length is less than i are not counted.

  In step 704, the program of the present invention obtains the top several indexes having many types. In step 706, a plurality of character string indexes to be used in the hash function are obtained.

  Next, the details of step 506 will be described with reference to the flowchart of FIG. The flowchart of FIG. 8 uses character string data in a group and character string index, that is, what number character is used for calculation. The program of the present invention performs the given character string in step 802. Using the index, a hash value is calculated using a hash function prepared in advance as described above, and this is repeated for all character strings in the group. In step 804, the program of the present invention counts the number of times the hash values collide.

  The program of the present invention repeats step 802 and step 804 for all the hash functions prepared in advance, and in step 806, selects the hash function with the least number of hash value collisions.

  Next, the details of step 508 will be described with reference to the flowchart of FIG. The flowchart of FIG. 9 uses a hash value obtained when a hash function is applied to each character string data in the group. The program of the present invention performs n in step 904 as shown in FIG. Shift right by bits and compute L bits as hash table index. Here, L is a constant depending on the size of the hash table. Next, in step 906, the program of the present invention counts the number of times the hash table index value collides, repeats step 904 and step 906 by the number of bits that can be shifted to the right, and in step 908, the hash table index value conflict occurs. The least shift operation is selected.

As a result, for example, the indexes with high information entropy are 3, 4, and 6, the hash function with the least hash value collision is hashFn3, and the hash table index collision of the operation shifted right by 2 bits If it is the least, the index index used to find a character of length 9 will be calculated as follows:
int hc = hashFn3 (char, offset, 3,4,6);
int index = (hc >> 2 & ((1 <<12)-1);

  Here, it supplements about a hash table index. In this embodiment, a hash table is prepared for each group in order to store a shared character string. Then, in order to store the shared character string in the hash table, it is necessary to determine at which position (index) in the hash table.

  Although it is calculated by a hash function using a character string as an input, the calculation result does not necessarily correspond one-to-one with the index of the hash table. Therefore, it is necessary to correct the calculation result so that it falls within the index range of the hash table. The correction is a process of “shifting right by n bits and using L bits as a hash table index”.

The point when using a hash table is to store only one character string as much as possible in the same index of the hash table. Would result in using the strings in the group:
(1) Result of calculation with hash function
(2) It is necessary to devise so that (2) can be dispersed as much as possible in the result of index calculation. In order to disperse (2), it is desirable that the value of (1) is disjoint in the first place. This is the process of “selecting the hash function with the least number of collisions among hash functions prepared in advance”.

Furthermore, we use a string as input to get (1), but it is not necessary to use all the information in the string. So, check the character string in the group, and use only the places you think are necessary. For example, assume that three character strings “ISOLATION”, “ASSERTION”, and “ASSOCIATE” belong to a group. At this time, when arranged as follows,
"ISOLATION"
"ASSERTION"
"ASSOCIATE"
It turns out that it is sufficient to use only the indexes 3 and 4 that make the most difference among the strings in the group.

  The program of the present invention creates a DLL storing the shared character string, and causes the JVM to load the DLL to the same address every time. This can be accomplished using the prelink command for Linux ™.

  Then, when starting the JVM, place only String and char [] Class in a place where the DLL can be written. FIG. 11 shows the definition of the structure string_len3 of java.lang.String and char [] as the shared character string data for this purpose. Such a structure is defined for each length of a character string with java.lang.String and char [] as a set. FIG. 12 shows a state in which a value is actually stored in string_len3. FIG. 12 shows only two pairs of java.lang.String and char [], but it should be understood that a larger number is actually included.

  FIG. 13 is a diagram illustrating a state in which a DLL storing a shared character string is loaded at the same address across a plurality of JVMs in a single guest VM. Thereby, each JVM can search the shared character string defined in the DLL in the same manner and independently. In the search, a hash function and an index selected for each group of shared character string data are used. The shared character string is preferably arranged outside the heap (off-heap). Access from the JVM to the memory outside the heap can be achieved by using a technique such as JNI (Java (R) Native Interface).

  FIG. 14 is a flowchart of the String constructor call process in the Java (R) program. In step 1402, the program checks the characteristics (length) of the character string represented by the argument char [].

  In step 1404, it is determined whether there is a group for the characteristic. If not, the program updates the String object in step 1406. If there is a group for the property, the program will search the DLL above to see if the same character as the char [] argument is in the group.

  If the program determines in step 1410 that no character string has been found, the program updates a string object in step 1412. When the program determines that a character string has been found in step 1410, in step 1414, the program updates String and refers to char [] in the DLL.

  FIG. 15 is a flowchart of String.intern () call processing in the Java (R) program. In step 1502, the program checks the characteristics (length) of the character string represented by the argument char [].

  In step 1504, it is determined whether there is a group for the characteristic. If not, the program executes String.intern () in step 1506. If there is a group for the property, the program will search the DLL above to see if the same character as the char [] argument is in the group.

  When the program determines in step 1510 that no character string has been found, it executes String.intern () in step 1512. If the program determines that a character string has been found in step 1510, it returns a String object in the DLL in step 1514.

  As described above, the embodiment of the present invention has been described on the implementation of the JVM in the guest VM. However, the present invention is not limited to this, and the used character string is extracted from a plurality of VM environments and is common to a number of VM environments. It should be understood that the present invention is applicable to any environment where a string can be made accessible. That is, the present invention can be implemented without being limited to a specific language environment or platform.

  Moreover, as a mechanism for making a common character string accessible from a plurality of VM environments, a DLL is used in the above embodiment. However, this is only an example, and a memory outside the management of individual JVMs such as a memory outside the heap. If it is arranged, the memory may be arranged by an arbitrary method.

104 CPU
106 RAM
108 Hard disk drive 204 Host VM
206a, 206b, ... 206n Guest VM
306a, 306b, ... 306m JVM

Claims (13)

In a computer system in which an application program is executed in each of a plurality of guest VMs,
In each guest VM, extracting a common character string from each heap area used by each application program;
Placing the extracted common character string in a memory of the system so that each application program can access the character string;
Method.   The guest VM includes a plurality of JVMs, the application program is a Java program, and the extracted common character string is configured so that the JVM performs class definition at the same address each time. The method according to claim 1, wherein the extracted common string object is configured such that a class pointer is the same among the plurality of guest VMs.   The method of claim 2, wherein the string object is loaded into memory as a DLL. Grouping the extracted common strings according to the length of the string or the characteristics of how it was created;
For each group of the grouped character strings, the method further comprises the step of constructing a hash index for performing a search for the target character string.
The method of claim 1. Building the hash index comprises:
Finding the position to use for the index in terms of the height of information entropy in the string,
Calculating a plurality of hash functions at the found index and selecting a hash function having the least collision among the plurality of hash functions;
Bit shifting the character string to select the shift position with the least collision,
The method of claim 4. In a computer system in which an application program is executed in each of a plurality of guest VMs,
In the computer system,
In each guest VM, extracting a common character string from each heap area used by each application program;
Placing the extracted common character string in a memory of the system so that each application program can access the character string;
program.   The guest VM includes a plurality of JVMs, the application program is a Java program, and the extracted common character string is configured so that the JVM performs class definition at the same address each time. The program according to claim 6, wherein the extracted common character string object has a class pointer that is the same among the plurality of guest VMs.   The program according to claim 7, wherein the character string object is loaded into a memory as a DLL. In the computer system,
Grouping the extracted common strings according to the length of the string or the characteristics of how it was created;
For each group of the grouped character strings, a step of constructing a hash index for searching for the target character string is further executed.
The program according to claim 6. Building the hash index comprises:
Finding the position to use for the index in terms of the height of information entropy in the string,
Calculating a plurality of hash functions at the found index and selecting a hash function having the least collision among the plurality of hash functions;
Bit shifting the character string to select the shift position with the least collision,
The program according to claim 9. In a computer system in which an application program is executed in each of a plurality of guest VMs,
Means for extracting a common character string from each heap area used by each application program in each guest VM;
Means for placing the extracted common character string in a memory of the system so that each application program can access the character string;
Computer system.   The guest VM includes a plurality of JVMs, the application program is a Java program, and the extracted common character string is configured so that the JVM performs class definition at the same address each time. The computer system according to claim 11, wherein the extracted common string object has a class pointer that is the same among the plurality of guest VMs.   13. The computer system according to claim 12, wherein the character string object is loaded into a memory as a DLL.

Images