JP2003058401A - Data structure for storage in computer memory - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide technology and a tool for optimizing performance characteristics of a digital tree and similar constitution. SOLUTION: A data structure incorporates branch nodes (105, 106, 113) selected from a group consisting of linear branch nodes and bit map branch nodes 400, expanded branch nodes selected according to the number of subexpanses with contents and the whole status of the digital tree and leaf nodes (108, 110, 116 to 123) which are selected with the group consisting of linear leaf nodes and bit map leaf nodes, have a plurality indexes respectively, accord with the level of the numbers of leaves in the digital tree and indexes in the leaves, and include only undecoded indexes.

Description

DETAILED DESCRIPTION OF THE INVENTION [0001] The present invention generally relates to data structures.
In the field of construction, in particular, the data in which the structure of the data organization is stored.
Data depending on the data and compressed to match the data
It relates to a hierarchical data structure with structural elements. [0002] BACKGROUND OF THE INVENTION Computer processors and corresponding
Memory continues to increase in speed. Hardware is the thing
However, as the speed limit approaches,
Data access time must be significantly reduced
It is. Even if these limitations are not a major factor, software
Hardware platform by maximizing hardware efficiency
Maximize form efficiency and reduce hardware / software
Expand the capacity of the entire software system. One way to increase the efficiency of a system is to use
Although it is based on efficient data management,
Smart selection, related storage, and retrieval algorithms
Is done. Various data structures, eg related to the prior art
Storage and retrieval algorithms for arrays, hash
Tree, binary tree (binary tree), AVL tree (height)
Binary tree balanced with b), b-tree, and skip
It has been developed for managing data, including lists. [0004] These prior art data structures, related
For each successive storage and search algorithm,
Faster access time and minimized memory overhead
There is an inherent trade-off between For example,
Rays are fast-read through address calculations for single array elements.
Be prepared for indexing, but before a single value is stored
Requesting that the entire array be pre-arranged in memory,
And unused intervals in the array waste memory resources
I do. Instead, binary trees, AVL trees, b-trees, and
Kiplist pre-allocates memory for data structures
And minimize the allocation of unused memory.
But as populations increase,
It becomes clear that the access time increases. The array has a simplified structure,
Prior art data for high-speed access to encrypted data
Structure. Memory must be located throughout the array
No, but its structure is not flexible. Array values are
Increase the index placed on each element of the ray by size
Palm, adding the offset of the base address of the array
, And can be checked in numerical units for each position. Typically, a single central processing unit key is used.
Cache linefill is required for arrays
Necessary to access the element and the value stored here
It has been. Explained and typically embodied
As such, arrays are inefficient and have relatively inflexible memory.
It is. However, access is given as O (1)
It is. In other words, independent of the array size (disk
Ignoring swaps). [0007] Alternatively, the data structure previously described is
Branch tree, b-tree, skip list and hash table
Memory is efficient but not desirable.
It will be available in a form that includes signs. For example, hashing
Is the scattered, possibly multi-word index
(For example, a string) to an array index
Used to do. A typical hash table is
Is a fixed-size array, where each index is
Hashing algo performed with original index
The result of the rhythm. [0008] However, efficient hashing
In order for the hash algorithm to be
Box. Hash table
Also, all data nodes have the original index
Seeking to include a copy of (or a pointer to)
Identify the nodes in each synonym chain
can do. Using hashing, like an array
Requires that memory be pre-arranged
But if properly designed, ie stored
The characteristics of the data to be
Hashing algorithms created and embodied, collision resolution
If the technology and storage structure match,
It is part of memory that must be located for
Minutes. In particular, a digital tree or a trie
e) provides fast access to data, but generally
In addition, memory is inefficient. Keep tree branches narrow
Should handle a sparse set of indices.
And may be more memory efficient, but the result is a deeper tree.
Memory references, bypasses, and cache lines
Increase the average number of files, resulting in less access to data
Become slow. This latter factor, cache efficiency
Maximizing that such a structure is still being discussed
Sometimes ignored, but may affect system performance
May be the dominant factor. A trie is a tree of small arrays or branches.
Where each branch is one or more of the indices
Decodes more bits. Prior art digital tools
A tree is a simple pointer or array of addresses.
Has a launch node. Typically, a pointer or ad
Less size improves the memory efficiency of the digital tree
To be minimized. In the "tail" of the digital tree, the last brand
Switch is the index of the last bit, and
Decode the element point to the fixed storage. Tree
"Leaf" is a special index
A memory chunk that has a specific application structure.
You. The digital tree has no index
Or population zero (or empty subex
Allocate memory to branches that are called pans)
It has the advantage that there is no need to use it. In this case
Poi pointing to an empty subexpanse
Counters are given unique values and represent valid address values.
Is called a null pointer that indicates no. [0013] Furthermore, the in-stored in the digital tree
Dex allows the identification of neighbors, in sorted order
Can be accessed. Digital used here
The "expanse" of the tree is a digital tool
The range of values that can be stored in the tree
Lee's population is in the tree
A set of values that are actually stored. Similarly, the execution of the branch of the digital tree
The span is the width of the index that can be stored in the branch
And the population of the branch is the branch
Is the number of values (eg, counts) that are actually stored in
You. (As used herein, "population"
The term is a set of indices or their indices.
About any of the box counts,
The meaning of the term is relevant in the context in which the term is used.
It is clear to the trader. ) "Adaptive Algorithms," by Acharya, Zhu and Shen.
for Cache-efficientTrie Search ”
A cache efficiency algorithm is described. each
The algorithm uses a different data structure,
The structure of represents the different nodes in the trie,
Partitioned arrays, B-trees, hash tables, and
It contains a vector. Selected data structure
Has the same cache characteristics as node fanout.
Depends. [0015] The algorithm further defines
By dynamically switching the data structures used,
To adapt to changes in the fan-out of the Finally, each
The size and layout of the data structure depends on the cache characteristics and
Also determined based on the size of the alphabet symbol
You. This publication further discusses real and simulated
It includes a performance evaluation of the specified memory hierarchy. In other publications, known and used by those skilled in the art,
The following describes the data structure:
There is Fundamentals of Data Structures in Pasca
l, 4th edition, Horowitz and Sahni; pp582-594; The Art of
Computer Programming, Volume 3; Knuth; pp490-492; Al
gorithm in C, Sedgewick, pp245-256, 265-271; "Fas
t Algorithm for Sorting and Searching String ”; Be
ntley, Sedgewick; "Ternary Search Trees"; 587192
6, INSPEC summary number: C9805-6120-003; Dr. Dobb's Journ
al; "Algorithm for Trie Compaction", ACM Transan
ctions onDatabase Systems, 9 (2): 243-63, 1984;
uting on longest-matching prefixes ”; 5217324, INS
PEC summary number: B9605-6150M-005, C9605-5640-006; "So
meresults on tries with adaptive branching ”; 6845
525, INSPEC general number: C2001-03-6120-024; "Fixed-bu
cket binary storage trees "; 01998027, INSPEC overview
No .: C83009879; “DISCS and other related data st
ructures ”; 03730613, INSPEC outline number: C90064501;
And `` Dynamic sources in informationtheory: a g
eneral analysis of trie structure "; 6841374, INSP
EC outline number, B2001-03-6110-014, C2001-03-6120-023. The extended storage structure is disclosed in US patent application Ser.
09/457164, filed on December 8, 1999, titled `` A Fast Eff
icient Adaptive, Hybrid Tree ”(hereinafter 164 patent applications)
), And is filed in the same manner as the previous application. here
The data structure and storage method described in
Perform self-tuning and use the “Expans” based storage
Self-conforming structure that places cards to minimize storage requirements
For efficient and scalable data storage and tuning.
Provide search and search capabilities. The structure described here is
However, those that make full use of the specified data distribution
is not. In the storage structure described in the above-mentioned patent application,
The extensions are detailed in the following application: It's rice
National Patent Application No. 09/725373, filed November 29, 2000,
Ittle `` A Data Structure And Storage And Retrieval
 Method Supporting Ordinality Based Searching and
Data Retrieval '' and filed in the same manner as the previous application.
You. In this latter patent application, the data structure, associated data
Data storage and search methods are described.
The method is based on a hierarchical structure of stored or ordered elements.
Sum of more referenced elements, ordinal values in the structure
Access to elements based on their identity and order of elements
It is provided promptly. In a structure embodied by an ordered tree
Stores the sum of the indexes that exist in each subtree
Is done. In other words, the root of each subtree is
Stored in or associated with the code. This high level
A node points to its subtree or
At or associated with the head node. Data structure
Specific requirements (e.g., creating new nodes,
Allocation, balancing, etc.), as well as data insertion and deletion
Includes an update step that affects the sum. [0020] However, the present structure
Cannot make full use of certain scattered data situations.
Therefore, the performance characteristics of digital trees and similar configurations are optimized.
There is a need for technology and tools to optimize. [0021] SUMMARY OF THE INVENTION A data structure according to the present invention
Is a digital tree (or "traffic") stored in memory.
B)) As a dynamic array based on the data structure
Can be handled and handled through the root pointer.
Includes a self-modifying data structure that can be Empty tree
This root pointer is null, otherwise
If so, it is the first of the branch nodes in the digital tree.
Refers to the hierarchy. Low fan-out branches are avoided.
Or an alternative structure that consumes less memory
However, this consumes less memory. On the other hand, conventional digital
Most or all of the performance benefits of the tree structure
, Index insertion, search, access and delete
Operation. This improvement would otherwise be scattered
In a solid or wide or shallow digital tree
Widespread null pointers are wasted
Reduce and eliminate memory. In particular, in reducing the size of the structure
Branch fixes in comparison to
The additional processing time required to complete and contain
Yes, so fetching data from memory is more efficient
And in each CPU's cache line fill,
Reduce Null Pointers while Capturing More Data
You. The present invention uses, for example, a rich pointer structure.
Linear and bitmap branches and embodied
Contains leaf. Relocate flexible nodes
Change the subexpansion population by
In order to automatically rearrange. [0025] DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a data processing system.
Access by application programs running on
Data storage system in computer memory for
System and method. The system is stored in memory,
Root pointer pointing to "wide / shallow" digital tree
Data structure and related information. Digital
Trees are arranged hierarchically and hybrid abstract data
Compressed to fit using type (ADT)
(Branch node) and multi-index leaf
(Leaf nodes). In this application example, the ADT is virtually the same
Multiple data structure with the meaning of but extended to different characters
See In addition, the "index
The word “s” can be a number, string, token, symbol, or
Is a field that constitutes another such specification or expression
Key or set. By embodying a digital tree, data
Data (set of indexes or keys) is purely popu
"By expansion" rather than "by translation"
Organized first. It simplifies tree traversal
And has various advantages in modifying the algorithm.
In particular, a wide digital tree potentially has a high
Large fan-out and large population
The tree becomes shallower, even for those that are
To be faster and therefore "scalable enough"
ing. By using a compression branch, a wide
Performance benefits of the switch
Out, and thus use of memory
Data (index or key) to match
I have to. By using this technology, the sub
Only expands, i.e. the stored index
Containment must be represented as a compressed branch
No. Empty subexpanses are typically (but not required)
Lacks). Further, a plurality of indexes (or keys)
And its associated values, if any
However, it is in the "multi-index leaf"
However, this means that the tree is broken down by one or more levels.
Shallower, thus reducing memory usage and increasing access
Speed up. Compressed multi-index leaves are more
Keep many indexes, but keep the same set of indexes
More brans throughout the tree to hold
It doesn't mean you have to insert a switch. Such "cache efficient" compressed
Branches and leaves minimize “cache fill”
Access to random access memory (RAM)
Access so that it is relatively slow
Most desirably designed in terms of cache lines
You. Therefore, the present invention provides a digital tree
Several types of blocks to optimize the performance of the structure
Includes lunch and leaf compression. Improvements like this
Contains the linear and bitmap branches (ie
Nodes), linear and bitmap leaves, and these
Includes rules and methods for accomplishing the use of nodes. This
This includes, for example, general-purpose, memory-efficient operations.
Working, convenient, expansion and leaf compression branch
The use of index compression. The linear branch node according to the present invention has
There is a subexpanse and the corresponding next level pointer
Low fan-out by having a list of
Address a branch. More generally, a linear bran
Key is a key or a set of fields that make up a key.
Select one or more subexpanses
Contains a list of subexpense descriptors, including criteria.
According to a preferred embodiment of the present invention,
The descriptor is a 1-byte segment with a 32-bit index
It is. Preferably, the linear branch is
Platform bound to a single CPU cache line
You. As the content of the subexpans grows,
Map branch nodes contain subexpenses
Binary vector indicating if there is (ie not empty)
Can be used, including This binary vector
Subexpans (or equivalent multi
List of pointers to level data structures). The linear leaf node according to the present invention is similar
The use of multi-index leaves
To the index of the most popular population. Ma
Multi-index leaf for large populations
Contains a list of valid indexes for Tree low
Medium to high population concentrations at high levels
Bitmap leaf nodes, on the other hand, have a valid index
With a binary index of
For example, a value area corresponding to each effective index is included. In the present invention, a high-frequency, memory-efficiency-oriented
Of the compression branch according to the machine of the compression branch. data
According to this aspect of the structure, the data stored in the data structure
Threshold used for the entire set per index
Value (probably measured in bytes per index
Occupies less memory, or
Is the pop-up of the subexpansion under the bitmap branch
When the ratio is sufficiently large, even if the high-frequency distance is sufficient
But not linear or bitmap branches
Placed in a stretched form (ie, a stretched branch node)
Can be replaced. As a result, additional memory costs
Has reduced computation time, and
The schfil is reduced. Big population
Index, especially well-clustered indexes
The use of this option for data
Maintain high-speed access to services and related data
"Clearing out" the excess memory needed for Between the branch and the node, ie, the linear
Branch and linear leaf, and also bitmap branches
Note the degree of symmetry between the and the bitmap leaf
It is. In this embodiment, this symmetry is determined by each index.
It becomes apparent when a box is mapped to an associated value.
The internal nodes of the tree point to pointers to auxiliary nodes.
Index part (digits)
Terminal node is actually outside the tree
A pointer defined by a source
A well-decoded index for the range that often contains
Map. However, with this symmetry, the extended bran
It means that there is no leaf equivalent to Ji. Yo
Higher level reef exceeds specified population
The subtree under the new branch, or
Appropriately at lower levels for more compressed leaves
Is converted to The minimum level of linear leaf is
When it goes over the bitmap leaf
Is converted to According to another aspect of the invention, a target
Part of the index is decoded at each level of the digital tree
Exploits the fact that the leaf index is compressed
are doing. The index is partly while the tree is inverted.
Since each segment is decoded separately,
Only the remaining parts need to be stored in the leaf.
And the number of bits or bytes decreases at each lower level.
Constitute this undecoded part. As a result, lower level leaves
(I.e., leaves further away from the root)
More indices in the same space as the bell leaf
Remember, the latter has more bits, but each index
To represent a larger undecoded part of the data. Obedience
The worst case index insertion and deletion
Exclusion, but localized, one level above the tree
No cascade up or down, each worst
In this case, the number of insertions and deletions is minimized. Be careful
It is clear that this type of compression is best for fixed-size compression.
This is also true, but it can be a character string or bitstring.
It is not useful for variable size indexes such as tags. Bits common to a plurality of keys (indexes)
Digital tools so that they can be skipped (not represented).
Note that the tree can be compressed. Such a tree
-Whether it is fixed or variable size,
To clarify a leaf, a key is
-The entire copy must be remembered (needs clarification)
Unless otherwise required). This is, as embodied in the present invention,
Distinguished from leaf compression, where the decoded part
Index for all indexes in the subexpansion
Tree passing or skipping
(Compression), whichever is required, always remembered
Is recoverable from the branch node, and
There is no need to store it in the node. The present invention provides an efficient cache cache.
Proper combination of different ADTs for lunch and leaf
Provide a combination, and the combination is stored in an example.
Based on a contingency data set
A wide digital tree is shared with memory efficiency
Access or modify a wide dynamic range
It will be. Wide dynamic range
Are small to large data sets.
That is, the index is low to high (billions)
Unit), and the type of data set
Or crowded, regular, or random
It can be a dex or a key. Well designed, broad and dynamic
Hybrid digital trees with ranges
Does not require (or even makes it possible)
Software interface as a simple and dynamic array
Interface. The present invention passes through a data structure containing a pointer.
A wide range of constructs and data structures connecting nodes
Other methods to prepare for the passage of traffic
And may be embodied as For purposes of illustration, a preferred embodiment of the present invention
Is the range of the digital tree construct that contains the extended pointer
It may be embodied within. This is because of the pending US patent
In the application, the title "Cash effect with rich pointer
System and Method for Efficient Digital Tree "
Is disclosed. The pointer is a null pointer,
Or used to refer to a branch or leaf node
In some cases, such as shown in Figure 2A or with an immediate index
In such a case, the first configuration shown in FIG. 2B may be used.
No. The object pointed to by using the rich pointer
Object type, such as linear or bitmap, branch or
Give designation of leaf, etc. As another embodiment of the present invention, a conventional poi
Other components such as a printer may be used. For example, target
Object that identifies or points to the object
Is self-identifying (ie, the type information is child, not parent)
The pointer itself (which is stored in the node)
May be an object that is 8 bytes in a row
Otherwise the least significant 3 bits are not used
The least significant bit of
And so on. As shown in FIG. 2A, the basic points
Data structures, for example on 32-bit platforms
Contains two 32-bit words, where one complete word
The body is used by the pointer to
Row is decoded between other nodes, 0 and 2 bytes
Index population between 1 and 3 bytes
Fields and 1 byte type fields
Will be fixed. For null pointers, type field
All bytes are zero except for. Instead, the first
Word is a pointer to an auxiliary branch or leaf node
It is. The decryption and population fields are both
All but one byte of the second word are satisfied. Pointer construct containing immediate index
Accesses the index, as shown in FIG. 2B
Eliminates the need to redirect and point to other nodes
Has been removed. As explained in the referenced patent application,
Another variation of these pointer constructs is to set the value to
Used to map to each index,
Adapted to accommodate word sizes of various machines
You. The present invention uses these pointers to
, Ie, interior nodes and leaves, ie
Form an ADT containing terminal nodes. This data
According to the structure, the digital tree consists of branch nodes (lines
Shaped, bitmap, or decompressed) and leafno
Code (linear or bitmap)
Contains. Each branch is either a constant (stretched) or
Array of virtual (linear or bitmap) pointers
And preferably 256 such rich pointers.
You. That is, each node has up to 256 sub-expansions
With fan out. In a preferred embodiment, the index
The data is decoded eight bits at a time, ie one byte.
In other words, each digit is one byte, and each
Real or virtual fanout
256. Digital tree at branch node
Can have any fanout, for example a tree
Is 26 when decoding the 26-character alphabet
However, it is possible that fanouts that are not a power of 2
It will be clear to those skilled in the art. The binary tree is usually populated
Tree (binary storage tree)
Where the key is stored in each node
Key value. However, also
The narry tree was separated by expanse (binary data
Digital), two fan-outs with one digit each
Can be a tree with In addition, the hive
The lid tree, in different branches or trees,
It may have various fan-outs. However,
Computers are inherently part of word-sized objects.
To efficiently handle byte-sized objects.
With 256 consistent fanouts, or 1 byte
It is a finding of the present invention that the digit size is the most efficient.
I know by the lighter. The compressed branches are linear and bitmap
To supplement the elongate type branch. This latter
Branch types are, for example, 256 subexpansion sports
Conventional digital tree function using arrays with interconnects
Is supported. The actual fanout (ie,
The number of subexpans with content) is relatively limited
New branch is created during index insertion
When done, the "compressed"
A punch is used instead. This compressed branch has 256 sub-edges.
It may be viewed as a virtual array of xpans pointers,
(For reasons described below, instead of one, two
Need schfil to pass through related nodes
Requires much less memory)
No. Referring to FIGS. 1A to 1E, the root pointer
Card 101 is the underlying data structure of the digital tree.
Used to access. Root pointer no
The node 101 is a diagram, the first or “top” level node 10.
2, this description is indicated by the arrow pointing to the branch node,
Address information. (Supplementally, it is used here
The terminology used assumes 32-bit implementation.
Where the index is a single
Is a word. Then, as a result, "level 4"
Label the top node of the tree pointed to by the
The children of the bell 4 node are designated as level 3 nodes
And so on. On 64-bit machines, the root
Is a level 8 node, its children are level 7, etc.
Become. Thus, the level of a branch or leaf node is
In the index stored or below the node,
Equal to the number of digits (bytes) remaining to be decoded. This
The numbering method of the 32-bit and 64-bit trees
With the additional benefit of keeping the same minimum level.
To use with trees of various sizes
Simplifies the source code needed for Further supplement
So, while this arrangement is typical, this explanation
For example, a leaf node
-As the highest (eg 4) level
Even if you adopt other arrangements, such as those that include
Good. ) Top level node 102 is an extended branch node.
This references up to 256 lower level nodes.
Includes an array of 256 rich pointers for data
Structure, i.e., from 00000000 to FFFFFFFF in octal notation
Represents the entire expanse of the index. Top level
Node 102 responds to expansions from 00000000 to 00FFFFFF.
1st rich, corresponding to level 3 linear branch 105
Contains pointer 103 (also called a conforming object)
No. The other rich pointer 104 has the index FF000000
From FFFFFFFF to the final expanded part
Have been. Rich pointer 104 is the most important level
The 1 / 256th above 3 points to the expanded branch 106. The first subexpansion of level 3 is linear
Contains auxiliary nodes in the form of branches 105. As shown
And the linear branch 105 is fan-out (NumRP = branch
Count of number of child nodes referenced by), branch
Index corresponding to the subexpansion referenced by
List of columns (number of digits)
Contains a list of pointers to subexpanses. sub
Similar from the expansions E1 to E3 slots
Although the pointer is not shown, it exists,
Is listed as E4, from 00FD0000 to 00FDFFFF
The final subexpansion, representing the subexpansion containing
Only pointers to the resources are shown. Therefore, the linear bra
The fourth rich pointer of the
Refer to bitmap branch 113 of
Linear branch leaf 118-122 and bitmap leaf 11
6, 117 and 123. At the end of the higher-order node 102,
Bell 3 extension branch referenced by rich pointer 104
Is done. Only two of such references are shown for illustrative purposes.
However, typically, the extension branch 106 has a large number of complements.
Refers to an auxiliary node. Supplementally, the contents are separated
One branch is linear or otherwise to save memory.
Converted to bitmap branch format,
To nodes using one or two cache line fills
Still have access. As shown in FIGS. 1A to 1E, level 3 extension
Branch 106 includes a link to level 1 linear leaf node 108.
Array of 256 rich pointers, including switch pointer 107
including. Supplementally, the rich port according to an embodiment of the present invention is described.
The use of interfaces allows pointers to "street" through the levels of the tree.
Kip ", but this is because the middle branch
Unused indirect references (indirectio
n) to avoid. The other rich pointer 109 is 2
Level 2 linear leaf containing two 2-byte indices
Refers to node 110. The rich pointer is a branch according to the present invention.
And data compression that are compatible with leaf compression
It may be used to embody. Although it is not necessary,
The use of a switch pointer is compatible with one embodiment of the present invention,
Support this. Such a rich pointer structure has few
At least two types of rich pointers or compatible
It covers objects, which are represented in Figure 2A.
Pointer type and immediate type represented in FIG. 2B
including. Immediate type supports immediate index
You. That is, the population of Expans
When the pointer is relatively scattered, the rich pointer
In the tree tree branch, index "immediately"
Can be used to remember
Must pass through the digital tree to the lowest level
No need. This format is the same as immediate machine instructions.
Kind, where the instruction is assigned to every displacement byte.
Identify the immediate operand that follows. Therefore, an immediate index or a small index
The dex is stored in the node, and one or more
Avoid re-orientation or otherwise traverse the tree
Requested and reaches a remote leaf node. It
The more immediate the index, the more memory is allocated
For multiple memory references and data access
Instead of requiring a small cache fill,
Packing (or a small number of indices)
Is provided directly to the rich pointer structure. [0062] Two-word formater according to preferred embodiment
Will immediately support the inclusion of the immediate index.
To This means that in the rich pointer
Index digits of the entire rich pointer excluding fields
Achieved by storing numbers. 32-bit system
A rich pointer embodied in a
From hour index to seven 1-byte indexes
In a 64-bit system.
The up-pointer records up to 15 1-byte immediate indexes.
You may remember. A rich policy that supports an immediate index
The generalized structure of an interface is
2B), as shown in FIG. 2B. Rich pointer
Is one or more platform word services.
Index depending on size and index size
Index “I”, index size and immediate index
The number of bits also includes an 8-bit type field to encode. FIG. 3 shows a 32-bit platform.
When embodied, the linear branch structure according to the invention
Details are explained. Linear branches are fan-out,
That is, the subexpansion referenced by the branch
It consists of 1 byte indicating the number of subexpanses.
Subex with content indicating a number (eg, 0 to 255)
Saw consisting of one byte (ie number of digits) for each bounce
Following the array that was reset. The number of solid subexpanses is
Following the corresponding array of expand pointers. The present invention
Combines the embedded at the ends of the two arrays,
Arrays are "same" for fast insertion and deletion.
"Grow in place". Subexpans
Both arrays (ie, digits and pointers) are purely
Populated, organized or packed,
Not uniformly addressed by spans
Are organized or accessed by EXPANS
Is there. Typically, the linear branch shown in FIG.
The mode is used for the actual fanout,
The subexpans with the contents is relatively small,
7 out of 256 possible subexpans per lunch
Up to the rich pointer. One embodiment of the present invention
The linear branch node according to the state includes the above-mentioned three continuous areas.
Is a subexpansion with content,
Sub-expanses (1 byte each)
List and the list of corresponding rich pointers
Includes each two words long. (By those skilled in the art
Number, type, size, and area
For ordering, other configurations may be used in alternative embodiments of the invention.
May be used. ) When using this predetermined method,
The largest linear branch containing rich pointers is
1 byte for the number of spans, subexpansion resource
7 bytes per strike, thus 2 words in combination
(In a 32-bit system). Coun
And the list of subexpense lists are rich
14 words about the pointer itself, followed by the entire construct
Fits 16 words or an entire cache line
I do. Returning to FIG. 3, four solid subexpans
The entire space is moved from E (expansion) 1 to E
(Expans) 4 is referred to. FIG. 4 shows a bitmap branch,
Re-embodied on 2-bit word size platform
It was done. The bitmap branch node is medium
256 bits to indicate the presence of a subexpense
The first part 401 containing the first part (32 bytes), followed by the second part 4
02 is a rich pointer to the sub-expansion with contents
And a normal pointer pointing to an independent sub-array. This
The structure of
Bytes per index into bits per index
Compressed and bitmap 0 bit for invalid index
Savings of up to 7/8, except where
You may think. Conceptually, the sub-expansion pointer
Is stored in a single array (402 parts) following the bitmap
Is done. However, according to a preferred embodiment of the present invention,
Simplifies memory management and speeds up insertions and deletions
In order for the bitmap to follow 8 normal pointers
Each with a subexpr between 0 and 32
Corresponding to independent subarrays 408 and 409
You. Since it can be addressed by the number of digits (0-255)
Bitmaps are now organized in Expands
But the latter, on the other hand,
Only the corresponding, sub-arrays are packed into
The expansion pointer is reset "by population".
It is turned into a strike. According to another embodiment of the present invention, once the
Bit pointer branch sub-array is memory
Use of the maximum, i.e. the number of pointers reached, subarray
Has 32 sub-expansion spoilers
Once the subarray is ready to hold the
Files are expanded to save time during insertion, insertion, and deletion.
Decompressing the rich pointer sub-array means
Even for subexpanses with empty indexes
Even if all bits are
Set in pans, rich pointer subarray
Is unpacked (unpacked) and simple,
Accessible arrays and null rich poi
Represents a solid subexpansion with
means. Therefore, as shown in FIG.
A launch is a two-row object that can be linear or extended.
Somewhat more complicated than lunch. The first level (401)
The bitmap itself, but the 32-bit word
8 sub-expanses according to the realization of
Includes subdivided 256 bits (32 bytes)
ADT or sub-array (eg, arrays 408 and 409)
Eight pointers (402 parts) follow. Each ADT 400 is packed with a rich pointer.
Each rich pointer is composed of a related linear list.
For each bit set in the linked bitmap
I have. In a 32-bit system, eight words are bit-
Is determined for the step (32/4), and 8 words are
On the other hand, a total of 16 words are required. The total of 16 words here is the system puff.
Important in performance,
According to the embodiment, it is one CPU cache line
Because it is equal to To add, 64-bit system
In, only 4 words are needed for the bitmap
While 8 words are still
Required, resulting in a 16-word cache line.
Assuming that 4 words are wasted. For example, the bitmap 404 is a hexadecimal value 0000b
074, which is a binary vector and
Has a dex value. [0074] [Table 1] According to this example, it is represented by the column after Table 1.
The binary vector is 40_{he x}From 5F_hexSub-exe in the range
Subs including spans 42, 44, 45, 46, 4C, 4D, and 4F
Indicates that an index exists in the expansion
are doing. Related normal pointer 406 in this range (FIG. 4)
Is the subexpansion indicated by the associated binary vector
Corresponding to each of the subexpanses
Points to the array 408 that contains the rich pointer. For comparison, the extended branch is represented in FIG.
Have been. This construct is a simple array of rich pointers.
B, including in this case 256 such rich pointers
Null null used to represent empty expanses
With a pointer. Here per rich pointer
Assuming two words, such a decompression branch would be 512 words.
Required. In the present invention, the general-purpose memory efficiency is further increased.
Have. That is, the cache occupied by the linear branch
Too many lines (according to a preferred embodiment of the present invention)
The limit is a single 16 word cache line)
Until fan-out (ie subexpans with content)
Number of branches), the branch is a bitmap brand
Is converted to j. Such bitmap constructs are
Can handle the `` added fan-out ''
It should be noted that no exchange is required. On the linear
Is a null subex in any of the bitmap branches.
Don't waste memory on Pounce. However, linear or bitmap blocks
Population under lunch is needed for extension branch
Memory is large enough to “amortize”
Data structure (preferably measured in bytes per index)
Overall or general memory efficiency remains
Not exceed the selected / tunable value.
The punch is converted into a stretch type as appropriate. This is the null subexpansion pointer
While wasting memory on the
One round trip (and cache fill) is secured.
To add, the latter parameter, a generic note
At least a larger poppi to support re-efficiency
In the compilation tree, the root pointer is
The total number of bytes used by the
Intermediate to remember total count of remembered indexes
Refers to the data structure. This intermediate data structure
Neighbor of the top branch node or the top of the resulting tree
It may exist at a point that becomes a branch. According to the present invention, leaf compression is also performed according to the linear compression method described above.
Indexing, including bitmap leaf types
Used in the form of a leaf. Typically, digital tools
Each lookup in one branch of Lee is the next lowest
Level sub-expansion pointer, probably memorable
Reduce the expansion or breadth of the effective index. Therefore, each unique, not yet decoded
The remaining bits need to be stored. I already explained
As in the population within the expansion (ie
The number of effective indexes) is small, a single object
It is useful to store it in Single object here
Can be searched sequentially, or at least instantly,
Through more tree branches, each with a single index
To the specific use leaf related to
There is no. In the simplest case according to one embodiment,
Index-only leaves are a list of valid indexes
It becomes. As the present inventor has determined empirically,
The most desirable size of the leaf is relatively small
For example, two cache lines, that is, a typical
On a typical 32-bit word size platform
32 words or 128 bytes or less. Two cups
Sorted index on cache line
In the sequential search of the list of
Half of the time found in Shrine (1 fill) and the second
After half an hour on the line (2 fills), on average 1.5
Requires cache fill (data enters cache
Not assumed). That is,
When the relation is small enough,
A digital tool as a list, bitmap, or other ADT.
1 to 2 cashiers, not more levels of Lee
It turns out that it is desirable to memorize in the memory. FIGS. 6 (A)-(D) and FIGS. 7 (A)-(C)
Shows an example of a linear leaf according to the invention. Linear
Is an ordered list of indices,
Each is composed of N undecoded bytes. Where N
Is the lowest level, that is, the level furthest from the root
At the level of the tree using the level 1 convention
You. (Additionally, this is how the tree is described in the traditional way.
The opposite is true. In the traditional way, level numbering
Starts with the top node at level 1 and each child
The level is higher than the parent's level. ) According to a preferred embodiment, population of leaves
(The number of indexes is the same as the leaf size)
Stored with the pointer to the leaf, but not on the leaf itself.
Not exist (complete with a single root level linear leaf
There is a fairly small array configured in
An exception is the execution example used for ). As shown in FIGS. 6 (A)-(D),
Is the leaf in the tree for each index
Minimum number of bytes remaining to be decoded at
Of sorted indexes that only remember
This is a locked array. 7 (A)-(C) show that the values are
Disjoint value used when related to the index
Alternative to add an area with this list of values
Is shown. In addition, at the root level
Unlike leafs, linear leaves are indexed
Need to include a population field for
Absent. Instead, according to a preferred embodiment of the present invention, the parent
Nodes carry a population field. Table 2 shows various levels of trees.
(In order to lower the leaf level,
Need more bytes to represent index), 3
For 2 and 64 bit word size platforms
Of the system with the value of the index
The arrangement and arrangement of leaves. [0085] [Table 2] To add, in each case the leaf index
The remaining undecoded bytes of each index
The number is the type field of the referenced rich pointer structure.
Enumerated in the field. Minimum leaf population
The index is an index that can be stored by the immediate rich pointer.
Is based on some, and as a result
Is "immediate", that is, rich points
Is stored in the data structure itself. In contrast, the maximum leaf population
Is two caches for an index-only leaf
Line capacity (eg, 32 words)
Or, if the value and index are related leaves,
Limited by cache line capacity (for example, 64 words)
Limited. Alternative implementation on 64-bit platforms
In the invention of the form, the leaf having only the index is 1
Immediate index as soon as index 6 is reached
Rearranged directly from type to bitmap leaf.
Its purpose is to use a single population size,
Linear leaf to bitmap leaf in insertion
And create 17 indices in the same subexpansion.
To avoid reaching the box. The memory cost of a linear leaf is equal to a predetermined threshold.
When the value exceeds the set value, for example,
Bitmap leaves are useful when reaching. Follow
At the minimum level of the tree,
Only the remaining single index digits (eg bytes)
However, the sub-expansion with 256 indices
Has a reasonable population (eg 17 indexes)
Each leaf in the subexpansion.
To be represented as a bitmap with 1 bit for each
More memory is conserved and therefore 256 bits or
This is 32 bytes. Implemented on 32-bit word platforms
Example of index-only bitmap leaf to be instantiated
Is represented in FIG. In the figure, each horizontal
The rectangular area represents one word. 64-bit platform
The word grows, and in the bitmap
Leaves are the same, except when the word is half
Become. Bits in the bitmap are expanded leaves
Which indexes in the source can actually exist,
In other words, it shows whether it can be memorized. FIG. 9 shows that the data structure in question is represented by a value.
Alternative implementation for associating with a forgotten index
FIG. As shown,
Value fields containing one word per index are bit mapped.
Included in the briefs. Like a bitmap branch,
Bitmap leaf embodiment is a two-column construct
Is a rich pointer array (2 words per element)
Instead of a value domain subarray, ie, one word per element
The difference is that it is a list of values that have On 64-bit platforms, the bit
The map needs 4 words instead, 4 words are used
I can't. Memory included as a result of using two rows of constructs
And the number of bytes in the cache line is smaller,
Modifying the list is faster. As in the case of the bitmap branch,
Is small enough, for example, 8 bits or 1 byte
If the node divided into 256 remains without being decoded
And the population of Expans is large enough
When it is greater than or equal to 25 indices
Index the effective index during expansion
It is not convenient to represent as a bitmap instead of a list of
(Ie, "cheap for memory")
Call This characteristic is based on the undecoded byte for each index.
Level 1 of the tree (that is, the root
True only at the leaf farthest from the node)
You. According to a preferred embodiment of the present invention, a bitmap
Using leaves is a level 1 leaf, that is, only
Limited with respect to an index containing one undecoded byte
You. The data structure further includes a leaf index
Including compression. As described in connection with linear leaves,
The traversal of the digital tree is based on the search, inserted, or deleted
Target index part (for example, 1 byte segment
G) to decode the index bits. Many
In many cases, when the leaf is reached, it is stored in the leaf
Some or most of the bits in the index
Most have already been decrypted. In other words, the tree
And stored for each location (ie, digitally). Therefore, the remaining undecoded index bits
Only the (suffix) is stored in the leaf. Therefore, 3
4-byte index on 2-bit platforms
Are decoded one byte at a time (ie, the tree
Two 64-byte wide keys)
Cache line size (ie 128 bytes)
The (terminal) leaf is as shown in Table 3.
It may contain the number of compression indexes. [0095] [Table 3] Referring to Table 3, one byte for each index
, Once population exceeds 24 indices
The 32 byte (ie 256 bit) object
Of the possible indexes on the low-level leaf
It is enough to hold a bitmap representing everything.
Also, leaf index compression has additional advantages
A point should be noted. In particular, the lower level indexing of the tree
Index retains more indices than the current level leaf.
You can have. As a result, even if the immediate index
Single overflows existing leaf without any
Even if you cascade by inserting an index,
Never create more than one additional level in the tree
Absent. Similarly, by removing a single index
More than one level in the tree without cascading
Never delete it. In other words, leaf compression is
It supports good locality of change during modification. As described above, this data structure is
One that describes fixed-size indexes
Character strings and bit strings of indefinite length
To accommodate various sizes of indexes
It may be modified immediately. For example, an undefined length bitstream
Single indexing by using indexing as an index.
The suffixes that were left on their own, if small enough
If so, it may be stored immediately in the rich pointer,
Longer than a single index of various sizes
It may be stored in the tail leaf. FIG. 10 illustrates a data structure embodying the present invention.
Support memory storage programs
A schematic diagram of a computer system that can be
is there. Thus, the present invention provides a wide variety of data structures,
Language, operating system, hardware platform
FIG. 15 is applicable to
Includes a suitable platform to support the invention
That is, such a computer system 1000 is shown. The computer system 1000 has a system
A central processing unit 1001 coupled to the CPU100
1 is HP PA-8500 or Intel Pentium processor
The CPU may be a general-purpose CPU as described above. But
However, the CPU 1001 determines here, for example,
As long as the described devised operation is supported, the present invention
Is not bound by the CPU1001 architecture
No. The system bus 1002 has a random access memory (RA
M) coupled to 1003, which can be SRAM, DRAM, or SDRAM
It may be. The ROM 1004 is also connected to the system bus 1002
However, it may be a PROM, EPROM, or EEPROM. RAM
1003 and ROM 1004 store user, system data, and
Make the program known in the industry. The system 1002 also provides input / output (I / O) control.
Card 1005, communication adapter card 1011, user interface
Face card 1008 and display card 1009
Are combined. The I / O card 1005 is
Drive, CD drive, floppy (registered trademark) disk drive
Storage devices such as eves, tape drives, etc.10
Connect 06 to the computer system. Communication card 1011
Connects computer system 1000 to network 1012
Network 1012 is
Network, local / wide area network (LA
N / WAN), Ethernet (registered trademark) network,
May be an Internet network, wired or wireless
It can be. User interface card 10
08 is keyboard 1013, pointing device 1007,
And other user input devices to the computer system 1000
Join. Display card 1009 is controlled by CPU 1001
It drives and controls the display device 1010. The present invention is considered to be the presently preferred embodiment.
Although the present invention has been described in relation to
The scope is not limited to the
Various amendments and equivalents included in the spirit and scope of the request
Understand that it is intended to cover the arrangement of
It is. The present invention includes the following embodiments as examples. (1) To store in computer memory
The data structure is a data processing system.
By the application program executed in the
Accessible, said data structure comprising a root pointer
(101), pointed by the root pointer,
Tree containing a first plurality of nodes arranged in a tree
(102, 105, 106, 112-113) and the second plurality of nodes
Is the number of solid subexpans and the digital tree
Linear branch, selected according to the overall status of the
Node (FIG. 3), bitmap branch node 400,
And extended branch nodes (Fig. 5)
Branch nodes (105,106,113) selected from
Shape leaf node (Figs. 6 and 7) and bitmap tree
8 (FIGS. 8 and 9)
Are selected, each holds multiple indexes,
The number of leaves in the tree and the indexes in the leaves
Only the undecoded index according to the level of
(108, 110, 116-123). (2) The second plurality of nodes are linear
Branch node (Figure 3), bitmap branch node (Figure
4), and (1) including an extended branch node (FIG. 5)
The described data structure. (3) The second plurality of nodes are linear
Fnode (Figs. 6 and 7) and bitmap leaf node
The data structure according to (1), including data (FIGS. 8 and 9). (4) The linear branch node (FIG. 3)
At least a linear list, the first list at least
The corresponding ins of each relevant subexpansion
Including a subexpanse descriptor with dex bits,
The second list contains one or more of each relevant subexpansion.
A pointer to a number of auxiliary nodes, said pointer being the previous
Corresponding to the subexpansion descriptor in the first list.
The data structure according to (1). (5) Each of the pointers is a rich pointer
Data according to (4), including interchanges (FIGS. 2A and 2B)
Construction. (6) The linear branch nodes (105 and
FIG. 3) contains at least two linear lists, the first
The strikes (E1-E4) are each relevant subexp
A sensor that includes at least the corresponding index bit of the
The second list (exp
Rich pointers 1-4 of each instance)
Contains pointers to one or more auxiliary nodes of the pan
And the pointer is the sub-exit of the first list.
The data structure according to (1), which corresponds to a Pance description word. (7) The bitmap branch 113,400
Is a sub-extract under each possible bitmap branch node.
First list of bits including one bit for spans
At least 401, each of said bits corresponding to
Indicates whether the subexpansion has an index,
The second list of pointers 402 contains the subexpansion
Refers to at least one auxiliary node for each,
The pointer indicates the status of the bit in the first list.
Data structure corresponding to (1). (8) The bitmap leaf node (see FIG.
8 and FIG. 9) correspond to each possible index in the leaf.
And at least a first list including one bit,
Each of the bits corresponds to a corresponding one of the indices
The data structure described in (1) indicating whether or not one is valid
Build. (9) Data structure to which index belongs
Identify compressed branch nodes (FIGS. 3 and 4) in (FIG. 1)
Determining the parameters of the data structure.
Responsive to the value,
Selectively convert a node into a decompressed branch node (FIG. 5)
Step and index under said decompression branch node
Storing the parameter, wherein the parameter is
Index value for data structure and said compression branch
Total memory used for each population under the node
Index into a data structure that contains one of the fields
How to remember. (10) The data structure is a data processing system.
Application programs running on stem 1000
Access to the computer memory 1003,1
004, said data structure is stored in the root pointer 101
Are pointed by the root pointer and are arranged hierarchically.
Digital tree 102, 105, 106, 11
2-113, and each of the nodes is
3 and 4 and the extended branch node
Bra selected from the group consisting of (FIG. 5)
And the linear leaf nodes (see FIG. 6).
And Figure 7) and bitmap leaf nodes (Figure 8 and Figure
Leafno selected from the group consisting of 9)
Cards 108, 110, 116-123, and each leaf node has multiple
Holds a number index for leafs in the digital tree
Undecoded according to level and number of indices in leaf
The method according to (9), including only index bits.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1A is an example of a digital tree combining hybrid abstract data type structures (ADTs) according to the present invention to maximize memory utilization while minimizing index access time. FIG. FIG. 1B is a schematic diagram of an example of a digital tree combining hybrid abstract data type structures (ADTs) according to the present invention. FIG. 1C is a schematic diagram of an example of a digital tree combining hybrid abstract data type structures (ADTs) according to the present invention. FIG. 1D is a schematic diagram of an example of a digital tree combining hybrid abstract data type structures (ADTs) according to the present invention. FIG. 1E is a schematic diagram of an example of a digital tree combining hybrid abstract data type structures (ADTs) according to the present invention. FIG. 1F is a schematic diagram of an example combining hybrid abstract data type structures (ADTs) according to the present invention. FIG. 2A is a general schematic of a matching object or “rich pointer”. FIG. 2B is a general schematic of a rich pointer that combines intermediate storage of indexes. FIG. 3 is a schematic diagram of an example of a linear branch. FIG. 4 is a schematic diagram of an example of a bitmap branch. FIG. 5 is a schematic view of an extension branch. FIG. 6 is a schematic diagram illustrating an example of a linear leaf for a structure that refers only to an index. FIG. 7 is a schematic diagram illustrating an example of a linear leaf for a structure having a value associated with each valid index stored in the structure. FIG. 8 is a schematic diagram of a bitmap leaf structure for a structure that refers only to an index. FIG. 9 is a schematic diagram of a bitmap leaf structure including a value associated with each index. FIG. 10 is a block diagram of a computer system in which the subject digital tree is embodied. DESCRIPTION OF SYMBOLS 101 Root pointer 102 Top level node 103, 104, 107, 109 Rich pointer 105 Linear branch 106 Decompression branch 108, 110 Linear leaf node 116, 117, 123 Bitmap leaf 118-122 Linear branch leaf 400 Bitmap branch

   ────────────────────────────────────────────────── ─── Continuation of front page    (72) Inventor Alan Silverstein             United States 80525 Fort Colorado             Collins, Sydney Drive 618

Claims (1)

Claims 1. A data structure for storage in a computer memory, the data structure being accessible by an application program running on a data processing system, the data structure comprising: a root pointer; A digital tree including a first plurality of nodes pointed by the root pointer and arranged in a hierarchy.
Are the branch nodes selected by a group consisting of a linear branch node, a bitmap branch node, and a decompression branch node, which are selected according to the number of solid subexpanses and the overall status of the digital tree. A leaf selected from the group consisting of a linear leaf node and a bitmap leaf node, each holding a plurality of indices and containing only undecoded indices according to the level of leaves in the digital tree and the number of indices in the leaves A data structure containing nodes.