DE112016003598T5 - Simultaneous mass processing of tree-based data structures - Google Patents

Abstract

Computer-implemented methods of concurrently processing operations on a tree-based data structure include: receiving an input to a storage system that manages a storage unit in which the tree-based data structure is stored, the input identifying a set of heterogeneous operations based on the tree-based Data structure to be applied; Determining one or more nodes of the tree-based data structure to which one or more operations of the set of heterogeneous operations are to be applied; and simultaneously and quantitatively performing one or more operations of the set of heterogeneous operations. Systems and methods for carrying them out are also disclosed.

Description

BACKGROUND

The present invention relates to the processing of data structures, and more particularly, this invention relates to efficient simultaneous mass processing of data stored in tree-based data structures, e.g. outsourced search tree data structures, and the processing of the data structures themselves.

Data stored on a conventional storage system is organized according to a data structure of a plurality of known data structures. In most cases, the data is organized according to tree-based data structures, such as paged search trees, that form a forked-branching network of nodes that store data and / or forwarding information to facilitate the search for data of interest. In a paged search tree, each node usually corresponds to a page.

Accordingly, it would be advantageous to provide a system and techniques to improve the effectiveness of changes to tree-based data structures to improve the operation and throughput of a wide variety of data storage systems by providing input / output (I / O) performance and reduce computational costs for storing, organizing, searching for and updating data entries and the corresponding data structures.

SUMMARY

In one embodiment, a method performed by a computer for concurrently processing operations on a tree-based data structure includes: receiving an input to a storage system that manages a storage unit in which the tree-based data structure is stored, the input identifying a set of heterogeneous operations, to be applied to the tree-based data structure; Determining one or more nodes of the tree-based data structure to which one or more operations of the set of heterogeneous operations are to be applied; and simultaneously and quantitatively performing one or more operations of the set of heterogeneous operations.

In another embodiment, a computer program product for simultaneously processing operations on a tree-based data structure includes a computer readable storage medium having program instructions implemented therewith. The computer-readable storage medium is not per se a short-term signal, and the program instructions may be executed by a storage system manager to cause the storage system manager to perform a method. The method includes receiving an input by the storage system manager identifying a set of heterogeneous operations to be applied to the tree-based data structure; determining, by the storage system manager, one or more nodes of the tree-based data structure to which one or more of the operations of the set of heterogeneous operations are to be applied; and simultaneously and quantitatively performing one or more operations of the set of heterogeneous operations by the storage system manager.

In yet another embodiment, a storage system for concurrently processing operations on a tree-based data structure includes a storage system manager and logic that is integrated with the storage system manager and / or that may be executed by the storage system manager. The logic is configured to cause the memory system to: receive an input identifying a set of heterogeneous operations to be applied to the tree-based data structure; determine one or more nodes of the tree-based data structure to which one or more of the operations of the set of heterogeneous operations are to be applied; and perform one or more operations of the set of heterogeneous operations simultaneously and quantity-based.

Other aspects and embodiments of the present invention will become apparent from the following detailed description which, taken in conjunction with the drawings, exemplifies the principles of the invention.

list of figures

• 1 eine Netzwerkarchitektur gemäß einer Ausführungsform darstellt;
• 2 eine repräsentative Hardware-Umgebung gemäß einer Ausführungsform zeigt, zu der die Server und/oder die Clients von 1 gehören können;
• 3 ein mehrschichtiges Datenspeichersystem gemäß einer Ausführungsform darstellt;
• 4 eine vereinfachte schematische Darstellung eines ausgeglichenen, ausgelagerten Suchbaums mit drei Ebenen gemäß einer Ausführungsform ist;
• 5 eine vereinfachte schematische Darstellung eines unausgeglichenen, ausgelagerten Suchbaums mit drei Ebenen gemäß einer Ausführungsform ist;
• 6 eine vereinfachte schematische Darstellung eines unausgeglichenen, ausgelagerten Suchbaums mit mehreren Ebenen gemäß einer Ausführungsform ist;
• 7 eine vereinfachte schematische Darstellung einer baumbasierten Datenstruktur mit n Ebenen und des Verlaufs der Verarbeitung mit Bezug auf die baumbasierte Datenstruktur während einer Suchphase und einer Aktualisierungsphase gemäß einer bevorzugten Ausführungsform ist;
• 8 einen Ablaufplan zeigt, der eine Verarbeitung während einer Suchphase gemäß einer bevorzugten Ausführungsform der aktuell offenbarten erfindungsgemäßen Konzepte darstellt;
• 9 einen Ablaufplan zeigt, der eine Verarbeitung während einer Suchphase gemäß einer bevorzugten Ausführungsform der aktuell offenbarten erfindungsgemäßen Konzepte darstellt;
• 10 ein Ablaufplan eines von einem Computer ausgeführten Verfahrens zur gleichzeitigen Massenverarbeitung von heterogenen Operationen an einer ausgelagerten Suchbaum-Datenstruktur gemäß einer Ausführungsform ist;
• 11 ein Ablaufplan eines von einem Computer ausgeführten Verfahrens zur Isolierung von gleichzeitigen Operationen an einer ausgelagerten Suchbaum-Datenstruktur gemäß einer Ausführungsform ist;
• 12 ein Ablaufplan eines Verfahrens zur effizienten Festlegung der Reihenfolge von Operationen an einer ausgelagerten Suchbaum-Datenstruktur gemäß einer Ausführungsform ist.

Embodiments of the present invention will now be described by way of example only and with reference to the accompanying drawings, in which:

• 1 represents a network architecture according to an embodiment;
• 2 shows a representative hardware environment according to an embodiment to which the servers and / or the clients of 1 can belong;
• 3 a multilayer data storage system according to one embodiment;
• 4 Figure 5 is a simplified schematic illustration of a balanced, paged, three-level search tree according to one embodiment;
• 5 Figure 5 is a simplified schematic illustration of an unbalanced paged search tree of three levels according to one embodiment;
• 6 Figure 5 is a simplified schematic illustration of an unbalanced, multi-level paged search tree according to one embodiment;
• 7 Figure 5 is a simplified schematic illustration of an n-level tree-based data structure and the history of processing with respect to the tree-based data structure during a search phase and an update phase in accordance with a preferred embodiment;
• 8th shows a flowchart illustrating processing during a search phase according to a preferred embodiment of the presently disclosed inventive concepts;
• 9 shows a flowchart illustrating processing during a search phase according to a preferred embodiment of the presently disclosed inventive concepts;
• 10 FIG. 5 is a flowchart of a computer-executed method for concurrently mass processing heterogeneous operations on a paged search tree data structure, according to one embodiment; FIG.
• 11 Fig. 10 is a flowchart of a method performed by a computer for isolating concurrent operations on a paged search tree data structure, according to one embodiment;
• 12 Figure 3 is a flow chart of a method for efficiently determining the order of operations on an offloaded search tree data structure, according to one embodiment.

DETAILED DESCRIPTION

The following description is for the purpose of illustrating the general principles of the present invention, and is not intended to limit the inventive concepts claimed herein. In addition, certain features described herein may be used in conjunction with other described features in each of the various possible combinations and permutations.

Unless expressly defined otherwise, all terms are to be understood in their broadest possible interpretation, including the meanings that result from the specification, as well as meanings that the person skilled in the art understands and / or that are defined in dictionaries, treatises, etc.

It is also to be understood that the singular forms "a," "an," "an," and "the" include plural referents as the specification and the appended claims, unless otherwise specified , Furthermore, it is understood that the terms "comprising" and / or "having" when used in this specification indicate the presence of specified features, integers, steps, operations, elements and / or components, the presence or absence of however, do not preclude the addition of one or more other features, integers, steps, operations, elements, components, and / or groups of the foregoing.

The following description discloses several preferred embodiments of systems, methods, and computer program products for efficiently handling tree-based data structures using quantity-based, concurrent processing of types of heterogeneous operations.

In a general embodiment, a method performed by a computer for concurrently processing operations on a tree-based data structure includes: receiving an input to a storage system that manages a storage unit in which the tree-based data structure is stored, the input identifying a set of heterogeneous operations to be applied to the tree-based data structure; Determining one or more nodes of the tree-based data structure to which one or more operations of the set of heterogeneous operations are to be applied; and simultaneously and quantitatively performing one or more operations of the set of heterogeneous operations.

In another general embodiment, a computer program product for simultaneously processing operations on a tree-based data structure includes a computer-readable storage medium having program instructions implemented therewith. The computer-readable storage medium is not per se a short-term signal, and the program instructions may be executed by a storage system manager to cause the storage system manager to perform a method. The method includes receiving an input from the storage system manager that includes a set of identifies heterogeneous operations to be applied to the tree-based data structure; determining, by the storage system manager, one or more nodes of the tree-based data structure to which one or more of the operations of the set of heterogeneous operations are to be applied; and simultaneously and quantitatively performing one or more operations of the set of heterogeneous operations by the storage system manager.

In yet another general embodiment, a storage system for concurrently processing operations on a tree-based data structure includes a storage system manager and logic that is integrated with the storage system manager and / or that may be executed by the storage system manager. The logic is configured to cause the memory system to: receive an input identifying a set of heterogeneous operations to be applied to the tree-based data structure; determine one or more nodes of the tree-based data structure to which one or more of the operations of the set of heterogeneous operations are to be applied; and perform one or more operations of the set of heterogeneous operations simultaneously and quantity-based.

definitions

Neighboring nodes

According to this document, nodes within a tree-based data structure are adjacent to immediate neighboring nodes of the same level in the tree, all nodes of the same tree level being the same distance to the root node of a tree and neighboring nodes may or may not be descended from the same parent node in the next higher tree level can descend. Neighboring nodes of a given level are also called siblings.

Simultaneous mass processing

As contemplated in this document, concurrent mass processing involves simultaneous processing of multiple operations on a tree-based data structure. Operations are processed simultaneously if their processing occurs during the same time window. Operations are processed quantity-based when multiple operations are processed together (and not individually) during the same time window.

In various embodiments, concurrent mass processing includes simultaneous, independent processing of multiple nodes of a tree-based data structure that belong to the same level of the tree. In particularly preferred embodiments, concurrent mass processing includes parallel processing of all nodes of a particular level of a tree-based data structure using independent operation threads.

Heterogeneous operations

As contemplated in this document, heterogeneous operations include any two or more different types of operations that may be performed to modify or process data stored in a tree-based data structure and / or to organize the tree-based data structure itself to change or process. Exemplary operations include updating, inserting, or removing data entries and / or routing entries stored in nodes of a tree-based data structure, as those skilled in the art will understand after reading the present descriptions.

Thus, a set of heterogeneous operations includes any two or more of the exemplary variety of operations described herein. In preferred embodiments, heterogeneous operations do not concern tree nodes, but data entries. In particularly preferred embodiments, heterogeneous operations do not involve inner nodes, but leaf nodes.

Delete nodes

As stated in this paper, deleting nodes involves removing a node from the tree-based data structure.

In various embodiments, the deletion of nodes following a merge of nodes, e.g. removing the node that was dumped because its contents were taken over into another node in the data structure. The deletion of nodes may include, for example, changing and / or removing forwarding entries in one or more nodes belonging to one or more levels from which the deleted node originated.

Inserting nodes

As suggested in this document, the insertion of nodes involves creating a previously non-existent node within the tree-based data structure.

In various embodiments, the insertion of nodes may be performed in response to the determination that one or more nodes in a particular level are crowded, e.g., by moving data entries from the crowded node to the new inserted node.

Merging nodes

According to the view in this document, a merge of nodes involves the merging of two or more existing nodes into a single node.

In various embodiments, merging may be performed in response to determining that an underfilled node already exists or will be present (e.g., as a result of performing the heterogeneous operation (s) at a particular level of the data structure). A merge of nodes is preferably performed by merging the underfilled node with a sibling node.

node authorization

A user may be granted permission to access a particular node of a tree-based data structure indivisible in relation to all other users performing operations on the same data structure.

The permission can be shared, in which case the node can only be read. A shared read-only permission can be shared by multiple users.

Alternatively, the privilege may be exclusive, in which case the node may also be changed. An exclusive privilege mutually excludes other users accessing the node.

Node Redistribution

Correspondingly, in this document, node reallocation involves the transfer of data entries from a full node to its sibling nodes or to an underfilled node from its sibling nodes to keep the nodes in a required memory usage area.

In various embodiments, node redistribution is particularly useful to maintain adequate memory utilization across the nodes of a tree-based data structure.

Thus, based on these descriptions, those skilled in the art will understand that nodal reallocation may involve a process or node redistribution may be followed by a process whereby, in preferred embodiments, routing information in the tree-based data structure is changed to accommodate any changes to the organization the node and / or the storage space of the data entries.

node Distribution

According to this document, a node-splitting operation involves splitting a node into two or more nodes, at least one of which was not part of the data structure prior to performing the splitting operation.

In various embodiments, the node partitioning may occur in response to the determination that an existing node of a tree-based data structure is crowded or overcrowded, e.g. as a result of performing one or more heterogeneous operations at a particular level of the tree-based data structure, as those skilled in the art will understand after reading the present descriptions.

node update

Refreshing a node as considered in this document involves changing the payload of a data entry (eg, for leaf nodes of a B + tree) and / or changing one or more forwarding entries stored in the node (eg, for inner nodes of a B + tree). ,

In various embodiments, a node update may be performed in response to another operation causing a change in the organization of the data structure that may require an update of forwarding entries to account for the change.

Additionally and / or alternatively, a node may be updated to reflect a change in a make the data entry saved to the node.

memory Specification

As is understood in this document, a memory identifier is a data element that identifies the memory location of a node in a memory device.

In various embodiments, a node in a tree-based data structure is preferably associated with a unique memory identifier. For example, to point from node N1 to node N2 in a tree-based data structure, a node identifier of node N2 is preferably stored in node N1.

Tree-based data structure

As is understood in this document, tree-based data structures include any variety of data structures in which data entries are stored and organized according to a hierarchical branching structure. Preferably, the structure bifurcates, but structures with a higher order division are to be considered within the scope of the present disclosures, e.g. Quad bifurcation, eightfold bifurcation, etc., which a person skilled in the art might understand after having read the present descriptions.

Outsourced search tree data structure

As stated in this paper, paged search tree data structures are designed to organize and search for data. In offloaded search tree data structures, data is stored in nodes, where a node usually corresponds to a disk side, and the nodes are organized and connected in a hierarchical tree structure. These data structures are designed to enable fast and efficient searches in large datasets. Typically, such data structures are stored on disk and caches on faster memories (e.g., main memory or solid state disk) are used to store portions of the data structure to improve performance.

For example, B + trees are a variant of the paged search tree data structure. In a B + tree, data entries are stored exclusively in leaf nodes, while inner nodes store forwarding information to direct operations to the corresponding leaf nodes. Data entries stored in the leaf nodes contain a key (to simplify the search) and a payload containing the data of interest. Routing entries include a key and an internal connection or node identifier by which operations are directed to corresponding leaf nodes.

Computer, network and storage system architecture

The present invention may be a system, a method, and / or a computer program product. The computer program product may include a computer readable storage medium (or storage media) having computer readable program instructions for causing a processor to implement aspects of the present invention.

The computer readable storage medium may be a physically tangible unit that can receive and store instructions for use by an instruction execution unit. The computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A list of more specific examples of the computer readable storage medium that is not exhaustive includes: a portable computer diskette, a hard disk, random access memory (RAM), read only memory (ROM), an erasable programmable only Read only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, a mechanically coded unit such as punched cards or raised structures in a groove with instructions recorded thereon as well as any suitable combination of the foregoing. A computer-readable storage medium, as used herein, should not be construed to be short-term signals per se, such as radio waves or other free-propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (For example, light pulses that are passed through a fiber optic cable), or electrical signals that are transmitted via a cable acts.

Computer-readable program instructions described herein may be downloaded from a storage medium that can be read by a computer to appropriate computing / Processing units or over a network, for example the Internet, a local area network, a wide area network and / or a wireless network, to be downloaded to an external computer or an external storage unit. The network may include copper transmission cables, fiber optics, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing unit receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing / processing unit.

The computer readable program instructions for performing operations of the present invention may be assembly language instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state adjustment data, or either source code or object code as described in U.S. Pat any combination of one or more programming languages including an object-oriented programming language such as Smalltalk, C ++ or the like, as well as in conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partially on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (for example, via the Internet via an Internet Service provider). In some embodiments, electronic circuits including, for example, programmable logic circuits, general purpose custom programmable logic circuits (FPGA), or programmable logic devices (PLA) may execute the computer readable program instructions by using state information of the computer readable program instructions to encode the electronic circuits to personalize aspects of the present invention.

Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It is understood that each block of the representations in the flowcharts and / or the block diagrams as well as combinations of blocks in the representations of the flowcharts and / or the block diagrams of program instructions that can be read by a computer can be realized.

These computer readable program instructions may be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing device to generate a machine so that the instructions executed via the processor of the computer or other programmable data processing device provide means for Execute the functions / operations specified in the block or blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored in a computer readable storage medium that may instruct a computer, programmable computing device, and / or other devices to function in a particular manner such that the computer readable storage medium having instructions stored therein, having an article of manufacture containing instructions that implement aspects of the function / operation indicated in the block or blocks of the flowcharts and / or the block diagrams.

The computer readable program instructions may also be loaded on a computer, other programmable data processing device, or other device to effectuate a series of operations on the computer, other programmable device, or on another device to perform one of To generate a process executed on a computer, so that the instructions that are executed on the computer, another programmable device or another unit, perform the functions / operations specified in the block or the blocks of the flowcharts and / or the block diagrams ,

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, every block in the Schedules or in the block diagrams represent a module, segment or portion of instructions having one or more executable instructions for executing the specified logical function (s). In some alternative embodiments, the functions specified in the block may not occur in the order shown in the figures. For example, depending on the functionality associated with them, two blocks, represented as consecutive blocks, may actually be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order. It will also be noted that each block of the block diagrams and / or the representation in the flowcharts and combinations of blocks in the block diagrams and / or the representation in the flowcharts of systems based on special hardware and the specified functions or operations or combinations can be implemented from instructions of special hardware and computer instructions.

Moreover, a system according to various embodiments may include a processor and logic integrated with the processor and / or executed by the processor, wherein the logic is configured to perform one or more of the process steps recited herein. By "integrated in" is meant that the processor has logic embedded therewith as hardware logic, such as an application specific integrated circuit (ASIC), a custom programmable general purpose circuit (FPGA), and so forth. By "can be executed by the processor" is meant that the logic is hardware logic; Software logic such as firmware, part of an operating system, part of an application program; etc., or a particular combination of hardware and software logic that the processor can access and that is configured to cause the processor to perform certain functionality upon execution by the processor. Software logic may be stored on a local and / or a remote main memory of any major memory type, as known in the art. Any processor known in the art may be used, such as a software processor module and / or a hardware processor such as an ASIC, an FPGA, a central processing unit (CPU), an integrated circuit (IC), a graphics processing unit (GPU) etc.

1 represents an architecture 100 according to one embodiment. As in 1 is shown is a variety of remote networks 102 provided to the first remote network 104 and a second remote network 106 belong. A gateway computer 101 can be between the distant networks 102 and a nearby network 108 be connected. In the context of the present architecture 100 can any of the networks 104 . 106 take any form, including but not limited to, a LAN, a WAN such as the Internet, a public switched telephone network (PSTN), an internal telephone network, etc.

During operation, the gateway computer is used 101 as an entry point from the remote networks 102 in the near network 108 , Thus, the gateway computer 101 the function of a router capable of forwarding a particular packet of data to the gateway computer 101 arrives, and a switch that takes the actual path in and out of the gateway machine 101 for a particular package.

Moreover, there is at least one data server 114 that is connected to the near network 108 and to the remote networks 102 can access via the gateway computer 101. It should be noted that the data server (s) 114 can belong to any type of data processing unit / groupware. With every data server 114 a plurality of user units 116 are connected. The user units 116 can also directly through one of the networks 104 . 106 . 108 be connected. To these user units 116 may include a desktop computer, a laptop computer, a portable computer, a printer and / or any other type of logic. It should be noted that a user unit 111 in one embodiment, may also be connected directly to any of the networks.

A peripheral device 120 or a set of peripherals 120 For example, facsimile machines, printers, networked and / or local storage devices or systems, etc. may communicate with one or more of the networks 104 . 106 . 108 be connected. It should be noted that databases and / or additional components with any type of network element associated with the networks 104 . 106 . 108 is connected, used or integrated into it. In the context of the present description, a network element may refer to any component of a network.

In accordance with some approaches, methods and systems described herein may be implemented with and / or on virtual systems and / or systems that emulate one or more other systems, such as a UNIX system that emulates an IBM z / OS environment, a UNIX system that virtually hosts a MICROSOFT WINDOWS environment, a MICROSOFT WINDOWS system that emulates an IBM z / OS environment, and so on. This virtualization and / or emulation can be in some embodiments, by the use of VMWare software.

In further approaches, one or more networks may 104 . 106 . 108 represent a system composite commonly referred to as a "cloud". In cloud computing, shared resources such as processing power, peripherals, software, data, servers, etc. are provided to any system in the cloud as needed, allowing access to services and the distribution of services across many computing systems. Cloud computing typically includes an Internet connection between the systems that operate in the cloud, but other methods of connecting the systems may also be used.

2 shows a representative hardware environment according to an embodiment to which a user unit 116 and / or a server 114 from 1 belong. This figure illustrates a typical hardware configuration of a workstation that has a central processing unit 210 , such as a microprocessor, and several other units which have a system bus 212 are interconnected.

In the 2 shown workstation contains a random access memory (RAM) 214 , a read-only memory (ROM) 216 , an I / O adapter 218, to peripherals such as disk storage devices 220 by bus 212 to connect a user interface adapter 222 to a keyboard 224 , a mouse 226 , a speaker 228 , a microphone 232 and / or other user interface units such as a touch screen and a digital camera (not shown) with the bus 212 to connect a communication adapter 234 to the workstation with a transmission network 235 (eg with a data processing network) and a display adapter 236 to the bus 212 with a screen 238 connect to.

The workstation may include an operating system such as Microsoft's Windows® operating system, a MAC operating system, a UNIX operating system, and so on. It will be appreciated that a preferred embodiment may be practiced on platforms other than the named platforms and operating systems. A preferred embodiment may be written using the language XML, C and / or C ++ or other programming languages in conjunction with an object-oriented programming approach. Object Oriented Programming (OOP), which is increasingly being used to develop complex applications, can be used.

Now referring to 3 In one embodiment, it is a memory system 300 shown. Note that some of the in 3 shown elements according to various embodiments may be implemented as hardware and / or software. The storage system 300 can be a storage system manager 312 included with a variety of disks on at least one higher storage tier 302 and at least one upper storage animal 306 Exchange data. The or the higher storage animal 302 may preferably have one or more random access media and / or random access media 304 such as hard disk drives (HDDs), nonvolatile memories (NVMs), solid state drives in solid state drives (SSDs), flash memories, SSD arrays, flash memory arrays, etc., and / or contain other media mentioned here or known in the art. The or the upper storage animal 306 may preferably have one or more upper executing storage media 308 , including sequential access media such as magnetic tape in tape drives and / or optical media, HDDs, SSDs, etc., and / or other media referred to herein or known in the art. One or more additional storage animals 316 can contain any combination of main storage volumes, such as from a developer of the system 300 desired. Also, any of the higher storage animals 302 and / or the upper storage animal 306 contain a specific combination of storage units and / or storage media.

The storage system manager 312 can with the storage media 304 . 308 on the higher storage animal or the higher storage animal 302 and the upper storage animal or the upper storage animal 306 over a network 310 , such as a storage area network (SAN) that resides in 3 is shown, or another type of suitable network exchange data The storage system manager 312 can also communicate with one or more host systems (not shown) via a host interface 314, which is part of the storage system manager 312 may or may not be, exchange data. The storage system manager 312 and / or any other component of the storage system 300 may be embodied in hardware and / or software and use a processor (not shown) to execute instructions of a kind known in the art, such as a central processing unit (CPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc. Of course, any arrangement of a memory system may be used as well will be apparent to those skilled in the art after having read the present specification.

In further embodiments, the storage system 300 contain any number of data stores and contain the same or other main storage volumes in each store. For example, each data storage device may include the same type of main storage media, such as HDDs, SSDs, sequential access media (tape in tape drives, optical disk in optical disk drives, etc.) random access media (CD-ROM, DVD-ROM, etc.) or any one Combination of types of disk storage. In such a configuration, a higher storage animal 302 contain a majority of SSD storage media for storing data in a higher executing storage environment, and the remaining storage animals, including the upper storage tier 306 , as well as additional storage animals 316 can contain any combination of SSDs, HDDs, tape drives, etc., to store data in an upper executing storage environment. In this way, data that is accessed more frequently, data that has a higher priority, data that requires faster access, etc., may be in the higher memory tier 302 while data that does not have any of these attributes is stored in the extra memory tiers 316 including the upper storage animal 306 get saved. Of course, having read the present specifications, those skilled in the art may devise many other combinations of types of storage media for translation into various storage schemes in accordance with the embodiments presented herein.

According to some embodiments, the storage system (such as the storage system 300 ) Logic configured to receive a request to open a record, logic configured to determine if the requested record is in an upper memory tier 306 a multi-layered data storage system 300 stored in several associated parts, logic that is configured to move any associated portion of the requested record into a higher memory tier 302 the multilayer data storage system 300 shifts, and logic that is configured to place the requested record on the higher storage tier 302 the multilayer data storage system 300 composed of the associated parts.

Of course, according to various embodiments, this logic may be implemented as a method on any unit and / or on any system or as a computer program product.

Exemplary tree-based data structures

In general, the presently disclosed embodiments of the present invention relate to the processing and updating of tree-based data structures with practical application to paged search trees, which are typically used effectively to facilitate the organization of data in a storage system. The discussion presented here concerns the exemplary case of a paged search tree (eg, a "B + tree") in which all data entries are stored in end nodes of the data structure (ie, "leaf nodes"), while all inner nodes (ie, "branch nodes" and "root nodes ") Store forwarding information that is configured to facilitate traversal of the tree during a search for particular data stored therein.

However, it should be understood that the scope of the present disclosures is not limited to B + trees, B trees, or any other particular variant of a tree-based data structure. Rather, the presently disclosed inventive concepts can be applied to any suitable tree-based data structure that a person skilled in the art would find advantageous after reading the present descriptions.

An exemplary embodiment of a tree-based data structure 400 is in 4 shown graphically. In this embodiment, tree 400 is a balanced three-level data structure. The topmost level of the tree (root level 410a ) contains a root node 402 from which all other nodes in the tree descend. The first node level 410b of the tree 400 that are directly under the root node 402 contains two inner nodes 404 that from the root node 402 branch forked. In embodiments in which the tree 400 A B + tree contains these inner nodes 404 Forwarding information, but no data entries. In an embodiment with a B + tree of 4 Instead, the data entries are exclusively in the leaf nodes 406 stored in the second node level 410c located just below the first node level, as in 4 is shown. Of course, the expert understands that the tree 400 in other embodiments may not be a B + tree, but may be another suitable form of tree-based data structure and data entries in inner nodes 404 as well as in leaf nodes 406 can save.

A similar tree-based data structure 500 is in 5 shown, according to the illustrated embodiment, the tree 500 however unbalanced. As in 4 points the tree 500 a root plane 510a that have a root node 502 contains, a first node level 510b below the root node level and a second node level 510c below the first node level. However, as in 5 shown is the tree 500 asymmetrical and contains an inner node 504 in the first node level 510b but it also contains leaf nodes 506 in several node levels 510b . 510c of the tree 500 , In practice and in accordance with various embodiments, the presently disclosed embodiments of the invention are equally applicable to symmetric as well as asymmetric tree species.

Still referring to the 4 and 5 and according to the opinion in this document contains the first node level 410b . 510b of every tree 400 . 500 two horizontally adjacent siblings (see node 404 , as in 4 is shown; the knots 504 and 506 as in 5 is shown). Since these siblings are each directly from the corresponding root node 402 . 502 They are also vertically adjacent to their respective root nodes.

However, in some embodiments, siblings do not necessarily derive from the same ancestor node. Like in 4 is shown contains the second node level 410c four leaf nodes 406 which can be considered as leaf nodes containing three sets of sibling nodes: the two extreme left sibling nodes are from the far left inner node 404 in the first node level 410b from. The two outermost right siblings are from the outermost right inner node 404 in the first node level 410b from. Each of the middle two siblings originates from another parent node, one from the far left inner node 404 in the first node level 410b and the other from the far right inner node 404 in the first node plane 410b. In various embodiments, all such horizontally and immediately adjacent pairs of nodes in a particular node level are to be considered as siblings.

Likewise and as based on 6 4, the presently disclosed embodiments of the present invention are applicable to and useful in connection with much more complex tree structures, eg, an n-level asymmetric tree, such as the tree-based data structure 600. According to the in 6 The structure shown has the tree 600 in one embodiment, a root node 602 in a root node plane 610a and two inner nodes 604 in a first node level 610b located just below the root node level 610a located.

The next node level, which in some embodiments is a second node level 610d can be referred to, is located below the first node level and can be both one or more inner nodes 604 as well as one or more leaf nodes 606 contain. In various embodiments, the tree may 600 with n levels any number of intermediate additional levels (not shown), each level having inner nodes 604 and / or leaf nodes 606 has, which is between the first node level 610b and the second node level 610d and / or between the third node level 610e and the end node level 610n are arranged as in 6 is indicated by the dashed lines representing the branches between the first node level 610b and the second node level 610d and between the third node level 610e and the end node level 610n separate.

In preferred approaches, the currently disclosed algorithms are implemented in memory systems and / or architectures having optimal minimum and maximum fan-out characteristics, ie, the tree-based data structures realized by the memory system and / or architecture are balanced and optimize the number of Nodes per level of the tree to optimize the number of levels in the tree. Moreover, the algorithms are applied in preferred approaches to B + trees that have such fan-out characteristics.

Advantageously, the use of data structures having fan-out features as described above usually allows for volume-based concurrent processing of heterogeneous operations on data entries as well as corresponding changes to the tree structure in three or fewer iterations of an update phase will be described in more detail below. This highly efficient process further enhances the operation of the memory systems that implement the presently disclosed techniques by reducing the computational cost and execution time of manipulating data and manipulating the tree-based data structures themselves according to multiple approaches.

Simultaneous mass processing of tree-based data structures

As mentioned below, conventional techniques for processing data and tree-based data structures such as paged search trees must address the problem of efficient simultaneous processing of data stored in the tree / tree. A unique approach presented here suggests the use of simultaneous mass processing of types of heterogeneous operations to reduce the I / O and processor costs associated with typical techniques for processing data stored in the tree / tree and processing the data Reduce tree structure. In short, since the presently disclosed techniques of the present invention enable the quantity-based processing of types of heterogeneous operations, detect necessary minimum structural changes to the tree, and pass these necessary minimum structural changes up the tree in the process of changing the data and / or tree structure, can achieve a significant improvement in efficiency.

The presently disclosed inventive concepts provide techniques for performing heterogeneous operations on data entries in a data structure, such as a paged search tree, as well as making necessary changes to the organization of the data structure resulting from the performance of the heterogeneous operations. The presently disclosed inventive concepts are unique in that the processing of heterogeneous operations is performed concurrently and quantity-based for each level of the tree data structure, which is the number and cost of the processing and I / O operations necessary to set the set of heterogeneous ones To perform operations significantly reduced.

In general, the techniques described herein perform the aforementioned simultaneous mass processing using a two-phase approach. First, a search phase traverses the tree data structure and locates the leaf nodes to which one or more operations of an input set of heterogeneous operations are to be applied. Second, an update phase goes through the tree data structure plane-wise and iteratively, proceeding from the leaf level to the root.

An implementation of one of the currently disclosed inventive concepts is for example schematically in 7 where a n-level tree-based data structure has multiple levels 710a . 710b . 710c ... contains 710n. The root level 710a has a root node 702 , of which four inner nodes 704 descend, which is another node level 710b form. Likewise come from the knots 704 This node level from each four nodes, which still another node level 710c form. The tree continues to grow in this way until it reaches the nth level 710n Achieved a variety of leaf nodes 706 each of which stores one or more data entries. Any number of levels can be between the root level 710a and the leaf node plane 710n without departing from the scope of the presently disclosed inventive concepts.

In essence, the processing performed according to the present disclosure takes place in two main phases. The search phase starts at the root level 710a and proceeds step by step to the leaf nodes 706 in the nth level 710n continued.

Conversely, the update phase begins at the leaf node level 710n in a first iteration (i = 0) and progresses through one or more iterations (i≥1) in the upward direction, possibly all the way to the root node 702 in an nth iteration (i = n). Of course, in various embodiments, the update phase may end before the root node level 710a achieved, eg, in response to determining that no nodes are specified in a set of nodes and operations generated during the immediately preceding iteration of the update phase, eg, during iterations i = (n-1), i = (n -2), 2 <i <(n-2), i> 1 etc. as in 7 is shown.

In preferred embodiments, during each iteration of the update phase, one level of the data structure is processed, and the corresponding operations are performed concurrently and quantity based on all corresponding nodes in that level. The update phase deals with nodes that become full or underfull as a result of the application of the heterogeneous operations. The output of each update phase consists of a set of nodes in the next higher level of the tree as well as in operations applied to that node, these operations being the result of the operations applied to the nodes in the current level of the tree. The operations generated during an update phase to be applied to the node of the next higher level of the tree support the new form of the nodes created in the processing of the current level of the tree, and have the contents of their entries updated. The output node-and-operation set of the current update phase is then provided as input to the next update phase, namely, to the next higher level of the tree. The processing of the algorithm is completed when the set of nodes and operations is empty, which was generated by the processing of a level of the tree.

Consequently and with reference to 10 is a flowchart of a procedure 1000 according to one embodiment. The procedure 1000 can be carried out according to the present invention in various approaches in any of the environments described inter alia in the 1 to 9 are shown. Of course, more or less surgeries can be done than those in 10 are expressly described in the process 1000 what the skilled artisan should understand after reading the present specifications.

Each of the steps of the procedure 1000 may be performed by any suitable component of the operating environment. For example, the procedure 1000 in various embodiments may be performed in whole or in part by a disk manager of a hierarchical storage system or other device in which one or more processors reside. The processor, eg, one or more processing circuits, one or more chips, and / or one or more modules embodied in hardware and / or software, and preferably having at least one hardware component, may be used in any unit to provide one or several steps of the procedure 1000 perform. Illustrative processors include, but are not limited to, a central processing unit (CPU), an application specific integrated circuit (ASIC), a custom programmable general purpose circuit (FPGA), etc., combinations thereof, or any other suitable ones in which Technology known data processing unit.

The procedure 1000 is for concurrently processing operations on a tree-based data structure, such as any of the tree-based data structures that exist in the 4 to 6 Configured among other similar tree-based structures that would be appreciated by one skilled in the art after having read the present descriptions. The root node level and leaf node level may be separated by any number of intermediate levels. The tree-based structure may correspond to the definitions of a B + tree, a B-tree, or any other type of tree-based structure.

Regardless of the details of the tree-based data structure, and as in 10 is shown, includes the method 1000 the operation 1002 in which an input is received from a storage system in which the tree-based data structure is stored. The input identifies a set of heterogeneous operations to apply to the tree-based data structure.

Preferably, the input includes a reference to the tree-based data structure and a list of pairs, each pair indicating a data entry and an associated operation. Each data entry may include a key and a payload in which data is stored.

In various approaches, the set of heterogeneous operations may include one or more of the following: insert operations, delete operations, and / or update operations. An insert operation into a node may cause the node to overflow and therefore require partitioning of the node, which in turn may require changes to be applied to entries in the tree planes above the leaves. A delete operation from one node may cause the node to become underfilled, and thus it may become necessary to merge the node with another node, which in turn may also require changes to be applied to entries in the tree levels above the leaves , To accommodate changes in the organization of the data structure, such changes may include adding, updating, or removing routing entries, and may be passed up to the root node of the tree. The updating of a node preferably comprises updating the entries stored therein. Of course, the above-mentioned examples are not to be considered exhaustive, and heterogeneous operations may include any suitable type of operation which the skilled artisan may understand after having read the present specifications.

In one embodiment, decisions as to whether and how the data structure is to be changed are preferably based on node balancing criteria. Node balancing criteria may be predetermined based on the desired system memory utilization, performance, and so on. For example, in one embodiment, node balancing criteria may include a predetermined minimum, an average, etc., a memory utilization for each page (node) of the storage device to which the tree corresponds.

In preferred approaches, predefined node balancing criteria include a minimum capacity threshold of approximately 50% so that a node (page) can be considered "underfilled" if less than 50% of the page capacity is consumed. Node matching criteria may also include a predetermined average capacity threshold of approximately 75%.

Of course, node balancing criteria may also be determined instead of predetermined, and may be changed during processing based on properties of the storage system in which the tree-based data structure and techniques currently disclosed are implemented.

The storage system may be configured to determine the occurrence of one or more of the following example events or equivalent thereof, which the skilled artisan will understand after reading the present disclosure, and take appropriate action by establishing new optimal node balancing criteria which are suitable for the changed circumstances.

Additionally and / or alternatively, node balancing criteria may be specified or specified by a user.

The procedure 1000 also includes the surgery 1004 in which nodes, and preferably leaf nodes to which one or more operations of the set of heterogeneous operations are to be applied are detected. This finding is based, at least in part, on that in the operation 1002 received input. Better still, the operation involves 1004 based on the list of pairs indicating data entries and operations associated with the data entries, determining which nodes of the data structure require processing with one or more of the heterogeneous operations.

In particularly advantageous approaches that in the operation 1004 be carried out determination as a search phase. For example, and with reference to a tree-based data structure, such as in 6 In one embodiment, the search phase may include evaluating each level of the tree-based data structure in a descending order. For each level, the search phase may include concurrently determining, in whole or in part, the data entries specified in the list of pairs, whether any of the nodes in the level being evaluated is a node, one or more operations of the set of heterogeneous operations are to be applied to or on the offspring node thereof. The determination may in some approaches exclude access to the nodes storing data entries (ie, leaf nodes) and instead rely on routing information stored in internal nodes.

For example, and in accordance with an example approach, the search phase may include descending the tree at the same time for each pair entered and retrieving a memory identifier of the leaf node in the data structure to which the operation specified by the input pair is to be applied. The determination that an operation or set of heterogeneous operations is to be applied to a particular leaf node may include comparing data entries stored in the node to a data entry specified by the input pair and to the one or more Include operations.

Preferably, the search is performed concurrently for all input pairs, using shared permissions to access the nodes. The search phase may use caching in order to further minimize memory accesses and improve the function of the corresponding memory system.

In additional and / or alternative approaches, the search phase may be performed in subsets rather than individually for each pair entered. Namely, the inputted pairs may be divided into subscripts at the root node corresponding to the nodes in the next lower level to be accessed, and the access to each of these nodes may be performed by another operation thread processing the subset concerned, whereby the descent will continue. In this way, the presently disclosed inventive concepts allow for parallelization of the search phase across multiple operational processing threads, significantly reducing the time required to complete the search phase.

Regardless of whether this is done individually for each pair entered or in sets, in preferred embodiments, the leaf nodes of the data structure storing data entries are not directly accessed in the search phase but only in the update phase.

The output of the search phase is a "set consisting of nodes and operations" that generally identifies the nodes identified in the search phase to which one or more operations of the set of heterogeneous operations are to be applied, and which preferably contains a list of memory identifiers Contains detected nodes to which one or more operations of the set of heterogeneous operations are to be applied.

In various embodiments, in the set of nodes and operations, each of the memory identifiers of nodes in the list of memory identifiers is a list of Assigned to pairs consisting of an input data entry and an operation (ie insert, delete, update), this list should be applied to the node identified by the associated memory identifier.

In further approaches, the search phase can largely take place according to a process flow that is in 8th is shown.

In different approaches, the operation can 1004 of the procedure 1000 thus, involve generating an output having a set of nodes and operations. The set of nodes and operations contains a list of memory tags and each memory tag identifies one of the nodes to which one or more operations of the set of heterogeneous operations are to be applied. Moreover, in some approaches, each of the memory identifiers is associated with at least one pair of data entry and operation identified in the input that is in the operation 1002 Will be received.

The procedure 1000 further includes an operation according to an embodiment 1006 in which the one or more operations of the set of heterogeneous operations are performed simultaneously and quantity-based. The tree data structure is preferably processed iteratively and level-by-level, with all nodes identified in the set of nodes and operations belonging to a particular level being processed concurrently and quantity-based. This processing will be described in more detail below in the context of the inventive update phase disclosed herein.

In preferred embodiments, the application of the set of heterogeneous operations to the various nodes of the tree data structure is performed over one or more "update phase" iterations involving a planar, simultaneous mass update of the tree-based data structure. The update phase can begin, eg for a tree-based data structure as described in the 4 - 6 by performing one or more operations of the set of heterogeneous operations on one or more nodes in a lowest level of the tree-based data structure (eg, the second node level 410c . 510c as in the 4 5, or the nth level 610n , as in 6 shown) where these leaf nodes have a memory identifier listed in the set of nodes and operations.

Thus, at each of the nodes in the second node level (ie, the level above the previously processed level) having a memory identifier listed in the set of nodes and operations generated by the previous level processing, the set may heterogeneous operations applied to the node on which the data entry / operation pair (s) belong to the memory identifier of that node in the set of nodes and operations.

In one approach, above all, the set of heterogeneous operations is applied simultaneously and quantity-based to each of the nodes in the second node level that has a memory identifier listed in the set of nodes and operations. Preferably, the concurrent mass processing involves isolating each node that is being processed at the current node level in a manner that enables evaluation and qualification of neighboring nodes that are eligible for involvement in node balancing operations due to the application of the heterogeneous operations the nodes in the current level are required.

In various embodiments, applying the heterogeneous operation (s) to the node (s) at a particular node level may include inserting new nodes into one or more levels of the tree; deleting existing nodes in one or more levels of the tree; merging existing nodes in one or more levels of the tree; splitting one or more existing nodes into one or more levels of the tree; and / or updating one or more data entries of existing nodes in one or more levels of the tree.

If the application of the heterogeneous operation (s) at a level above the leaf node level involves the insertion of an entry, the insert operation is preferably performed in response to the determination that one or more existing nodes have been split in at least one of the levels below the current level. Moreover, the insert operation may require the creation of one or more nodes in the current level of the tree-based data structure.

If the application of the heterogeneous operation (s) at a level above the leaf node level involves the deletion of an entry, the delete operation is preferably performed in response to the determination that one or more existing nodes in at least one level of the tree-based data structure are below the current one Level were merged. The delete operation may also require the deletion of one or more nodes in the current level of the tree-based data structure.

The entries and operations generated during the processing of the current level of the tree support the new shape of the nodes that were updated, created, and deleted when processing the current level of the tree. Three exemplary cases that produce such entries and operations are shown immediately below.

First, in one embodiment, new nodes that are created as processed nodes are split and create entries to be added to the next higher level of the tree.

Second, in another embodiment, the complete transmission of entries (eg, via a merge) from a processed node to sibling nodes causes the processed nodes to be deleted, and it creates an erase operation of an entry on that deleted node at the next higher level of the tree.

Third, in further embodiments, a processed node in which the contents of its entries have been changed and whose representative entry has changed thereby generates entries to be deleted and inserted into the next higher level of the tree.

Of course, in various embodiments, one or more of the exemplary cases in other cases, which one of ordinary skill in the art will understand after reading the present descriptions, may occur in a single iteration (or multiple iterations) of processing the tree-based data structure.

In general, the update phase preferably generates an output that identifies a set of operations to be applied during a next iteration, which preferably correspond to nodes of another level of the tree-based data structure, more preferably a level above the level that during the current iteration Update phase, and preferably a layer that is directly above the level being processed during the current iteration of the update phase.

In some embodiments, the update phase generates as output a new set of nodes and operations based at least in part on applying the set of heterogeneous operations to the node level being processed during the update phase. The new set of nodes and operations contains a list of memory identifiers. Each memory identifier in the new set of nodes and operations identifies one of the nodes in the next node level to be updated. The nodes in the next node level identified in the new set of nodes and operations are nodes to which one or more operations of the set of heterogeneous operations are to be applied. Preferably, each memory identifier of the memory identifiers in the new set of nodes and operations is associated with at least one pair of data entry and operation.

The update phase may include any number of iterations, wherein the tree-based data structure is updated step-by-step level-by-level (bottom-up) from the end nodes of the tree-based data structure to the root node.

In various embodiments, an iterative update may begin at a lowest level of the tree-based data structure and progressively update each level until the level containing the root node is reached. Alternatively, the update phase may progressively update only a subset of the levels of the tree-based data structure.

In other approaches, the iterative update operation may also result in creating new levels of the tree structure, e.g., to account for node balancing operations and / or criteria. Node balancing operations may require the creation of new levels, e.g. in response to node balancing causing the distribution of a root node, requiring a new root node in a plane above the previous root node. Node balancing criteria, such as node utilization thresholds, may also require the creation of new nodes, e.g., by requiring the partitioning of a root node, etc.

Thus, the set of nodes and operations generated by an iteration of the update phase may serve as input to the next iteration of the update phase. Preferably, this new set of nodes and operations identifies a set of nodes in a level above the level that is being updated in the current iteration, where the designated nodes are those nodes to which one or more operations of a node Set of heterogeneous operations to be applied in the next iteration of the update phase.

In further approaches, one or more iterations of the update phase may also include determining whether the input set consisting of nodes or operations is empty, and if so, completing the processing of the tree-based data structure, since there are no further changes to the upper levels the tree-based data structure. In some embodiments, the completion of the processing may also include the release of a permission on the root node, if such permission was previously obtained on the root node. In other embodiments, such authorization may be exclusive.

If the input set consisting of nodes and operations is not empty, the techniques discussed herein may generate a new empty set of nodes and operations; filling the empty set of nodes and operations with the memory identifiers of the nodes to which one or more operations of a set of heterogeneous operations are to be applied in the next iteration of the update phase; and associating the memory identifiers with a corresponding set of one or more heterogeneous operations to be performed on the corresponding node in the next iteration of the update phase.

In one embodiment, iterations of the update phase may include the application of one or more operations of the set of heterogeneous operations to each node at a particular node level having a memory identifier listed in the new set of nodes and operations during a previous one Iteration of the update phase for a subsequent (i≥1) iteration of the update phase or during the search phase for the first (i = 0) iteration of the update phase. If the nodes in the particular node level having memory tags listed in the new set of nodes and operations contain leaf nodes, the heterogeneous operation (s) applied to the leaf nodes are preferably based at least one pair of data entry and operation associated with the memory identifier for the leaf node and provided as input to the memory system. Moreover, the heterogeneous operation (s) will be applied simultaneously and quantity based on each of the nodes in the particular node level having a memory identifier listed in the new set of nodes and operations.

Referring to 4 and in accordance with preferred embodiments, this iterative process of the update phase includes updating one or more nodes 406 in the first node level 410c during the first iteration (i = 0), generating a new set of nodes and operations that includes one or more nodes 404 in the second node level 410b to which one or more heterogeneous operations are to be applied, and updating the one or more nodes 404 the second node level 410b in the second iteration (i = 1) of the update phase.

In particularly preferred embodiments, the nodes 406 in the first node level 410c Leaf nodes that store data entries and belong to a B + tree data structure. The knots 404 in the second node level 410b are inner nodes that store forwarding information.

Of course, additional embodiments that are within the scope of the present disclosure may include tree-based data structures in which nodes of multiple levels store data entries and / or forwarding information, where nodes store data entries at multiple levels of the tree, etc., such as generally in US Pat 5 and 6 is shown. In particular, according to these embodiments, the iterative update phase may include updating data entries for nodes at any level of the tree-based data structure, updating routing information for nodes at any level of the tree-based data structure, changing relationships between nodes at any level of the tree-based data structure, transmitting information between nodes at any level of the tree-based data structure, etc., as those skilled in the art will understand after reading the present descriptions.

Regardless of the particular tree structure, according to various embodiments, the update phase may include determining whether the original and / or new node-and-operation set lists memory identifiers corresponding to a node in a next higher level of the tree. In response to determining that the set of nodes and operations does not contain memory identifiers that correspond to a node in the next higher one Level of the tree, the update phase is preferably terminated.

In particularly preferred approaches, the access to nodes in the search phase takes place by means of a shared authorization, and in the update phase the access to nodes takes place by means of an exclusive authorization.

The application of heterogeneous operations to nodes at a particular level occurs concurrently, and quantity-based, which involves applying operations to each node using a different process thread. Even better, each process thread is executed in parallel to reduce computation time and to improve the memory system that implements the currently disclosed techniques by speeding up the process of finding and updating data in a tree-based data structure while at the same time causing less computational cost.

For example, by processing heterogeneous operations on a volume-based basis and accommodating necessary node balancing or other manipulations of the tree in a single, iterative process, the techniques currently disclosed eliminate the need to apply a particular type of operation across trees, then perform another search to obtain data entries for find another type of operation to determine if any of the data entries that were the target of the other type of operation are now in a different location than before the cross-tree application of the particular type of operation.

The processing of nodes during the update phase may, in various embodiments, be performed in any suitable manner that permits concurrent mass processing of heterogeneous operations across multiple nodes at a particular level of the tree-based data structure. In a particularly preferred embodiment, the processing during the update phase largely takes place as shown in FIG.

In preferred embodiments, the operations associated with the node in the input set of nodes and operations are performed in a particular order that makes the presently disclosed inventive concepts even more efficient and further improves the operation of the memory system itself. The order of operations will be discussed in more detail below.

During a first iteration of an update phase that can process a level containing leaf nodes, operations associated with a node in the input set of nodes and operations generated by the search phase may be performed according to a user-provided order of operations or processed according to the preferred order given here. The preferred order given herein preferably minimizes the structural changes that result from the application of operations to a particular node and therefore contributes to the efficiency of the algorithm and further improves the operation of the memory system in which the algorithm is executed. Regardless of the particular order of application, the update phase may begin by obtaining exclusive access to the root node.

The preferred order involves performing update operations, followed by delete operations and subsequent insert operations.

More specifically, the first operation to be performed is preferably to update the payload of data entries in the node. The second operation is to delete entries from the node. At this point, no further node equalization or merge operations need to be performed until processing of all input operations is completed. In fact, preferably, no equalization or merge operations are performed unless the node being processed is underfilled due to all input operations being applied. The third operation is to insert entries in the nodes. In the case of insertion, a node may become full before all input operations are applied, and thus, in these situations, it may be necessary for node redistribution or node partitioning to be used during the insert operations if the node becomes full.

In various embodiments, a node that becomes full may be determined by comparing a node utilization level (i.e., an amount of data stored in the node) with a maximum capacity of the node.

Similarly, a node that is underfilled can be determined by comparing the node utilization level to a minimum node utilization threshold. The minimum node utilization threshold may be predetermined according to user preference or according to specific requirements of the storage system, eg, memory utilization and / or performance requirements. In preferred Approaches, the minimum node utilization threshold is approximately 50% of the total node capacity. Of course, other values below 100%, eg, 25%, 30%, 35%, etc., may be used without departing from the scope of the present disclosures.

To counter situations in which a node becomes full, a node redistribution can be used. This node redistribution preferably includes determining whether the full node has adjacent nodes, preferably adjacent nodes that were not identified in the set of nodes and operations provided as input to the current iteration. In response to determining that such an adjacent node is present, the node reallocation involves determining whether transmitting entries from the full node results in the adjacent node itself becoming full. In response to determining that the adjacent node is not getting full, node reallocation involves determining whether the further insert operations pending in the input set of nodes and operations are due to the transfer of entries (all insert operations must be performed on the nodes applied in the input set consisting of nodes and operations, and not to the neighboring nodes) need not be applied to the neighboring nodes. In response to determining that the further insert operations do not have to be applied to the neighboring nodes, a node reallocation involves determining whether the adjacent node will not become full due to the transfer of entries. In response to determining that the adjacent node is not becoming full due to the transmission of entries, node reallocation involves determining whether the transmission of entries to the adjacent node results in the node node full node utilization within a specified node utilization range (FIG. eg above the minimum node utilization thresholds). In response to the determination that the transmission will achieve this result, the entries are preferably transmitted. More preferably, multiple entries may be transferred to create capacity for the additional pending insert operations in the node being processed.

Even better, the nodes to which the entries are transferred are only blocked for the duration of the transmission (e.g., by exclusive access). This node redistribution is best performed after all insert operations have been applied to nodes in the layer being processed and no insert operations have been applied to the neighboring nodes that receive the transmitted entries.

In further embodiments, node redistribution to relieve a full node may involve splitting the node, for example, if the above transmission procedure is not possible because any of the above-noted findings have a negative detection content.

In particular, the application of a set of heterogeneous operations to nodes of a particular level of the tree-based data structure may, under certain circumstances, result in multiple occurrences of a node being processed becoming full during that iteration of the update phase, and thus it may be desirable to perform a node redistribution multiple times per iteration of the update phase.

For nodes that are underfilled during an iteration of the update phase, node redistribution may be performed analogously thereto according to several embodiments. For example, in one embodiment, node redistribution, to address the problem of underfilled nodes, concludes whether the underfilled node has adjacent nodes that were not identified in the set of nodes and operations, which is input to the current iteration provided.

In other embodiments, and in response to determining that such an adjacent node exists, the node reallocation includes determining whether transmitting entries from the adjacent node results in the adjacent node itself being underfilled.

In still further embodiments, and in response to determining that the adjacent node is not underfilled, node reallocation includes determining whether the transmission of entries to the underfilled node results in the node utilization level of the underfilled node within a specified node utilization range (Fig. eg above the minimum node utilization thresholds). In response to determining that the transmission will achieve this result, entries are preferably transmitted.

Even better, the nodes from which the entries are transmitted are in some approaches only blocked for the duration of the transmission (eg by exclusive access). This node redistribution best involves only a few entries or a lot of information being transferred necessary to bring the underfilled node to the desired node utilization area.

In further embodiments, to redress an underfilled node, node redistribution may involve merging the underfilled node, for example, if the above transmission procedure is not possible because one or more of the above-noted findings have a negative detection content. Under these circumstances, merging may involve a number of findings. For example, in one approach, merging nodes involves determining whether the underfilled node has an adjacent node that is not included in the input set of nodes and operations. Further, in response to determining that there is such an adjacent node, merging an underfilled node further comprises determining whether the adjacent node becomes full due to the transmission of all entries from the underfilled node to the adjacent node. In response to determining that the adjacent node does not become full due to the transmission of all entries from the underfilled node to the adjacent node, merging an underfilled node further includes transmitting all entries from the underfilled node to the adjacent node. After all entries have been transferred from the previously underfilled, now empty node, this node can be deleted. Entries may be transmitted from the underfilled node to more than one adjacent node to allow the underfilled node to be emptied.

Advantageously, the alignment of the transmission in the merge operation discussed immediately above, ie the transfer of entries from the underfilled node to the adjacent node and not from the adjacent node to the underfilled node, serves to cause adjacent nodes to join merge operations involved, do not need to be deleted. This is advantageous because other concurrent operations may need to simultaneously access these neighboring nodes and this access must be possible in some approaches because these nodes are still referenced from another upper tree level. Moreover, the alignment desirably results in a situation where the deleted node is not needed for other concurrent operations, thereby maintaining consistency with the other concurrent operations.

In preferred embodiments, when the processing of the current node is complete, an exclusive privilege for the node, if previously acquired, is released. During processing of a particular tree level, access to the other, e.g. upper tree levels are performed concurrently for the purpose of finding neighboring nodes of all concurrent operations using, for example, shared permissions.

As mentioned above, the output of concurrent processing of operations on the current level nodes is a new set of nodes and operations having a list of memory identifiers of the nodes in the next level of the tree to which operations are to be applied. In some embodiments, the new set of nodes and operations may consist of the list of memory tags and a set of one or more heterogeneous operations associated with each of the memory tags.

Preferably, the operations specified in the new set of nodes and operations that are to be applied to the next level of the tree are the result of the operations applied to the nodes in the current level. In such embodiments, the new set of nodes and operations contains lists of pairs, each pair containing an entry, e.g. a data or forwarding entry, and an associated operation (e.g., insert, delete, update, etc.) to be applied thereto. Each such list of pairs may advantageously be associated with a memory identifier of a node to which the operations in the list are to be applied.

The entries and operations generated during the processing of the current tree level also advantageously support the new shape of the nodes that have been updated, created, and deleted in the course of processing the current tree level.

In various embodiments, situations that are likely to include entries and Create operations that support the new shape of the nodes, new nodes that are created when processing nodes are split. This creates entries to be added to the next higher level of the tree. Such entries are contained in the set of nodes and operations generated during processing of a current tree level.

In further embodiments, situations that are likely to produce entries and operations that support the new form of the nodes include transfers of entries, particularly the complete transmission of entries (e.g., by merge operations) from a processed node to adjacent nodes. More specifically, if such a transfer results in the processed node being deleted, it may be advantageous to require the deletion of an entry referencing that deleted node in the next level of the tree, since the reference to the deleted node is now invalid.

In still further embodiments, situations that are likely to produce entries and operations that support the new form of nodes include update operations that involve changing entries in a manner that causes the representative entry of a node to change. In this case, it is advantageous to create entries for deletion from and / or insertion into the next higher level of the tree. As those skilled in the art will understand after reading the present descriptions, such deletion and insertion may help to maintain consistency within the tree structure and within routing entries by ensuring that representative entries provide data and / or routing information corresponding to the changed one Mark tree structure correctly.

In preferred embodiments, the set of heterogeneous operations indicated in the received input is performed for the particular level, node balancing operations are performed for the particular level, and upon completion of these operations, the new set of nodes and operations is created and used used in a subsequent iteration of the update phase.

If no operations are required for the next level of the tree-based data structure, the new set of nodes and operations is empty. Thus, each iteration of the update phase may involve determining whether the set of nodes and operations is empty and, in response to that determination, terminating the update phase. The completion of the update phase may involve the release of exclusive access to the root node of the tree-based data structure and / or the completion of the update phase may be followed by the release of exclusive access to the root node of the tree-based data structure.

Simultaneous mass processing of a B + tree with n levels

Similarly, in a more specific case involving a n-level B + tree, the techniques described herein can similarly be leveraged to significantly improve the performance of a data storage system in which the n-level B + tree is implemented as a data structure , The computer-implemented method is intended to simplify the concurrent processing of heterogeneous operations on a paged search tree data structure.

The method preferably includes receiving an input on a storage system in which the tree-based data structure is stored; and simultaneously and quantitatively performing a plurality of heterogeneous operations on one or more of the leaf nodes based on the input. The input identifies a set of heterogeneous operations to be applied to the paged search tree data structure.

As with the procedure 1000 For example, the more specific implementation in various embodiments may include a search phase. The search phase preferably includes: evaluating each level of the tree-based data structure in an order from the root node level to the leaf node level; and simultaneously determining the leaf nodes to which the set of heterogeneous operations is to be applied.

Further, the search phase generates an output in the form of a set of nodes and operations containing a list of memory IDs. Each memory identifier preferably identifies one of the leaf nodes to which one or more operations of the set of heterogeneous operations are to be applied; and each of the memory identifiers is assigned to at least one pair of data entry and operation identified in the received input.

Also in a similar way as in the procedure 1000 For example, the method may include an update phase associated with the application of one or more operations of the set of heterogeneous operations to each leaf node at the leaf node level (eg, the leaf node level 410c , in the 4 shown) having a memory identifier listed in the set of nodes and operations. More specifically, for each of the leaf nodes having a memory identifier listed in the set of nodes and operations, the theorem of heterogeneous operations applied to the particular leaf node in this first iteration (ie, i = 0) preferably on the data entry / operation pair (s) associated with the memory identifier for that leaf node. Moreover, in preferred approaches, the set of heterogeneous operations is applied simultaneously and in a quantity-based manner to each of the leaf nodes having a memory identifier listed in the set of nodes and operations.

The update phase can generate a new set of nodes and operations for each completed iteration. The new set of nodes and operations identifies nodes at a higher level than the level being processed in the current iteration and operations to be performed on that node. Preferably, this definition is based on the result of the operations applied to the current level. Generating this output in each iteration involves generating a new set of nodes and operations based at least in part on the application of the heterogeneous operations to the nodes having a memory identifier listed in the set of nodes and operations ,

Thus, in one embodiment, the new set of nodes and operations generated during the first iteration preferably comprises a list of memory tags, and each memory tag identifies one of the inner nodes in the next higher node level of the B + tree to which one or more several operations of the set of heterogeneous operations should be applied. Thus, the set of heterogeneous operations in the next iteration (i = 1) of the update phase is applied to the inner nodes in the next higher node level that have a memory identifier listed in the new set of nodes and operations. Preferably, the set of heterogeneous operations is applied simultaneously and quantity-based to each of the inner nodes in the second node level having a memory identifier listed in the set of nodes and operations.

The iterative process described above may be continued, with any necessary changes to the entries in the B + tree and / or changes to the structure of the B + tree being passed in a plane until no further operations are needed, it is determined that the Node and operations existing sentence is empty, and thus the update phase ends. Preferably, the set of heterogeneous operations is applied simultaneously and quantity-based to each of the nodes in a particular layer having a memory identifier listed in the set of nodes and operations generated during the previous iteration of the update phase.

Isolate concurrent operations while editing the tree

The presently disclosed inventive concepts enhance the operation of conventional memory systems that use tree-based data structures by reducing the computational cost of updating data entries and the tree structure. As mentioned above, one significant aspect of this improved efficiency stems from the techniques for simultaneous mass processing of heterogeneous operations in the context of tree-based data structures.

An additional aspect of the presently disclosed techniques of the present invention further enhances computational performance by isolating the processing of nodes that are being processed at a particular level of the tree-based data structure to facilitate concurrent and volume-based processing of these heterogeneous operations. This technique of isolating, in various embodiments, generally includes a qualification of those nodes that can receive entries from nodes and / or pass entries to nodes that are being processed in a current iteration of the update phase.

In effect, this allows the updating process to deal efficiently with full and underfilled nodes, thereby simplifying the independent and isolated processing of each node. In particular, when this isolation is coupled with parallelization of the various process threads of an operation, it improves the efficiency to which concurrent processing of the nodes of a particular tree level contributes.

In general, the currently disclosed isolation procedure of the present invention isolates the processing of nodes specified for processing in the node-and-operation input set for a particular level by qualifying nodes that are eligible for node redistribution and other structural change operations , in particular the transfer of entries.

For example, in one aspect, the nodes that can qualify are nodes that are not dependent on any of the operations that are performed, so that the nodes may be involved in transfer operations, merge operations, and so on, without interfering with the other operations that make up the entire tree change / update process. In practice, these qualifying nodes include nodes that ( 1 ) Are adjacent to nodes currently being processed ( 2 ) are not themselves contained in the input set of nodes to be processed, and ( 3 ) meet one or more additional qualification criteria depending on the case. These additional qualification criteria are described in detail below.

In an embodiment in which entries may be transferred from a processed node to a qualifying node, additional qualification criteria include the question of whether the proposed transmission would cause the adjacent node to become full. If so, the node can not qualify for the transfer, otherwise the node may qualify for the transfer.

Conversely, in one embodiment, a node may qualify to participate in a transfer of entries from the node to an adjacent node undergoing processing if the node from which entries are to be transmitted is not undermined as a result of the transfer itself. Thus, additional qualification criteria may generally include the desired utilization range for nodes, and the qualification process may preferably include an evaluation of node utilization levels relative to the desired utilization range, determination of whether the proposed transfer results in a violation of the desired utilization range, and the qualification of nodes in response to the finding that the proposed transfer does not result in such an infringement.

In further embodiments, additional qualification criteria in this scenario may include the criterion of whether the transfer requires further pending insert operations to be applied to the neighboring nodes to which the transfer is proposed. If so, the knot can not qualify again, but otherwise it can qualify. In yet another scenario, additional qualification criteria may include targeting a proposed merge operation.

More specifically, a qualification may include determining whether a proposed merge of a processed node with adjacent nodes involves a transfer of all entries of the processed node to the adjacent nodes, or vice versa. In response to determining that the proposed merge involves transmitting all entries of the processed node into the adjacent nodes, the neighboring nodes may qualify for the transmission of entries. Otherwise, the neighboring nodes can not qualify.

In addition, the unilateral nature of the allowed merge means that not the neighboring nodes, but the processed node is subject to deletion after the merge operation has been completed. Preferably, the adjacent nodes are not deleted by the merge operation.

Regardless of the particular scenario and qualification criteria, the adjacent nodes in preferred approaches are locked by an exclusive entitlement to allow the transfer (s) to be performed concurrently with other operations. To minimize delay associated with such exclusive access, the neighboring nodes are preferably temporarily disabled, and more preferably, they are only disabled for the duration of the transfer operation. Upon completion of the transfer, exclusive access to the node is released and other concurrent processing can be continued efficiently. Likewise, nodes that are processed by a change operation are preferably only temporarily disabled by exclusive access, and more preferably, they are locked only for the duration of the respective change operation.

As one of ordinary skill in the art will understand after reading the present descriptions, there are certainly possible scenarios in which a particular node being processed by one or more heterogeneous operations may not have a qualifying adjacent node that is suitable for one Transmission as described here. In such cases, if the processing results in the processed node becoming full, and in response to the determination that there are no adjacent nodes that qualify for transmission of entries from the processed node, the processed node is preferably split , It is also worth noting that there may be multiple such splitting operations occurring, particularly during the processing of insert operations on a processed node.

It is also worth noting that the presently disclosed embodiments according to various approaches achieve excellent concurrency through horizontal concurrent processing without the need for additional indirect address mapping. Additional indirect address mapping undesirably entails extra overhead and resource consumption (eg, additional I / O operations, extra processing time, extra memory, etc.), the avoidance of which improves the operation of the memory system.

Now referring to 11 is a flowchart of a procedure 1100 for isolating nodes according to an embodiment to facilitate their simultaneous mass processing. The procedure 1100 can be carried out according to the present invention in various embodiments in any of the environments described inter alia in the 1 to 6 are shown. Of course, more or less surgeries can be done than those in 11 are described in detail in the process 1100 what the skilled artisan should understand after reading the present specifications.

Each of the steps of the procedure 1100 may be performed by any suitable component of the operating environment. For example, the procedure 1100 in various embodiments may be performed in whole or in part by a storage system manager or other entity in which one or more processors reside. The processor, eg, one or more processing circuits, one or more chips, and / or one or more modules embodied in hardware and / or software, and preferably having at least one hardware component, may be used in any unit to provide one or several steps of the procedure 1100 perform. Illustrative processors include, but are not limited to, a central processing unit (CPU), an application specific integrated circuit (ASIC), a custom programmable general purpose circuit (FPGA), etc., combinations thereof, or any other suitable ones in which Technology known data processing unit.

As in 11 The procedure can be shown 1100 with the surgery 1102 begin in which an input is received on a storage system that manages a storage unit in which the tree-based data structure is stored. The input identifies a set of heterogeneous operations to apply to the tree-based data structure.

The procedure 1100 also includes the surgery 1104 in which one or more nodes of the tree-based data structure to which one or more operations of the set of heterogeneous operations are to be applied are identified or otherwise determined. The determination may be made in various embodiments in any suitable manner consistent with the present descriptions. In preferred approaches, the determination is based on searching the input entries in the tree-based data structure to determine the nodes that host them, or based on the output generated by the search phase or a previous iteration of the update phase in the form of nodes and Operations, as discussed above.

Furthermore, the procedure includes 1100 the operation 1106 in which one or more sets of the set of heterogeneous operations to be applied to the one or more nodes are identified or otherwise determined. The determination may be made in various embodiments in any suitable manner consistent with the present descriptions. Preferably, the determination is made in different approaches based on which nodes the one or more operations are to be applied to, the determination being made on the basis of data contained in the set of nodes and operations.

In preferred approaches, the determination is based, at least in part, on the nodes and operations issued by the search phase or the previous iteration of the update phase, as discussed above. Specifically, the finding may include grouping together heterogeneous operations corresponding to the nodes to which the heterogeneous operations are to be applied, as those skilled in the art will understand after reading the present descriptions.

In more preferred approaches, each node in the set of nodes and operations occupies the same level of the tree-based data structure.

In the operation 1108 includes the procedure 1100 isolating the processing of each node in the set of nodes and operations for independent processing. As is understood in this document, isolating node processing involves any suitable technique or mechanism configured to enable the processing of the node by one or more heterogeneous operations, the processing taking place independently of the processing of heterogeneous operations on other nodes in the set of nodes and operations. Preferably, node isolation involves isolating each node in the set of nodes and operations so that the entire set of heterogeneous operations to be applied to the node (s) in the set can be performed without interfering with the processing of heterogeneous operations on other nodes in the tree, especially at the current level. For example, in one embodiment, independent processing of isolated nodes involves processing each node on a different process thread.

The procedure 1100 therefore includes the surgery 1110 in which each node in the set of nodes and operations is processed using the one or more sets of the set of heterogeneous operations designated for application to the set of nodes. Each node in the set of nodes is preferably processed simultaneously and independently.

Preferably, all nodes are processed simultaneously, so that each process thread is executed substantially simultaneously. Of course, some threads may take more time than others to complete, but as one skilled in the art will understand after reading the present descriptions, concurrent, independent processing includes embodiments in which the processing of all threads in the Set is started at the same time and all threads are processed during a largely same time window.

As those skilled in the art will understand after reading the present specifications, the method set forth above provides 1100 independent, isolated, and concurrent processing of a set of nodes using a set of one or more heterogeneous operations designated for application to one or more nodes in the set. Of course, the procedure can 1100 comprise processing a plurality of different sets of heterogeneous operations on a plurality of different sets of nodes in a similar manner as set forth above.

In an exemplary embodiment, the method 1100 For example, processing includes a plurality of different sets of operations on different sets of nodes, where all nodes in the different sets occupy a single level of the tree-based data structure. For the sake of simplicity, we assume that the exemplary tree-based data structure has a plane that has four nodes N1, N2, N3, and N4 (eg, as in FIGS 4 to 6 is shown).

In such an exemplary scenario, the method 1100 determining that a group of heterogeneous operations, including an update operation and an insert operation, are to be applied to a set of nodes including N1, N2, and N3, as well as determining that another group of heterogeneous operations , which include an update operation and a delete operation, are to be applied to a set of nodes to which N3 and N4 belong. As will be understood by those skilled in the art after reading the present descriptions, any number of such groups and sets may be identified and the corresponding nodes isolated for concurrent, independent processing, as discussed herein.

Of course, the procedure can 1100 any number of additional and / or alternative features, such as those discussed above, as well as the illustrative features set forth below.

In one embodiment, the method 1100 determining whether one or more nodes qualify for participation in a node reallocation operation. Preferably, this determination is based on node redistribution criteria, and may in particular include any combination, permutation, or synthesis of the following considerations.

In one approach, determining whether one or more nodes qualify to participate in the node reallocation operation includes: identifying one or more nodes adjacent to at least one node in the set of nodes; Determining whether any of the one or more adjacent nodes are excluded from the set of nodes; and qualifying at least one of the one or more adjacent nodes in response to determining that the at least one of the one or more adjacent nodes in the set of nodes is excluded. Preferably, the neighboring nodes are not the target of operations specified in the input received from the storage system.

In preferred embodiments, the node reallocation operation includes at least one transfer of entries between a node in the set of nodes and a node excluded from the set of nodes. Of course, the procedure can 1100 also include node redistribution operations such as update operations, partitioning operations, merge operations, etc., as those skilled in the art will understand after reading the present descriptions.

In embodiments in which the transfer of entries involves transferring entries from the node in the set of nodes to the nodes excluded from the set of nodes, the determination of whether one or more nodes are eligible for participation in a node reallocation operation qualify, include one or more of the following sub-operations. In one approach, determining whether nodes qualify includes determining whether the node excluded from the set of nodes becomes full as a result of the transfer of entries; and qualifying the node excluded from the set of nodes for transmission in response to determining that the node excluded from the set of nodes is not becoming full due to the transmission of entries.

In addition, the determination of whether the node excluded from the set of nodes becomes full as a result of the transfer of entries may include: estimating a utilization level of the node excluded from the set of nodes after the transfer; and comparing the estimated utilization level of the node excluded from the set of nodes with a maximum utilization threshold or the storage capacity of the node. If the estimated utilization level does not exceed the maximum utilization threshold or the storage capacity of the node, the node may be qualified. If the estimated utilization level exceeds the maximum utilization threshold or the storage capacity of the node, the node is preferably not qualified.

Similarly, and in embodiments in which the transmission of entries involves transmitting entries from the node excluded from the set of nodes to the nodes in the set of nodes, the determination of whether one or more nodes are eligible for participation may be made qualify a node redistribution operation, include the following. In one embodiment, the process concludes whether the node excluded from the common set of nodes is being underfunded as a result of the transfer of entries; and qualifying the node excluded from the set of nodes for transmission in response to determining that the node excluded from the set of nodes is not underfilled due to the transmission of entries.

Thus, determining whether the node excluded from the set of nodes is under-populated as a result of the transfer of entries may include: estimating a utilization level of the node excluded from the set of nodes after the transfer; and comparing the estimated utilization level of the node excluded from the set of nodes with a minimum utilization threshold. When the estimated utilization level reaches or exceeds the minimum utilization threshold, the node may be qualified. Preferably, if the estimated utilization level does not reach or exceed the minimum utilization threshold, the node is not qualified.

In still other embodiments, the node reallocation operation may include a merge operation. Thus, the method may include transmitting all entries in the node in the set of nodes to the nodes excluded from the set of nodes. The determination of whether one or more nodes qualify for participation in a node reallocation operation involving merge operations preferably includes: determining whether the node excluded from the set of nodes becomes full as a result of the transfer of entries; and qualifying the node excluded from the set of nodes for transmission in response to determining that the node excluded from the set of nodes is not becoming full due to the transmission of entries. To complete the merge operation after all entries have been transferred to the node excluded from the set, the node in the set of nodes from which the entries were transferred and which became empty is deleted.

As mentioned above, in various embodiments, node redistribution may include obtaining exclusive access to a node that qualifies to participate in the node reallocation operation. Preferably, the exclusive access is obtained before starting the update of the nodes involved in the node reallocation operation; and, better still, immediately before the update of the nodes involved in the node redistribution operation is started. Exemplary embodiments of this process may include performing the node reallocation operation using the node that is in favor of sharing qualified at the node reallocation operation; and including, upon completion of the node reallocation operation, granting exclusive access to the node qualifying to participate in the node reallocation operation. Consequently, the exclusive access preferably exists for a period substantially equal to a duration of the node reallocation operation.

In still further embodiments, the node reallocation operation may include transmitting entries from the node in the set of nodes, for example, according to a splitting operation. The determination of whether one or more nodes qualify for participation in a node reallocation operation may thus include: identifying one or more nodes adjacent to at least one node in the set of nodes; Determining whether any of the one or more adjacent nodes are excluded from the set of nodes; Determining whether the adjacent node excluded from the set of nodes becomes full as a result of the transfer of entries; and splitting the node in the set of nodes in response to determining that the adjacent node excluded from the set of nodes becomes full as a result of the transfer of entries.

Of course, the above embodiments are merely exemplary embodiments of a technique of the invention for isolating nodes for concurrent mass processing, and they should not be considered as embodiments limiting the scope of the present disclosure.

Efficiently determining the order of operations

As noted above, the presently disclosed inventive concepts provide conventional architectures of memory systems having an improved function by allowing a novel and efficient determination of the order of the heterogeneous types of operations to be applied to a node: the heterogeneous operations are performed according to the Groups that affect them are grouped together and then placed in a specific order within each group according to their particular nature, minimizing structural changes resulting from the application of operations. A beneficial result is to significantly reduce the amount of output set of operations generated by each phase of the algorithm, thereby reducing the total number of operations necessary to complete the processing of the tree-based data structure, and to the improved computational power contributed to by the present techniques.

For example, in one embodiment, a particular group of nodes is affected by an update operation in which entries of each node in the particular group are to be changed. The nodes are grouped together for processing, and within the group, any structural changes necessary due to the performance of the update operations (e.g., insert operations, delete operations, etc.) are evaluated.

After the necessary set of operations has been determined to perform the update and any desirable node balancing, the set of operations is preferably performed according to the preferred order shown here. The preferred order again involves performing update operations, followed by delete operations and subsequent insert operations. However, in other embodiments, a custom order may be predetermined (e.g., in the received input) and implemented.

Consequently, and as in 12 The procedure can be shown 1200 begin with operation 1202, in which an input is received at a storage system that manages a storage unit in which the tree-based data structure is stored. The input identifies a set of heterogeneous operations to apply to the tree-based data structure.

The procedure 1200 also includes the surgery 1204 in which one or more nodes of the tree-based data structure to which one or more operations of the set of heterogeneous operations are to be applied are identified or otherwise determined. The determination may be made in various embodiments in any suitable manner consistent with the present descriptions. In preferred approaches, the determination is based on searching the input entries in the tree-based data structure to determine the nodes that host them, or based on the output generated by the search phase or a previous iteration of the update phase in the form of nodes and Operations, as discussed above.

Furthermore, the procedure includes 1200 the operation 1206 in which one or more groups of the set of heterogeneous operations, the to be applied to the one or more nodes, identified or otherwise determined. The determination may be made in various embodiments in any suitable manner consistent with the present descriptions.

In preferred approaches, the determination is based, at least in part, on the nodes and operations output by the search phase and / or the set of nodes and operations output from a previous iteration of the update phase, as discussed above. Specifically, the determination may include identifying a set of nodes based on their memory IDs, and determining that the entire set of nodes has the destination of an insert operation and / or an erase operation and / or a merge operation and / or a split operation and / or an update operation, etc. is what a person skilled in the art might understand after having read the present descriptions.

In more preferred approaches, each node in the set of one or more nodes occupies the same level of the tree-based data structure.

The procedure 1200 further includes the operation 1208 in which the one or more groups of the set of heterogeneous operations are applied. Importantly, the set of operations used in each group is applied according to a predetermined order. Preferably, the predetermined order is the preferred order, as indicated herein.

As those skilled in the art will understand, after having read the present specifications, the method may 1200 Of course, also include any number of additional functions and / or features, as described here. In various embodiments, the method 1200 include any one or more of the following features and / or functionalities.

Advantageously, the presently disclosed embodiments of the invention, including the application of the set of heterogeneous operations to the set of nodes according to the predetermined order, improve the operation of storage systems by: minimizing a number of structural changes to the tree-based data structure; and reduce a scope of an output set of nodes and operations generated as a result of applying the set of heterogeneous operations to a particular level of the tree-based data structure.

As noted above, the predetermined order includes the application of update operations, followed by delete operations followed by insert operations. In one embodiment, the method 1200 Consequently, it may include performing update operations identified in the set of heterogeneous operations before performing delete operations identified in the set of heterogeneous operations, and may perform delete operations identified in the set of heterogeneous operations before it Performs insert operations identified in the set of heterogeneous operations.

Likewise, in other embodiments, the predetermined order may include performing node redistribution operations (eg, transfer of entries, partitioning or merge operations) after preferably performing all update, delete, and insert operations described in the set of heterogeneous operations for a specific node were identified. The implementation of the node reallocation is preferably performed in response to the determination that a node in the set of nodes is full or underfilled after the group of operations specified in the input has been performed.

In further approaches, the predetermined order includes performing a partitioning operation in response to determining that a node in the set of nodes becomes full in the course of performing the group of operations.

In addition, as noted above, in some embodiments, the order of the operations may be specified by a user, particularly if the operations are to be applied to a set of nodes storing data entries, and more particularly if the operations during a first iteration of a Update phase to a set of leaf nodes of a B + tree, as described herein.

Consequently, the process can 1200 additionally and / or alternatively, overwriting the predetermined order in response to determining that a user-specified order was specified in the input, wherein the user-provided input corresponds to one or more operations of the set of heterogeneous operations that occur on the tree-based data structure to be used, which stores data entries.

Of course, the above embodiments are merely exemplary embodiments of a technique of the invention for isolating nodes for concurrent mass processing, and they should not be considered as embodiments limiting the scope of the present disclosure.

Using the above inventive concepts, the presently disclosed techniques provide and provide a variety of advantageous features that act individually and in combination to improve the operation of conventional data storage systems, such as accessing the memory system, I / O, and processor load is reduced. More specifically, the presently disclosed inventive concepts enable the quantity-based processing of heterogeneous types of operations in a manner optimized for efficient concurrent processing of sets of heterogeneous operations. In addition, isolating the processing of each node being processed at a current tree level enables efficient concurrent mass processing of operations. Furthermore, structural changes resulting from the application of operations are minimized by providing a maximum efficient order of operations, which contributes to the efficiency of the algorithm.

Consequently, the presently disclosed inventive concepts fit well and provide a comprehensive solution for applications accessing and modifying data structure transactions, which is a very common architecture for many use cases and applications that aggregate operations and then potentially reduce operations before apply them to data structures. In particular, the presently disclosed inventive concepts are applicable to the general family of paged search tree data structures and provide a solution with a wide range of applications to improve the efficiency of a widely used component in many applications. However, it should be understood that these disclosures provide concepts that are applicable to many systems that utilize tree-based data structures, such as database systems, file systems, storage and deduplication systems, and therefore have a wide variety of applications.

It will be understood that the various features of the foregoing systems and / or methodologies may be combined as desired to create a variety of combinations of the descriptions set forth herein. It should be further understood that embodiments of the present invention may be provided in the form of a service installed on behalf of a customer to provide service-on-demand.

While various embodiments have been described below, it should be understood that they have been presented by way of example only and not limitation. Thus, the scope and scope of a preferred embodiment should not be limited by any of the exemplary embodiments described below, but should be defined only in accordance with the following claims and their equivalents.

Claims (26)

A computer-executed method for concurrently processing operations on a tree-based data structure, comprising: Receiving an input to a storage system managing a storage unit in which the tree-based data structure is stored, the input identifying a set of heterogeneous operations to be applied to the tree-based data structure; Determining one or more nodes of the tree-based data structure to which one or more operations of the set of heterogeneous operations are to be applied; and performing simultaneous and quantity-based one or more operations of the set of heterogeneous operations. Method according to Claim 1 wherein the set of heterogeneous operations comprises one or more of: insert operations, delete operations, and update operations. Method according to Claim 2 wherein the input comprises: a reference to the tree-based data structure and a list of pairs, each pair consisting of a data entry and at least one associated operation. Method according to Claim 1 wherein the determination does not include access to leaf nodes of the tree-based data structure; wherein the leaf nodes store one or more data entries; and where at least some of the data entries are specified in the input. Method according to Claim 1 comprising a search phase, the search phase comprising: evaluating nodes in each level of the tree-based data structure in a descending order; and wherein the evaluation for each level comprises simultaneously determining whether any of the nodes in the level being evaluated are one or more of: a node to be used to route the search to the tree; and a node to which one or more operations of the set of heterogeneous operations are to be applied. Method according to Claim 5 wherein the search phase generates an output comprising: a set of nodes and operations, the set of nodes and operations having a list of memory identifiers, each memory identifier identifying one of the nodes to which one or more operations of a set of heterogeneous operations to be applied; and wherein at least some of the memory identifiers belong to at least one operation generated by the search phase. Method according to Claim 6 comprising an update phase, wherein the update phase comprises: applying to each node one or more operations of the set of heterogeneous operations having a memory identifier listed in the set of nodes and operations, wherein each of the nodes, has a memory identifier listed in the set of nodes and operations, the one or more operations of the set of heterogeneous operations applied to the node are based on the at least one operation associated with the memory identifier for the node ; and wherein the one or more operations of the set of heterogeneous operations are applied concurrently, independently, and quantity-based to each of the nodes having a memory identifier listed in the set of nodes and operations. Method according to Claim 7 wherein the application of the one or more operations of the set of heterogeneous operations to each node having a memory identifier listed in the set of nodes and operations comprises one or more of: inserting one or more new nodes ; Delete one or more existing nodes; Merging one or more existing nodes; Splitting one or more existing nodes; and updating one or more data entries from one or more existing nodes. Method according to Claim 8 wherein new nodes that are generated when processing nodes are split create entries to be added to a next higher level of the tree-based data structure in a next update phase, wherein the complete transmission of entries from a processed node to sibling nodes is an erase operation of a In the next update phase, an entry in the next higher level of the tree is generated, and wherein a processed node, in which the contents of its entries have been changed and whose representative entry has thereby changed, generates entries which are deleted in the next update phase and in the next higher level of the Tree should be inserted. Method according to Claim 7 where the update phase is an iterative process; and wherein the operations generated by each update phase result from generating nodes, deleting nodes, and changing nodes in a current iteration of the update phase. Method according to Claim 7 wherein the update phase comprises generating a new set of nodes and operations based, at least in part, on applying the one or more operations of the set of heterogeneous operations to each node having a memory identifier included in the one of nodes and operations existing sentence is listed; wherein the new set of nodes and operations comprises a list of memory identifiers and wherein each memory identifier in the new set of nodes and operations identifies one of the nodes to which one or more operations of a new set of heterogeneous operations are to be applied. Method according to Claim 11 wherein the update phase comprises: applying to each node one or more operations of the set of heterogeneous operations having a memory identifier listed in the new set of nodes and operations, and wherein the one or more operations of the set of heterogeneous operations simultaneously and applied to any of the nodes having a memory identifier listed in the new set of nodes and operations. Method according to Claim 7 wherein the update phase comprises: determining whether a new set of nodes and operations is in response to the application of the one or more operations of the set of heterogeneous operations on each of the nodes having a memory identifier that is in the one of nodes and operations existing set is generated, has memory identifiers corresponding to a node in a next higher level of the tree-based data structure; and in response to determining that the new set of nodes and operations does not have memory identifiers corresponding to a node in the next higher level of the tree-based data structure, terminate the update phase. Method according to Claim 7 wherein the tree-based data structure has a plurality of node levels and the method comprises iteratively performing the update phase for one or more of the plurality of node levels; wherein each iteration of the update phase generates a new node-level set of operations to be updated in a next iteration of the update phase; and wherein the node level to be updated in the next iteration of the update phase is above the node level at which the update phase is performed in a current iteration of the update phase. Method according to Claim 7 wherein access to nodes in the search phase takes place by means of a shared privilege, and wherein access to nodes to which the one or more operations of the set of heterogeneous operations are applied during the update phase is done by means of an exclusive privilege. Method according to Claim 7 wherein the concurrent and quantity-based application of one or more operations of the set of heterogeneous operations to each node having a memory identifier listed in the set of nodes and operations, applying operations to each node using a different process thread and each process thread is processed in parallel. A computer-executed method for concurrently processing heterogeneous operations on a tree-based data structure, comprising: a root node layer consisting of a root node; a first nodal plane originating at the root node level and having at least two inner nodes; a second nodal plane derived from the first nodal plane and having at least four inner nodes; and a leaf node plane derived from the second nodal plane and consisting of a plurality of leaf nodes; and the method comprising: Receiving an input at a storage system storing the tree-based data structure, the input identifying a set of heterogeneous operations to be applied to one or more leaf nodes of the tree-based data structure; and simultaneously and quantitatively performing one or more operations of the set of heterogeneous operations on one or more of the leaf nodes based on the input. Method according to Claim 17 comprising a search phase, the search phase comprising: evaluating each level of the tree-based data structure in a descending order from the root node level to the leaf node level; and simultaneously determining the leaf nodes to which the set of heterogeneous operations is to be applied. Method according to Claim 18 wherein the search phase generates an output comprising: a set of nodes and operations, the set of nodes and operations having a list of memory identifiers, each memory identifier identifying one of the leaf nodes to which one or more operations of the set of heterogeneous operations to be applied; and wherein each of the memory identifiers is associated with at least one data entry / operation pair identified in the received input. Method according to Claim 19 comprising an update phase, the update phase comprising: applying one or more operations of the set of heterogeneous operations to each leaf node having a memory identifier listed in the set of nodes and operations, wherein for each of the leaf nodes, the has a memory identifier that is made up of nodes and Operations, the one or more operations of the set of heterogeneous operations applied to the leaf nodes are based on the at least one data entry / operation pair associated with the memory node for the leaf node; and wherein the set of heterogeneous operations is applied simultaneously and in a quantity-based manner to each of the leaf nodes having a memory identifier listed in the set of nodes and operations. Method according to Claim 20 wherein the update phase comprises: generating a new set of nodes and operations based at least in part on the application of the one or more operations of the set of heterogeneous operations to the leaf nodes having a memory identifier consisting of nodes and operations existing set, the new set of nodes and operations having a list of memory identifiers and wherein each memory identifier identifies one of the inner nodes in the second node level to which one or more operations of the set of heterogeneous operations are to be applied; Applying one or more operations of the set of heterogeneous operations to the inner nodes in the second node level having a memory identifier listed in the new set of nodes and operations; and wherein the one or more operations of the set of heterogeneous operations are applied concurrently and quantity-based to each of the inner nodes in the second node level having a memory identifier listed in the new set of nodes and operations. Method according to Claim 21 wherein the update phase comprises: generating a second new set of nodes and operations based at least in part on the application of the one or more operations of the set of heterogeneous operations to the inner nodes having a memory identifier which is unique in the new one Node and Operations, the second new set of nodes and operations having a list of memory IDs and each memory ID identifying one of the inner nodes in the first node level to which one or more operations of the set of heterogeneous operations are applied should; Applying one or more operations of the set of heterogeneous operations to the inner nodes in the first node level having a memory identifier listed in the second new set of nodes and operations; and wherein the one or more operations of the set of heterogeneous operations are applied simultaneously and quantity-based to each of the inner nodes in the first node level having a memory identifier listed in the second new set of nodes and operations. A computer-executed method of processing operations on a tree-based data structure, comprising: Receiving an input to a storage system managing a storage unit in which the tree-based data structure is stored, the input identifying a set of heterogeneous operations to be applied to the tree-based data structure; Determining one or more nodes of the tree-based data structure to which one or more operations of the set of heterogeneous operations are to be applied; Determining one or more sets of the set of heterogeneous operations, wherein the determining is based, at least in part, on the one or more nodes to which the heterogeneous operations are to be applied. A computer-executed method for concurrently processing operations on a tree-based data structure, the method comprising: Receiving an input to a storage system managing a storage unit in which the tree-based data structure is stored, the input identifying a set of heterogeneous operations to be applied to the tree-based data structure; Determining one or more nodes of the tree-based data structure to which one or more operations of the set of heterogeneous operations are to be applied; Determining one or more sets of the set of heterogeneous operations corresponding to the one or more nodes to which the set of heterogeneous operations is to be applied; and Applying, for each of the one or more groups, the set of heterogeneous operations according to a predetermined order. A computer program comprising computer program code stored on a computer readable medium when loaded on a computer system and thereon is executed to cause the computer system, all the steps of a method according to one of Claims 1 to 24 perform. A storage system for concurrently processing operations on a tree-based data structure, the storage system having a storage system manager and logic integrated with and / or executable by the storage system manager, the logic being configured to cause the storage system: receive an input identifying a set of heterogeneous operations to be applied to the tree-based data structure; determine one or more nodes of the tree-based data structure to which one or more operations of the set of heterogeneous operations are to be applied; and perform one or more operations of the set of heterogeneous operations simultaneously and quantity based.

Images