US20180095996A1 - Database system utilizing forced memory aligned access - Google Patents
Database system utilizing forced memory aligned access Download PDFInfo
- Publication number
- US20180095996A1 US20180095996A1 US15/722,687 US201715722687A US2018095996A1 US 20180095996 A1 US20180095996 A1 US 20180095996A1 US 201715722687 A US201715722687 A US 201715722687A US 2018095996 A1 US2018095996 A1 US 2018095996A1
- Authority
- US
- United States
- Prior art keywords
- data
- application
- node
- storage
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G06F17/30283—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
- G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/16—Handling requests for interconnection or transfer for access to memory bus
- G06F13/1668—Details of memory controller
- G06F13/1673—Details of memory controller using buffers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
- G06F13/28—Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access DMA, cycle steal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
- G06F13/4282—Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
- G06F15/163—Interprocessor communication
- G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
- G06F15/17306—Intercommunication techniques
- G06F15/17331—Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2358—Change logging, detection, and notification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H04L67/2842—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
- G06F2212/20—Employing a main memory using a specific memory technology
- G06F2212/202—Non-volatile memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F2213/0026—PCI express
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F2213/0032—Serial ATA [SATA]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
Definitions
- the present disclosure generally relates to networked database management systems (DBMS) and supporting infrastructure. More particularly, the present disclosure relates to computer software application access to resources, such as memory and disk. More particularly still, the present disclosure relates to efficient access by a database application to memory and storage and a database network topology to support the same. Even more particularly still, the present disclosure relates to efficiently aligned direct access to network and disk structures by a database application, for example, accessing NVME (Non-Volatile Memory Express) or SATA (Serial Advanced Technology Attachment) directly through a kernel bypass, as well as a computer software architecture to accomplish the same.
- NVME Non-Volatile Memory Express
- SATA Serial Advanced Technology Attachment
- a DBMS is a suite of computer programs that are designed to manage a database, which is a large set of structured data.
- a DBMS is designed to quickly access and analyze data on large amounts of stored data.
- Most modern DBMS systems comprise multiple computers (nodes).
- the nodes generally communicate via a network, which will use a network protocol, such as HTTP, or raw TCP/IP.
- Information that is exchange between nodes is exchanged by packets, the specific format of which will be determined by the specific protocol used by the network.
- the data wrapped in the packet will generally be compressed to the greatest extent possible to preserve network bandwidth. Accordingly, when it has been received, it will have to be formatted for use by the receiving node.
- Database input/output (“I/O”) systems comprise processes and threads that identify, read, and write blocks of data from storage; e.g., spinning magnetic disk drives, network storage, FLASH drives, or cloud storage.
- DBMS evolved from standalone computers, to sophisticated client/server setups, to cloud systems.
- An example of a cloud based DBMS is depicted in FIG. 1 .
- a cloud system 2 will generally comprise a variety of nodes (computers) as well as software that operates on the nodes.
- the cloud system 2 will comprise numerous separate nodes, including multiple database servers 1 .
- Each database server will maintain separate storage (not depicted), which will store some part of the maintained data.
- Various clients can access the cloud system 2 through the Internet 4 .
- Clients can include, for example, a standard desktop or laptop computer 6 , a mobile device 7 , as well as various sensors 8 and control equipment 9 .
- DBMSs operate on computer systems (whether standalone, client/server, or cloud) that incorporate operating systems.
- Operating systems which are usually designed to work across a wide variety of hardware, utilize device drivers to abstract the particular functions of hardware components, such as, for example, disk controllers, and network interface cards.
- driver As drivers are generally accessed through an operating system, such accesses will typically entail significant resource overhead such as a mode switch; i.e., a switch from executing application logic to operating system logic, or a context switch; i.e., the pausing of one task to perform another.
- Such switches are typically time consuming; sometimes on the order of milliseconds of processor time.
- Data stored in a DBMS is usually stored redundantly, using, for example, a RAID controller, Storage Area Network (“SAN”) system, or dispersed data storage.
- SAN Storage Area Network
- other measures to ensure that data is stored correctly are usually taken as well.
- many DBMSs utilize a write log.
- a write log which is generally written before the actual database is updated, contains a record of all changes to the database, so that a change can be easily backed out, or, in case of a transaction processing failure, can be redone as needed.
- writing to the log prior to committing the data guarantees that committed transactions can be preserved; i.e., properly written to disk. Using prior art methods, disk commits are lengthy procedures, often taking milliseconds.
- boxcarring can reduce the number of transactions that a system must track and commit. However, there are penalties in user response time, lock contention, and memory usage. In addition, boxcarring can complicate system recovery.
- a DBMS can use Remote Direct Memory Access (RDMA) to transfer data between two nodes (computers) of the DBMS.
- RDMA is a technology that allows a network interface card (NIC) to transfer data directly to or from memory of a remote node without occupying the central processing unit (CPU) of either node.
- NIC network interface card
- the term network interface card includes all network interfaces, including chips, and units built directly into processors.
- a remote client can register a target memory buffer and send a description of the registered memory buffer to the storage server. The remote client then issues a read or write request to the storage server.
- the storage server performs an RDMA read to load data from the target memory buffer into the storage server's local memory.
- the storage server then causes a disc controller to write the target data to a storage disk and, once the write is complete, generates and sends a write confirmation message to the remote client.
- the storage server uses the disk controller to perform a block-level read from disk and loads the data into its local memory.
- the storage server then performs an RDMA write to place the data directly into an application memory buffer of the remote computer. After an RDMA operation completes, the remote client deregisters the target memory buffer from the RDMA network to prevent further RDMA accesses.
- RDMA RDMA over Converged Ethernet
- OmniPath OmniPath
- FIG. 2 depicts a block representation of a conventional system in which data is copied from a pinned buffer in a first node to a second node.
- the first node 10 includes a pinned buffer 20 , a driver 30 , and a NIC 40 .
- the pinned buffer 20 and the driver 30 are each operatively coupled to the NIC 40 .
- the second node 50 includes a pinned buffer 60 , a driver 70 , and a NIC 80 .
- the pinned buffer 60 and the driver 70 are coupled to the NIC 80 .
- the NIC 40 is operatively coupled to the NIC 80 via a network 90 .
- the driver 30 , 70 will be accessible by an operating system running on the respective node 10 , 50 .
- the driver 30 in the node 10 writes a descriptor for a location of the pinned buffer 20 to the NIC 40 .
- the driver 70 in the node 50 writes a descriptor for the location of the pinned buffer 60 to the NIC 80 .
- the driver 30 works with the operating system, as well as other software and hardware on the node 10 to guarantee that the buffer 20 is locked into physical memory; i.e., “pinned.”
- the NIC 40 reads data from the pinned buffer 20 and sends the read data on the network 90 .
- the network 90 passes the data to the NIC 80 of the host 50 .
- the NIC 80 writes data to the pinned buffer 60 .
- FIG. 3 is a sequence of steps performed by a prior art network RDMA system to receive data over a network.
- a NIC 40 allocates a pinned memory buffer 20 .
- data is received from the network (not shown) by the NIC 40 into the pinned memory buffer 20 .
- a driver notifies an application 35 that data has been received.
- the application 35 calls into the driver 30 (usually through an operating system, which is not depicted) to read the data in the pinned buffer 20 .
- the driver 30 copies the data in the pinned buffer 20 to a separate memory buffer (not depicted) allocated by the application 35 .
- Serial Advanced Technology Attachment is a computer bus interface that connects host bus adapters to mass storage devices such as hard drives, optical drives, and solid state drives (SSDs). While SATA works with SSDs, it is not designed to allow for the significant level of parallelism that SSDs are capable of.
- NVMe, NVM (Non-Volatile Memory) Express, or Non-Volatile Memory Host Controller Interface Specification is a logical device interface specification for accessing non-volatile storage media attached via PCI Express (PCIe) bus.
- PCIe PCI Express
- Microprocessor architectures generally operate on data types of a fixed width, as the registers within the microprocessor will all be of a fixed width. For example, many modern processors include either 32 bit wide or 64 bit wide registers. In order to achieve maximal efficiency, prior to operating on data, the data must be properly aligned in memory along boundaries defined by that fixed width.
- a prior art DBMS system may include a data manipulation step that effectively compresses data prior to storing it or transmitting it via the network, and a decompression step when reading data from storage or the network.
- Another object of the disclosure is to provide an efficient network infrastructure.
- Another object of the disclosure is to provide a network infrastructure for a DBMS that allows a database application to directly access pinned RDMA memory.
- Another object of the disclosure is to provide a network infrastructure for a DBMS that manages pinned RDMA memory.
- Another object of the disclosure is to provide a storage infrastructure for a DBMS that allows a database application to directly access storage buffers used for disk access.
- Another object of the disclosure is to provide a storage infrastructure for a DBMS that that manages pinned DMA memory.
- Another object of the disclosure is to provide a DBMS infrastructure that allows an application to directly access NVME drives.
- Another object of the disclosure is to provide a DBMS infrastructure that allows an application to directly access SATA drives.
- Another object of the disclosure is to provide a DBMS infrastructure whereby data is stored in a format that is usable by a processor with minimal adjustment.
- Another object of the disclosure is to provide a DBMS infrastructure whereby network data is maintained in a format that is usable by a processor with minimal adjustment.
- the disclosed DBMS is capable of handling enormous amounts of data—an Exabyte or more—and accessing each record within the database frequently.
- the disclosed DBMS comprises a first high speed storage cluster including a first plurality of storage nodes. Each storage node includes a server and one or more storage drives at a high performance level.
- the first high speed storage cluster also includes a first switch.
- the DBMS also comprises a second high speed storage cluster including a second plurality of storage nodes. Each storage node in this second cluster also includes a server and one or more storage drives that operate at a lower performance level.
- the second high speed storage cluster also includes a second switch.
- the DBMS allows comprises an index cluster including a plurality of index nodes and a third switch.
- the first switch is operatively coupled to the third switch by a high speed RDMA capable link
- the second switch is operatively coupled to the third switch by a high speed RDMA capable link.
- the disclosed DBMS includes an application with a drive access class.
- the drive access class includes a NVME drive access sub class, and a SATA drive access sub class.
- the NVME drive access sub class allows the application to directly interface with NVME drives
- the SATA drive access sub class allows the application to directly interface with SATA drives.
- NVRAM technologies are now viable, and a specific sub class is allocated to optimize access to drives utilizing NVRAM technology. As future storage technologies are introduced, additional optimized storage access subclasses can be developed.
- the disclosed DBMS incorporates a node.
- the node includes a network interface card including a controller.
- a DBMS application executing on the node coordinates with the controller to allocate a pinned memory buffer, which is directly accessed by the DBMS application, which allows the DBMS application to directly access and manipulate received network data.
- the disclosed DBMS incorporates a node.
- the node includes a drive controller.
- a DBMS application allocates a pinned memory buffer, and communicates this buffer to the drive controller, which allows the DBMS application to access and manipulate drive data with minimal overhead.
- a node in a massively parallel database management system includes a processor having a preferred data alignment.
- An application operates on the processor, and creates a pinned memory buffer.
- the application also creates a data structure having a plurality of fields, wherein each of the plurality of fields is aligned on the preferred data alignment of the processor. The application can then insert data into the data structure without any performance degradation as explained herein.
- the pinned memory buffer can be an RDMA memory buffer.
- the node can further comprise a network interface card, and the application can communicate the data structure to a remote node using the network interface card.
- the network interface card can receive data over a network from a remote node and write the data into the RDMA memory buffer.
- the data received over the network can be a second data structure (which may be identical to the first) having a plurality of fields wherein each of the fields is aligned on the preferred data alignment of the processor.
- the node can further comprise a disk controller, such as a SATA controller or an NVME controller, which can be used to write the received data to disk.
- FIG. 1 depicts a prior art simplified network diagram of a cloud based DBMS system
- FIG. 2 depicts a prior art simplified block diagram of a network based RDMA system
- FIG. 3 depicts a sequence of steps that a prior art network based RDMA system would execute to receive data over a network
- FIG. 4 depicts a simplified network diagram illustrating the disclosed DBMS system and the infrastructure supporting the same;
- FIG. 5 depicts a simplified block diagram of a network implementing the disclosed RDMA memory buffer management system
- FIG. 6 depicts a sequence of steps that a node utilizing the disclosed RDMA memory buffer system will execute to receive data over a network
- FIG. 7 depicts a simplified block diagram of a node implementing the disclosed DMA memory buffer management system
- FIG. 8 depicts a sequence of steps that a node utilizing the disclosed DMA memory buffer management system will execute to read data from storage
- FIG. 9 depicts a prior art data structure for data passed by network or stored on disk
- FIG. 10 depicts a disclosed data structure for data passed by network or stored on disk
- FIG. 11 depicts a simplified class abstraction for a drive access class
- FIG. 12 depicts a simplified software block diagram of the disclosed DBMS system implementing the disclosed drive access class.
- This application discloses a number of infrastructure improvements for use in a database system, along with a DBMS utilizing those improvements.
- the infrastructure is adapted to allow the database system to scale to a size of an Exabyte or even larger, and allow each record stored in the database to be accessed quickly and numerous times per day.
- Such a database will be useful for many tasks; for example, every bit of data exchanged over a company's network can be stored in such a database and analyzed to determine, for example, the mechanism of a network intrusion after it has been discovered.
- data access latency There are numerous sources of data access latency, including, for example, network accesses, disk accesses, memory copies, etc. These latencies are exacerbated from distributed systems.
- the network architecture 100 comprises three storage clusters; a blazing storage cluster 105 , a hot storage cluster 115 , and a warm storage cluster 125 .
- the network architecture comprises an index cluster 135 .
- Each cluster comprises multiple nodes.
- the blazing storage cluster 105 is depicted as comprising five blazing storage nodes 101 ;
- the hot storage cluster 115 is depicted as comprising five hot storage nodes 111 , and the warm storage cluster 125 is depicted as comprising five warm storage nodes 121 .
- the index cluster 135 is depicted as comprising three index nodes 131 . While the clusters 105 , 115 , 125 , 135 are each depicted with a specific number of nodes, it should be noted that each cluster 105 , 115 , 125 , 135 can comprise an arbitrary number of nodes.
- a blazing storage node 101 may include, for example, an array of NVDIMM (Non-Volatile Dual Inline Memory Module)(a type of NVRAM) storage, such as that marketed by Hewlett Packard Enterprise, or any other extremely fast storage, along with appropriate controllers to allow for full speed access to such storage.
- NVDIMM Non-Volatile Dual Inline Memory Module
- DRAM with a write ahead log implemented on an NVMe drive could be utilized to implement blazing storage.
- write-ahead logging is used to log updates to an in-memory (DRAM) data structure.
- log entries are appended to the end of the log, they are flushed to an NVMe drive when the size of in-memory log entries is near or exceeds the size of a Solid State Memory page, or after a configured timeout threshold, such as ten seconds, has been reached.
- a configured timeout threshold such as ten seconds
- a hot storage node 111 may include, for example, one or more Solid State NVME drives, along with appropriate controllers to allow for full speed access to such storage.
- a warm storage node 121 may include, for example, one or more Solid State SATA drives, along with appropriate controllers to allow for full speed access to such storage.
- Each index node 131 will also include storage, which will generally comprise high performance storage such as Solid State SATA drives or higher performance storage devices.
- storage will generally comprise high performance storage such as Solid State SATA drives or higher performance storage devices.
- the index nodes 131 will store the relational database structure, which may comprise, for example, a collection of tables and search keys.
- certain of the clusters are connected via high speed, RDMA capable links 108 .
- the index cluster 135 is connected to the storage clusters 105 , 115 , 125 by high speed, RDMA capable links 108 .
- the storage clusters 105 , 115 , 125 are connected to one another by standard (non-RDMA capable) high performance network links 109 , such as 10 Gbps Ethernet.
- the blazing storage cluster 105 also comprises a high speed switch 103 .
- Each blazing storage node 101 is operatively coupled to the high speed switch 103 through a high speed, RDMA capable link 108 .
- each hot storage node 111 is coupled to a high speed switch 113 through a high speed, RDMA capable, link 108
- each warm storage node 121 is coupled to the high speed switch 123 through a high speed, RDMA capable, link 108 .
- the high speed switches 103 , 113 , 123 coupled to each storage cluster 105 , 115 , 125 are each coupled to the high speed switch 133 of the index cluster 135 by a high speed, RDMA capable, link 108 .
- a data storage node 200 includes an RDMA capable network interface card 206 which communicates with other devices over an RDMA capable network 210 .
- the network interface card 206 receives data directly into a pinned memory buffer 202 .
- a data store application 204 directly accesses the pinned memory buffer 202 .
- the data store app 204 manages the pinned memory buffer 202 ; in particular, the data store application 204 will utilize a management algorithm and structure, such as a queue 220 to manage data.
- an index node 300 includes an RDMA capable network interface card 306 , which communicates with other devices over an RDMA capable network 210 .
- the network interface card 306 receives data directly into a pinned memory buffer 302 .
- An index app 304 directly accesses the pinned memory buffer 302 , which can be managed as a queue 320 .
- Each entry in the queues 220 , 320 will be a fixed memory size, such as, for example, 4 kilobytes.
- the size of a queue entry will be set to the maximal size of any network message expected to be passed between nodes.
- the corresponding application database or index
- the application directly operates on the information without copying it.
- a queue pointer is advanced, so that the no longer needed data can be overwritten.
- the application directly manages the RDMA buffer. This is in contrast to prior art systems, where the RDMA buffers are managed by a driver, and accessed through the operating system.
- step 331 the application 204 coordinates with a NIC 206 to allocate a pinned memory buffer 202 .
- the application 204 allocates the pinned buffer 202 into a managed queue 220 comprising multiple fixed size buffers.
- step 333 the NIC 206 receives data from the network (not shown) resulting in a filled memory buffer 220 A.
- step 334 the application 204 operates directly on the received data within the filled memory buffer 220 A; i.e., directly within the pinned memory buffer 202 .
- step 335 application 204 manages the queue by advancing a receive pointer to, for example, empty buffer 220 B. Accordingly, when data is received by the NIC 206 , it will be written to an empty buffer 220 B.
- step 336 application 204 recycles a no longer needed buffer, such as buffer 220 a when the application is done with it, by, for example, placing it at the tail of the queue, resulting in a recycled buffer 220 C.
- a data storage node 200 comprises a disk controller 406 .
- the disk controller 406 which can be incorporated into a microprocessor or chipset on the data storage node's 200 motherboard (not shown), or be an independent card, is coupled to storage device 410 , which can be a single drive (of any type), a RAID array, a storage area network, or any other configuration of storage.
- the disk controller 406 accesses a pinned memory buffer 402 for DMA memory access for reads.
- a computer software application 404 (which could be a data store application, index application or another type of application) directly accesses and manages the pinned memory buffer 402 .
- the application 404 manages the pinned memory buffer 402 as a queue 420 .
- Each entry in the queue 420 will be a fixed memory size, such as, for example, 4 kilobytes.
- the size of a queue entry will be set to the same as, or an integer multiple of, the page size of the storage drives used by a particular node.
- the application 404 directly operates on the data without copying it.
- a queue pointer is advanced, so that the no longer needed data can be overwritten. Accordingly, the application directly manages the DMA buffer. This is in contrast to prior art DBMS systems, where DMA buffers are managed by a driver, and accessed through the operating system.
- step 431 an application 404 allocates a pinned memory buffer 402 .
- step 432 the application 404 allocates the pinned buffer 402 into a managed queue 420 comprising multiple fixed size buffers.
- step 433 the disk controller 406 reads data from storage (not shown) resulting in a filled memory buffer 420 A.
- step 434 the application 404 operates directly on the read data within the filled memory buffer 420 A; i.e., directly within the pinned memory buffer 402 .
- the application 404 could calculate an arithmetic mean of the data in the pinned memory buffer, or some other operation.
- step 435 application 404 manages the queue by advancing a read pointer to, for example, empty buffer 420 B. Accordingly, when data is next read by the disk controller 406 , it will be written to an empty buffer 420 B.
- step 436 application 404 recycles a no longer needed buffer, such as buffer 420 A when the application is done with it, by, for example, placing it at the tail of the queue, resulting in a recycled buffer 420 C.
- each row comprises 64 bits, which assumes that the underlying processor is a 64 bit processor. It should be noted that this choice was made solely for purposes of illustration, and not to limit the application of this disclosure to 64 bit processors.
- the first row of the disclosed prior art data structure 500 comprises a header 501 , and the second row comprises two longword entries 502 , 503 , where a longword data entry is understood to be 32 bits.
- the third row comprises two word entries 504 , 505 , where a word entry is understood to be 16 bits, and a longword entry 506 .
- the fourth row comprises eight byte entries 507 , 508 , 509 , 510 , 511 , 512 , 513 , 514 , and the fifth row comprises sixty four bit entries 515 .
- a 64 bit alignment a 128 bit alignment, or even a 256 bit alignment could be preferred.
- the prior art data structure 500 efficiently stores 78 data entries in a mere 320 bits; in fact, not a single bit in the prior art data structure 500 is unused.
- each piece of data in the prior art data structure with the exception of the header will require multiple operations prior to being used. For example, long word 503 must be copied to a separate location, and the upper 32 bits masked off prior to being copied into a register and operated on. Each of these operations will use precious processor time in the interest of conserving memory, network bandwidth, and disk storage.
- each data entry is stored in a 64 bit memory location, as depicted in a simplified fashion in FIG. 10 .
- This allows for data to be operated on immediately by the processor without any additional operations. While an enormous amount of memory will be used—4992 bits instead of the 320 bits that the prior art data structure utilizes to store the same data—many fewer processor cycles will be required to operate on the same data, as all of it is properly aligned to be operated on by the processor.
- each data field will be copied directly to a processor register, operated on, and copied back—no format translation is required.
- a drive access class 602 provides storage access services to an application (not shown), into which it is incorporated or accessible.
- the drive access class 602 includes an NVME drive access class 604 , a SATA drive access class 606 , and, an NVRAM drive access class 608
- FIG. 12 further describes the disclosed drive abstraction and its novel interface with the disclosed DBMS system.
- a pinned memory buffer 702 is directly accessed by a DBMS application 704 , which could be a database application, an index store application, or another type of application.
- the application incorporates the drive access class 602 .
- the drive access class 602 utilizes the NVME drive access class 604 , which directly accesses a NVME controller 706 .
- the NVME controller 706 is coupled to the NVME drive 710 , which it controls.
- the drive access class 602 when the application 704 seeks to access a SATA drive 712 , the drive access class 602 utilizes the SATA drive access class 606 , which directly accesses a SATA controller 708 .
- the SATA controller 708 is coupled to the SATA drive 712 , which it controls.
- the drive access class 602 when the application 704 seeks to access an NVRAM Drive 714 , the drive access class 602 utilizes an NVRAM drive access class 608 , which directly access a NVRAM controller 709 .
- Storage access services include opening a file, reading a file, or writing a file.
- Such services are provided through an operating system, which utilizes a device specific driver to interface with the particular device. Operating system calls are time consuming, and thereby decrease the performance of DBMS systems that utilize them. Rather than suffering such performance penalties, the disclosed DBMS system directly accesses NVRAM, NVME and SATA controllers, as disclosed herein.
Abstract
Description
- This application claims the benefit and priority of U.S. Patent Application No. 63/403,328, entitled “APPLICATION DIRECT ACCESS TO NETWORK RDMA MEMORY,” filed Oct. 3, 2016, which is hereby incorporated by reference in its entirety. This application also claims the benefit and priority of U.S. Patent Application No. 62/403,231, entitled “HIGHLY PARALLEL DATABASE MANAGEMENT SYSTEM,” filed Oct. 3, 2016, which is hereby incorporated by reference in its entirety.
- The present disclosure generally relates to networked database management systems (DBMS) and supporting infrastructure. More particularly, the present disclosure relates to computer software application access to resources, such as memory and disk. More particularly still, the present disclosure relates to efficient access by a database application to memory and storage and a database network topology to support the same. Even more particularly still, the present disclosure relates to efficiently aligned direct access to network and disk structures by a database application, for example, accessing NVME (Non-Volatile Memory Express) or SATA (Serial Advanced Technology Attachment) directly through a kernel bypass, as well as a computer software architecture to accomplish the same.
- A DBMS is a suite of computer programs that are designed to manage a database, which is a large set of structured data. In particular, a DBMS is designed to quickly access and analyze data on large amounts of stored data. Most modern DBMS systems comprise multiple computers (nodes). The nodes generally communicate via a network, which will use a network protocol, such as HTTP, or raw TCP/IP. Information that is exchange between nodes is exchanged by packets, the specific format of which will be determined by the specific protocol used by the network. The data wrapped in the packet will generally be compressed to the greatest extent possible to preserve network bandwidth. Accordingly, when it has been received, it will have to be formatted for use by the receiving node. A variety of DBMSs and the underlying infrastructure to support them are well known in the art. Database input/output (“I/O”) systems comprise processes and threads that identify, read, and write blocks of data from storage; e.g., spinning magnetic disk drives, network storage, FLASH drives, or cloud storage.
- Like many software systems, DBMS evolved from standalone computers, to sophisticated client/server setups, to cloud systems. An example of a cloud based DBMS is depicted in
FIG. 1 . In particular, acloud system 2 will generally comprise a variety of nodes (computers) as well as software that operates on the nodes. Thecloud system 2 will comprise numerous separate nodes, includingmultiple database servers 1. Each database server will maintain separate storage (not depicted), which will store some part of the maintained data. Various clients can access thecloud system 2 through the Internet 4. Clients can include, for example, a standard desktop orlaptop computer 6, amobile device 7, as well asvarious sensors 8 andcontrol equipment 9. - Generally, DBMSs operate on computer systems (whether standalone, client/server, or cloud) that incorporate operating systems. Operating systems, which are usually designed to work across a wide variety of hardware, utilize device drivers to abstract the particular functions of hardware components, such as, for example, disk controllers, and network interface cards. As drivers are generally accessed through an operating system, such accesses will typically entail significant resource overhead such as a mode switch; i.e., a switch from executing application logic to operating system logic, or a context switch; i.e., the pausing of one task to perform another. Such switches are typically time consuming; sometimes on the order of milliseconds of processor time.
- Data stored in a DBMS is usually stored redundantly, using, for example, a RAID controller, Storage Area Network (“SAN”) system, or dispersed data storage. In addition, other measures to ensure that data is stored correctly are usually taken as well. For example, many DBMSs utilize a write log. A write log, which is generally written before the actual database is updated, contains a record of all changes to the database, so that a change can be easily backed out, or, in case of a transaction processing failure, can be redone as needed. In addition, writing to the log prior to committing the data guarantees that committed transactions can be preserved; i.e., properly written to disk. Using prior art methods, disk commits are lengthy procedures, often taking milliseconds. In addition, prior art systems utilizing traditional disk systems must write a complete block, and disk log records will rarely occupy a multiple of a block of data. Given that writing a block of data can be lengthy, prior art database systems generally buffer the log and write it to disk only periodically. Otherwise, if the log were written after each modification, the DBMS would be severely limited in transaction processing speed.
- The process of buffering log writes—sometimes known as “boxcarring”—can reduce the number of transactions that a system must track and commit. However, there are penalties in user response time, lock contention, and memory usage. In addition, boxcarring can complicate system recovery.
- In certain cases, a DBMS can use Remote Direct Memory Access (RDMA) to transfer data between two nodes (computers) of the DBMS. RDMA is a technology that allows a network interface card (NIC) to transfer data directly to or from memory of a remote node without occupying the central processing unit (CPU) of either node. It should be noted that the term network interface card, as used herein, includes all network interfaces, including chips, and units built directly into processors. By way of example, a remote client can register a target memory buffer and send a description of the registered memory buffer to the storage server. The remote client then issues a read or write request to the storage server. If the request is a write request, the storage server performs an RDMA read to load data from the target memory buffer into the storage server's local memory. The storage server then causes a disc controller to write the target data to a storage disk and, once the write is complete, generates and sends a write confirmation message to the remote client. On the other hand, if the request is a read request, the storage server uses the disk controller to perform a block-level read from disk and loads the data into its local memory. The storage server then performs an RDMA write to place the data directly into an application memory buffer of the remote computer. After an RDMA operation completes, the remote client deregisters the target memory buffer from the RDMA network to prevent further RDMA accesses. Using RDMA increases data throughput, decreases the latency of data transfers, and reduces load on the storage server and remote client's CPU during data transfers. Examples of RDMA capable networks include Infiniband, iWarp, RoCE (RDMA over Converged Ethernet), and OmniPath.
-
FIG. 2 depicts a block representation of a conventional system in which data is copied from a pinned buffer in a first node to a second node. Thefirst node 10 includes apinned buffer 20, adriver 30, and aNIC 40. The pinnedbuffer 20 and thedriver 30 are each operatively coupled to theNIC 40. Thesecond node 50 includes apinned buffer 60, adriver 70, and a NIC 80. The pinnedbuffer 60 and thedriver 70 are coupled to the NIC 80. The NIC 40 is operatively coupled to the NIC 80 via anetwork 90. Thedriver respective node - In operation, the
driver 30 in thenode 10 writes a descriptor for a location of thepinned buffer 20 to the NIC 40. Thedriver 70 in thenode 50 writes a descriptor for the location of thepinned buffer 60 to the NIC 80. Thedriver 30 works with the operating system, as well as other software and hardware on thenode 10 to guarantee that thebuffer 20 is locked into physical memory; i.e., “pinned.” The NIC 40 reads data from thepinned buffer 20 and sends the read data on thenetwork 90. Thenetwork 90 passes the data to the NIC 80 of thehost 50. TheNIC 80 writes data to the pinnedbuffer 60. -
FIG. 3 is a sequence of steps performed by a prior art network RDMA system to receive data over a network. In the first step 91 aNIC 40 allocates a pinnedmemory buffer 20. Instep 92 data is received from the network (not shown) by theNIC 40 into the pinnedmemory buffer 20. In step 93 a driver notifies anapplication 35 that data has been received. Instep 94, theapplication 35 calls into the driver 30 (usually through an operating system, which is not depicted) to read the data in the pinnedbuffer 20. Instep 95, thedriver 30 copies the data in the pinnedbuffer 20 to a separate memory buffer (not depicted) allocated by theapplication 35. - Serial Advanced Technology Attachment (SATA) is a computer bus interface that connects host bus adapters to mass storage devices such as hard drives, optical drives, and solid state drives (SSDs). While SATA works with SSDs, it is not designed to allow for the significant level of parallelism that SSDs are capable of. NVMe, NVM (Non-Volatile Memory) Express, or Non-Volatile Memory Host Controller Interface Specification (NVMHCI) is a logical device interface specification for accessing non-volatile storage media attached via PCI Express (PCIe) bus. NVMe allows parallel access to modern Solid State Drives (SSDs) to be effectively utilized by node hardware and software.
- Microprocessor architectures generally operate on data types of a fixed width, as the registers within the microprocessor will all be of a fixed width. For example, many modern processors include either 32 bit wide or 64 bit wide registers. In order to achieve maximal efficiency, prior to operating on data, the data must be properly aligned in memory along boundaries defined by that fixed width.
- Storage and network bandwidth are substantial drivers of cost for DBMS systems. Accordingly, prior art DBMS systems tend to optimize data handling to conserve storage and network bandwidth. For example, a prior art DBMS system may include a data manipulation step that effectively compresses data prior to storing it or transmitting it via the network, and a decompression step when reading data from storage or the network.
- Accordingly, it is an object of this disclosure to provide an infrastructure for a DBMS, and an apparatus and method that operates more efficiently than prior art systems.
- Another object of the disclosure is to provide an efficient network infrastructure.
- Another object of the disclosure is to provide a network infrastructure for a DBMS that allows a database application to directly access pinned RDMA memory.
- Another object of the disclosure is to provide a network infrastructure for a DBMS that manages pinned RDMA memory.
- Another object of the disclosure is to provide a storage infrastructure for a DBMS that allows a database application to directly access storage buffers used for disk access.
- Another object of the disclosure is to provide a storage infrastructure for a DBMS that that manages pinned DMA memory.
- Another object of the disclosure is to provide a DBMS infrastructure that allows an application to directly access NVME drives.
- Another object of the disclosure is to provide a DBMS infrastructure that allows an application to directly access SATA drives.
- Another object of the disclosure is to provide a DBMS infrastructure whereby data is stored in a format that is usable by a processor with minimal adjustment.
- Another object of the disclosure is to provide a DBMS infrastructure whereby network data is maintained in a format that is usable by a processor with minimal adjustment.
- Other advantages of this disclosure will be clear to a person of ordinary skill in the art. It should be understood, however, that a system or method could practice the disclosure while not achieving all of the enumerated advantages, and that the protected disclosure is defined by the claims.
- A networked database management system along with the supporting infrastructure is disclosed. The disclosed DBMS is capable of handling enormous amounts of data—an Exabyte or more—and accessing each record within the database frequently. In one embodiment the disclosed DBMS comprises a first high speed storage cluster including a first plurality of storage nodes. Each storage node includes a server and one or more storage drives at a high performance level. The first high speed storage cluster also includes a first switch. The DBMS also comprises a second high speed storage cluster including a second plurality of storage nodes. Each storage node in this second cluster also includes a server and one or more storage drives that operate at a lower performance level. The second high speed storage cluster also includes a second switch. The DBMS allows comprises an index cluster including a plurality of index nodes and a third switch. The first switch is operatively coupled to the third switch by a high speed RDMA capable link, and the second switch is operatively coupled to the third switch by a high speed RDMA capable link.
- In a separate embodiment, the disclosed DBMS includes an application with a drive access class. The drive access class includes a NVME drive access sub class, and a SATA drive access sub class. The NVME drive access sub class allows the application to directly interface with NVME drives, and the SATA drive access sub class allows the application to directly interface with SATA drives. In addition, NVRAM technologies are now viable, and a specific sub class is allocated to optimize access to drives utilizing NVRAM technology. As future storage technologies are introduced, additional optimized storage access subclasses can be developed.
- In another embodiment, the disclosed DBMS incorporates a node. The node includes a network interface card including a controller. A DBMS application executing on the node coordinates with the controller to allocate a pinned memory buffer, which is directly accessed by the DBMS application, which allows the DBMS application to directly access and manipulate received network data.
- In another embodiment, the disclosed DBMS incorporates a node. The node includes a drive controller. A DBMS application allocates a pinned memory buffer, and communicates this buffer to the drive controller, which allows the DBMS application to access and manipulate drive data with minimal overhead.
- In yet another embodiment, a node in a massively parallel database management system includes a processor having a preferred data alignment. An application operates on the processor, and creates a pinned memory buffer. The application also creates a data structure having a plurality of fields, wherein each of the plurality of fields is aligned on the preferred data alignment of the processor. The application can then insert data into the data structure without any performance degradation as explained herein.
- In a further refinement of this embodiment, the pinned memory buffer can be an RDMA memory buffer. The node can further comprise a network interface card, and the application can communicate the data structure to a remote node using the network interface card. In addition, the network interface card can receive data over a network from a remote node and write the data into the RDMA memory buffer. The data received over the network can be a second data structure (which may be identical to the first) having a plurality of fields wherein each of the fields is aligned on the preferred data alignment of the processor. The node can further comprise a disk controller, such as a SATA controller or an NVME controller, which can be used to write the received data to disk.
- Although the characteristic features of this disclosure will be particularly pointed out in the claims, the invention itself, and the manner in which it may be made and used, may be better understood by referring to the following description taken in connection with the accompanying drawings forming a part hereof, wherein like reference numerals refer to like parts throughout the several views and in which:
-
FIG. 1 depicts a prior art simplified network diagram of a cloud based DBMS system; -
FIG. 2 depicts a prior art simplified block diagram of a network based RDMA system; -
FIG. 3 depicts a sequence of steps that a prior art network based RDMA system would execute to receive data over a network; -
FIG. 4 depicts a simplified network diagram illustrating the disclosed DBMS system and the infrastructure supporting the same; -
FIG. 5 depicts a simplified block diagram of a network implementing the disclosed RDMA memory buffer management system; -
FIG. 6 depicts a sequence of steps that a node utilizing the disclosed RDMA memory buffer system will execute to receive data over a network; -
FIG. 7 depicts a simplified block diagram of a node implementing the disclosed DMA memory buffer management system; -
FIG. 8 depicts a sequence of steps that a node utilizing the disclosed DMA memory buffer management system will execute to read data from storage; -
FIG. 9 depicts a prior art data structure for data passed by network or stored on disk; -
FIG. 10 depicts a disclosed data structure for data passed by network or stored on disk; -
FIG. 11 depicts a simplified class abstraction for a drive access class; and -
FIG. 12 depicts a simplified software block diagram of the disclosed DBMS system implementing the disclosed drive access class. - This application discloses a number of infrastructure improvements for use in a database system, along with a DBMS utilizing those improvements. The infrastructure is adapted to allow the database system to scale to a size of an Exabyte or even larger, and allow each record stored in the database to be accessed quickly and numerous times per day. Such a database will be useful for many tasks; for example, every bit of data exchanged over a company's network can be stored in such a database and analyzed to determine, for example, the mechanism of a network intrusion after it has been discovered.
- One issue that such a database must overcome is data access latency. There are numerous sources of data access latency, including, for example, network accesses, disk accesses, memory copies, etc. These latencies are exacerbated from distributed systems.
- Turning to the Figures, and to
FIG. 4 in particular, an embodiment of anetwork architecture 100 for supporting a high performance database system is disclosed. Thenetwork architecture 100 comprises three storage clusters; a blazingstorage cluster 105, ahot storage cluster 115, and awarm storage cluster 125. In addition, the network architecture comprises anindex cluster 135. Each cluster comprises multiple nodes. For example, the blazingstorage cluster 105 is depicted as comprising fiveblazing storage nodes 101; thehot storage cluster 115 is depicted as comprising fivehot storage nodes 111, and thewarm storage cluster 125 is depicted as comprising fivewarm storage nodes 121. In addition, theindex cluster 135 is depicted as comprising threeindex nodes 131. While theclusters cluster - A blazing
storage node 101 may include, for example, an array of NVDIMM (Non-Volatile Dual Inline Memory Module)(a type of NVRAM) storage, such as that marketed by Hewlett Packard Enterprise, or any other extremely fast storage, along with appropriate controllers to allow for full speed access to such storage. For example, DRAM with a write ahead log implemented on an NVMe drive could be utilized to implement blazing storage. Specifically, write-ahead logging is used to log updates to an in-memory (DRAM) data structure. As log entries are appended to the end of the log, they are flushed to an NVMe drive when the size of in-memory log entries is near or exceeds the size of a Solid State Memory page, or after a configured timeout threshold, such as ten seconds, has been reached. This guarantees an upper bound for the amount of data that is lost if a power outage or system crash should occur; this also allows for the entire in-memory structure from disk log to be rebuilt after a restart. Since the log is written sequentially to the SSD, write amplification on the Solid State drive is minimized. - In addition, NVRAM technology can also be utilized to implement blazing storage. In such a case, the in-memory structures would be preserved in the event of a power outage. A
hot storage node 111 may include, for example, one or more Solid State NVME drives, along with appropriate controllers to allow for full speed access to such storage. Awarm storage node 121 may include, for example, one or more Solid State SATA drives, along with appropriate controllers to allow for full speed access to such storage. - Each
index node 131 will also include storage, which will generally comprise high performance storage such as Solid State SATA drives or higher performance storage devices. Generally, theindex nodes 131 will store the relational database structure, which may comprise, for example, a collection of tables and search keys. - To allow for information exchange as fast as possible, certain of the clusters are connected via high speed, RDMA
capable links 108. In particular, theindex cluster 135 is connected to thestorage clusters capable links 108. On the other hand, thestorage clusters performance network links 109, such as 10 Gbps Ethernet. - As discussed above, Infiniband is an example of a high speed, RDMA capable link. Importantly, such links allow different nodes in each cluster to exchange information rapidly; as discussed above, information from one node is inserted into the memory of another node without consuming processor cycles of the target node. The blazing
storage cluster 105 also comprises ahigh speed switch 103. Each blazingstorage node 101 is operatively coupled to thehigh speed switch 103 through a high speed, RDMAcapable link 108. Similarly eachhot storage node 111 is coupled to ahigh speed switch 113 through a high speed, RDMA capable, link 108, and eachwarm storage node 121 is coupled to thehigh speed switch 123 through a high speed, RDMA capable, link 108. Similarly, the high speed switches 103,113,123 coupled to eachstorage cluster high speed switch 133 of theindex cluster 135 by a high speed, RDMA capable, link 108. - Turning to
FIG. 5 , the specific RDMA memory infrastructure used by the disclosed database infrastructure is depicted. In particular, adata storage node 200 includes an RDMA capablenetwork interface card 206 which communicates with other devices over an RDMAcapable network 210. Thenetwork interface card 206 receives data directly into a pinnedmemory buffer 202. Adata store application 204 directly accesses the pinnedmemory buffer 202. In particular, thedata store app 204 manages the pinnedmemory buffer 202; in particular, thedata store application 204 will utilize a management algorithm and structure, such as aqueue 220 to manage data. - Similarly, an
index node 300 includes an RDMA capablenetwork interface card 306, which communicates with other devices over an RDMAcapable network 210. Thenetwork interface card 306 receives data directly into a pinnedmemory buffer 302. Anindex app 304 directly accesses the pinnedmemory buffer 302, which can be managed as aqueue 320. - Each entry in the
queues - The steps by which network data is received by the disclosed DBMS system is illustrated in
FIG. 6 . Instep 331 theapplication 204 coordinates with aNIC 206 to allocate a pinnedmemory buffer 202. Next, instep 332, theapplication 204 allocates the pinnedbuffer 202 into a managedqueue 220 comprising multiple fixed size buffers. Instep 333, theNIC 206 receives data from the network (not shown) resulting in a filledmemory buffer 220A. In step 334, theapplication 204 operates directly on the received data within the filledmemory buffer 220A; i.e., directly within the pinnedmemory buffer 202. Instep 335,application 204 manages the queue by advancing a receive pointer to, for example,empty buffer 220B. Accordingly, when data is received by theNIC 206, it will be written to anempty buffer 220B. Instep 336,application 204 recycles a no longer needed buffer, such as buffer 220 a when the application is done with it, by, for example, placing it at the tail of the queue, resulting in arecycled buffer 220C. - Turning to
FIG. 7 , a specific disk buffer management scheme for use with the disclosed DBMS is depicted. In particular, adata storage node 200 comprises adisk controller 406. Thedisk controller 406, which can be incorporated into a microprocessor or chipset on the data storage node's 200 motherboard (not shown), or be an independent card, is coupled tostorage device 410, which can be a single drive (of any type), a RAID array, a storage area network, or any other configuration of storage. Thedisk controller 406 accesses a pinnedmemory buffer 402 for DMA memory access for reads. A computer software application 404 (which could be a data store application, index application or another type of application) directly accesses and manages the pinnedmemory buffer 402. In particular, theapplication 404 manages the pinnedmemory buffer 402 as aqueue 420. - Each entry in the
queue 420 will be a fixed memory size, such as, for example, 4 kilobytes. In a preferred embodiment of the disclosed DBMS and associated infrastructure, the size of a queue entry will be set to the same as, or an integer multiple of, the page size of the storage drives used by a particular node. As data is read fromstorage 410 theapplication 404 directly operates on the data without copying it. As data is used, and no longer needed, a queue pointer is advanced, so that the no longer needed data can be overwritten. Accordingly, the application directly manages the DMA buffer. This is in contrast to prior art DBMS systems, where DMA buffers are managed by a driver, and accessed through the operating system. - The steps by which data is read by a node from storage into memory for the disclosed DBMS system is set forth in
FIG. 8 . Instep 431 anapplication 404 allocates a pinnedmemory buffer 402. Next, instep 432, theapplication 404 allocates the pinnedbuffer 402 into a managedqueue 420 comprising multiple fixed size buffers. Instep 433, thedisk controller 406 reads data from storage (not shown) resulting in a filledmemory buffer 420A. Instep 434, theapplication 404 operates directly on the read data within the filledmemory buffer 420A; i.e., directly within the pinnedmemory buffer 402. For example, theapplication 404 could calculate an arithmetic mean of the data in the pinned memory buffer, or some other operation. Instep 435,application 404 manages the queue by advancing a read pointer to, for example,empty buffer 420B. Accordingly, when data is next read by thedisk controller 406, it will be written to anempty buffer 420B. In step 436,application 404 recycles a no longer needed buffer, such asbuffer 420A when the application is done with it, by, for example, placing it at the tail of the queue, resulting in arecycled buffer 420C. - While the applications of the disclosed DBMS system directly managing RDMA buffers will substantially improve performance of the DBMS system versus prior art implementations, additional improvements can still be made. In particular, prior art systems typically maintain data in packed, or compressed, format. For example, to conserve disk storage and network bandwidth, most systems will pack Boolean type data into a single bit. In addition, many systems will actually apply simple compression before transmitting data via a network or committing it to disk storage. Prior art systems do this to ensure that network bandwidth and persistent storage—both of which are scarce resources—are used efficiently.
- An example of a prior
art data structure 500 is shown inFIG. 9 . As depicted, each row comprises 64 bits, which assumes that the underlying processor is a 64 bit processor. It should be noted that this choice was made solely for purposes of illustration, and not to limit the application of this disclosure to 64 bit processors. The first row of the disclosed priorart data structure 500 comprises aheader 501, and the second row comprises twolongword entries word entries longword entry 506. The fourth row comprises eightbyte entries bit entries 515. As opposed to a 64 bit alignment, a 128 bit alignment, or even a 256 bit alignment could be preferred. - The prior
art data structure 500 efficiently stores 78 data entries in a mere 320 bits; in fact, not a single bit in the priorart data structure 500 is unused. However, each piece of data in the prior art data structure with the exception of the header will require multiple operations prior to being used. For example,long word 503 must be copied to a separate location, and the upper 32 bits masked off prior to being copied into a register and operated on. Each of these operations will use precious processor time in the interest of conserving memory, network bandwidth, and disk storage. - The disclosed DBMS system, however, is not optimized to minimize network usage or disk storage. Rather, the disclosed DBMS system is optimized to maximize performance. Accordingly, each data entry is stored in a 64 bit memory location, as depicted in a simplified fashion in
FIG. 10 . This allows for data to be operated on immediately by the processor without any additional operations. While an enormous amount of memory will be used—4992 bits instead of the 320 bits that the prior art data structure utilizes to store the same data—many fewer processor cycles will be required to operate on the same data, as all of it is properly aligned to be operated on by the processor. In particular, each data field will be copied directly to a processor register, operated on, and copied back—no format translation is required. - Another way in which the disclosed DBMS system optimizes performance is by the various DBMS applications directly accessing NVME, NVRAM and SATA drives. This is done through an abstraction layer, which is generally illustrated in
FIG. 11 . In particular, adrive access class 602 provides storage access services to an application (not shown), into which it is incorporated or accessible. Thedrive access class 602 includes an NVMEdrive access class 604, a SATAdrive access class 606, and, an NVRAMdrive access class 608 -
FIG. 12 further describes the disclosed drive abstraction and its novel interface with the disclosed DBMS system. In particular, a pinnedmemory buffer 702 is directly accessed by aDBMS application 704, which could be a database application, an index store application, or another type of application. The application incorporates thedrive access class 602. When theapplication 704 seeks to access anNVME drive 710, thedrive access class 602 utilizes the NVMEdrive access class 604, which directly accesses aNVME controller 706. TheNVME controller 706 is coupled to theNVME drive 710, which it controls. On the other hand, when theapplication 704 seeks to access aSATA drive 712, thedrive access class 602 utilizes the SATAdrive access class 606, which directly accesses aSATA controller 708. TheSATA controller 708 is coupled to theSATA drive 712, which it controls. And, when theapplication 704 seeks to access anNVRAM Drive 714, thedrive access class 602 utilizes an NVRAMdrive access class 608, which directly access aNVRAM controller 709. - Storage access services include opening a file, reading a file, or writing a file. Generally, such services are provided through an operating system, which utilizes a device specific driver to interface with the particular device. Operating system calls are time consuming, and thereby decrease the performance of DBMS systems that utilize them. Rather than suffering such performance penalties, the disclosed DBMS system directly accesses NVRAM, NVME and SATA controllers, as disclosed herein.
- The foregoing description of the disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. The description was selected to best explain the principles of the present teachings and practical application of these principles to enable others skilled in the art to best utilize the disclosure in various embodiments and various modifications as are suited to the particular use contemplated. It is intended that the scope of the disclosure not be limited by the specification, but be defined by the claims set forth below. In addition, although narrow claims may be presented below, it should be recognized that the scope of this invention is much broader than presented by the claim(s). It is intended that broader claims will be submitted in one or more applications that claim the benefit of priority from this application. Insofar as the description above and the accompanying drawings disclose additional subject matter that is not within the scope of the claim or claims below, the additional inventions are not dedicated to the public and the right to file one or more applications to claim such additional inventions is reserved.
Claims (7)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/722,687 US20180095996A1 (en) | 2016-10-03 | 2017-10-02 | Database system utilizing forced memory aligned access |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201662403328P | 2016-10-03 | 2016-10-03 | |
US201662403231P | 2016-10-03 | 2016-10-03 | |
US15/722,687 US20180095996A1 (en) | 2016-10-03 | 2017-10-02 | Database system utilizing forced memory aligned access |
Publications (1)
Publication Number | Publication Date |
---|---|
US20180095996A1 true US20180095996A1 (en) | 2018-04-05 |
Family
ID=61758179
Family Applications (7)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/722,968 Active 2038-06-28 US10713276B2 (en) | 2016-10-03 | 2017-10-02 | Data transition in highly parallel database management system |
US15/722,794 Abandoned US20180095914A1 (en) | 2016-10-03 | 2017-10-02 | Application direct access to sata drive |
US15/722,900 Active 2039-10-23 US11586647B2 (en) | 2016-10-03 | 2017-10-02 | Randomized data distribution in highly parallel database management system |
US15/722,687 Abandoned US20180095996A1 (en) | 2016-10-03 | 2017-10-02 | Database system utilizing forced memory aligned access |
US16/924,458 Active 2037-12-02 US11294932B2 (en) | 2016-10-03 | 2020-07-09 | Data transition in highly parallel database management system |
US17/652,261 Active US11934423B2 (en) | 2016-10-03 | 2022-02-23 | Data transition in highly parallel database management system |
US18/170,225 Pending US20230195749A1 (en) | 2016-10-03 | 2023-02-16 | Randomized data distribution in highly parallel database management system |
Family Applications Before (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/722,968 Active 2038-06-28 US10713276B2 (en) | 2016-10-03 | 2017-10-02 | Data transition in highly parallel database management system |
US15/722,794 Abandoned US20180095914A1 (en) | 2016-10-03 | 2017-10-02 | Application direct access to sata drive |
US15/722,900 Active 2039-10-23 US11586647B2 (en) | 2016-10-03 | 2017-10-02 | Randomized data distribution in highly parallel database management system |
Family Applications After (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/924,458 Active 2037-12-02 US11294932B2 (en) | 2016-10-03 | 2020-07-09 | Data transition in highly parallel database management system |
US17/652,261 Active US11934423B2 (en) | 2016-10-03 | 2022-02-23 | Data transition in highly parallel database management system |
US18/170,225 Pending US20230195749A1 (en) | 2016-10-03 | 2023-02-16 | Randomized data distribution in highly parallel database management system |
Country Status (2)
Country | Link |
---|---|
US (7) | US10713276B2 (en) |
WO (2) | WO2018067471A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180150220A1 (en) * | 2016-11-25 | 2018-05-31 | Samsung Electronics Co., Ltd. | System and method for improving storage device i/o performance |
US11240178B2 (en) * | 2019-04-16 | 2022-02-01 | Realtek Semiconductor Corporation | Data transmission method and data transmission system |
Families Citing this family (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11188541B2 (en) * | 2016-10-20 | 2021-11-30 | Industry Academic Cooperation Foundation Of Yeungnam University | Join method, computer program and recording medium thereof |
US10346315B2 (en) | 2017-05-26 | 2019-07-09 | Oracle International Corporation | Latchless, non-blocking dynamically resizable segmented hash index |
US10803039B2 (en) * | 2017-05-26 | 2020-10-13 | Oracle International Corporation | Method for efficient primary key based queries using atomic RDMA reads on cache friendly in-memory hash index |
US10956335B2 (en) | 2017-09-29 | 2021-03-23 | Oracle International Corporation | Non-volatile cache access using RDMA |
US11347678B2 (en) | 2018-08-06 | 2022-05-31 | Oracle International Corporation | One-sided reliable remote direct memory operations |
WO2020055921A1 (en) | 2018-09-10 | 2020-03-19 | GigaIO Networks, Inc. | Methods and apparatus for high-speed data bus connection and fabric management |
KR102241126B1 (en) | 2019-03-03 | 2021-04-19 | 주식회사 나오팩 | Automatic packing apparatus for ice pack |
US11403247B2 (en) | 2019-09-10 | 2022-08-02 | GigaIO Networks, Inc. | Methods and apparatus for network interface fabric send/receive operations |
US11500856B2 (en) | 2019-09-16 | 2022-11-15 | Oracle International Corporation | RDMA-enabled key-value store |
US11593288B2 (en) * | 2019-10-02 | 2023-02-28 | GigalO Networks, Inc. | Methods and apparatus for fabric interface polling |
EP4049143A4 (en) | 2019-10-25 | 2024-02-21 | Gigaio Networks Inc | Methods and apparatus for dma engine descriptors for high speed data systems |
CN111404931B (en) * | 2020-03-13 | 2021-03-30 | 清华大学 | Remote data transmission method based on persistent memory |
US11816342B2 (en) * | 2020-06-10 | 2023-11-14 | Samsung Electronics Co., Ltd. | Systems and methods for distributed in-storage computation-conscious erasure coding |
US11941027B2 (en) * | 2022-05-18 | 2024-03-26 | Sap Se | Scale-out of database systems in cloud computing environments |
Family Cites Families (98)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA2094410C (en) | 1992-06-18 | 1998-05-05 | Joshua Seth Auerbach | Distributed management communications network |
US5548770A (en) | 1993-02-25 | 1996-08-20 | Data Parallel Systems, Inc. | Method and apparatus for improving retrieval of data from a database |
US5742806A (en) | 1994-01-31 | 1998-04-21 | Sun Microsystems, Inc. | Apparatus and method for decomposing database queries for database management system including multiprocessor digital data processing system |
US5636359A (en) * | 1994-06-20 | 1997-06-03 | International Business Machines Corporation | Performance enhancement system and method for a hierarchical data cache using a RAID parity scheme |
US5764905A (en) * | 1996-09-09 | 1998-06-09 | Ncr Corporation | Method, system and computer program product for synchronizing the flushing of parallel nodes database segments through shared disk tokens |
US6253228B1 (en) | 1997-03-31 | 2001-06-26 | Apple Computer, Inc. | Method and apparatus for updating and synchronizing information between a client and a server |
US6477527B2 (en) | 1997-05-09 | 2002-11-05 | International Business Machines Corporation | System, method, and program for object building in queries over object views |
US5933840A (en) * | 1997-05-19 | 1999-08-03 | International Business Machines Corporation | Garbage collection in log-structured information storage systems using age threshold selection of segments |
US6230200B1 (en) | 1997-09-08 | 2001-05-08 | Emc Corporation | Dynamic modeling for resource allocation in a file server |
US6374336B1 (en) * | 1997-12-24 | 2002-04-16 | Avid Technology, Inc. | Computer system and process for transferring multiple high bandwidth streams of data between multiple storage units and multiple applications in a scalable and reliable manner |
US6732111B2 (en) | 1998-03-03 | 2004-05-04 | Siebel Systems, Inc. | Method, apparatus, system, and program product for attaching files and other objects to a partially replicated database |
US7774469B2 (en) * | 1999-03-26 | 2010-08-10 | Massa Michael T | Consistent cluster operational data in a server cluster using a quorum of replicas |
US7020695B1 (en) * | 1999-05-28 | 2006-03-28 | Oracle International Corporation | Using a cluster-wide shared repository to provide the latest consistent definition of the cluster (avoiding the partition-in time problem) |
ATE320130T1 (en) | 1999-08-06 | 2006-03-15 | Ibm | ADDRESS MANAGEMENT IN HIERARCHICAL PNNI NETWORKS |
US6505285B1 (en) * | 2000-06-26 | 2003-01-07 | Ncr Corporation | Scratch segment subsystem for a parallel processing database system |
WO2002017210A2 (en) | 2000-08-18 | 2002-02-28 | Cygnus, Inc. | Formulation and manipulation of databases of analyte and associated values |
US20030005087A1 (en) * | 2001-05-24 | 2003-01-02 | Microsoft Corporation | Server side sampling of databases |
US7499907B2 (en) | 2001-10-12 | 2009-03-03 | Teradata Us, Inc. | Index selection in a database system |
US6993701B2 (en) * | 2001-12-28 | 2006-01-31 | Network Appliance, Inc. | Row-diagonal parity technique for enabling efficient recovery from double failures in a storage array |
US7853508B2 (en) * | 2003-05-19 | 2010-12-14 | Serena Software, Inc. | Method and system for object-oriented management of multi-dimensional data |
JP2005165702A (en) * | 2003-12-03 | 2005-06-23 | Hitachi Ltd | Device connection method of cluster storage |
EP1725946A4 (en) | 2004-03-10 | 2012-07-11 | Enterasys Networks Inc | Dynamic network detection system and method |
US7908242B1 (en) | 2005-04-11 | 2011-03-15 | Experian Information Solutions, Inc. | Systems and methods for optimizing database queries |
US7894372B2 (en) | 2005-05-31 | 2011-02-22 | Iac Search & Media, Inc. | Topology-centric resource management for large scale service clusters |
US7496589B1 (en) * | 2005-07-09 | 2009-02-24 | Google Inc. | Highly compressed randomly accessed storage of large tables with arbitrary columns |
US8010485B1 (en) * | 2005-10-20 | 2011-08-30 | American Megatrends, Inc. | Background movement of data between nodes in a storage cluster |
KR100814645B1 (en) * | 2005-12-29 | 2008-03-18 | 성균관대학교산학협력단 | Efficient Key Pre-Distribution System for Secure Distributed Sensor Networks and Method Thereof |
US7519734B1 (en) | 2006-03-14 | 2009-04-14 | Amazon Technologies, Inc. | System and method for routing service requests |
US9015146B2 (en) | 2006-12-01 | 2015-04-21 | Teradata Us, Inc. | Managing access to data in a multi-temperature database |
US7930611B2 (en) * | 2007-03-09 | 2011-04-19 | Microsoft Corporation | Erasure-resilient codes having multiple protection groups |
US7971025B2 (en) * | 2007-03-14 | 2011-06-28 | Hitachi, Ltd. | Method and apparatus for chunk allocation in a thin provisioning storage system |
JP5099128B2 (en) * | 2007-04-20 | 2012-12-12 | 富士通株式会社 | Storage management program, storage management device, and storage management method |
US8640056B2 (en) * | 2007-07-05 | 2014-01-28 | Oracle International Corporation | Data visualization techniques |
US7975109B2 (en) | 2007-05-30 | 2011-07-05 | Schooner Information Technology, Inc. | System including a fine-grained memory and a less-fine-grained memory |
US8060484B2 (en) * | 2007-08-07 | 2011-11-15 | International Business Machines Corporation | Graphical user interface for data management |
US8862625B2 (en) * | 2008-04-07 | 2014-10-14 | Teradata Us, Inc. | Accessing data in a column store database based on hardware compatible indexing and replicated reordered columns |
US7895463B2 (en) | 2007-08-28 | 2011-02-22 | Cisco Technology, Inc. | Redundant application network appliances using a low latency lossless interconnect link |
US9043801B2 (en) | 2008-01-15 | 2015-05-26 | International Business Machines Corporation | Two-tiered dynamic load balancing using sets of distributed thread pools |
US8195642B2 (en) * | 2008-04-22 | 2012-06-05 | International Business Machines Corporation | Partial indexes for multi-node database |
US7840730B2 (en) * | 2008-06-27 | 2010-11-23 | Microsoft Corporation | Cluster shared volumes |
US8229923B2 (en) | 2008-09-26 | 2012-07-24 | Microsoft Corporation | Multidimensional data space queries with expression evaluation |
US20100199036A1 (en) * | 2009-02-02 | 2010-08-05 | Atrato, Inc. | Systems and methods for block-level management of tiered storage |
US7990797B2 (en) | 2009-02-11 | 2011-08-02 | Stec, Inc. | State of health monitored flash backed dram module |
US8145838B1 (en) * | 2009-03-10 | 2012-03-27 | Netapp, Inc. | Processing and distributing write logs of nodes of a cluster storage system |
US20100241893A1 (en) | 2009-03-18 | 2010-09-23 | Eric Friedman | Interpretation and execution of a customizable database request using an extensible computer process and an available computing environment |
US9160610B1 (en) * | 2009-03-31 | 2015-10-13 | Symantec Corporation | Method and apparatus for coordinating service execution within a shared file system environment to optimize cluster performance |
US8769049B2 (en) | 2009-04-24 | 2014-07-01 | Microsoft Corporation | Intelligent tiers of backup data |
US8161035B2 (en) | 2009-06-04 | 2012-04-17 | Oracle International Corporation | Query optimization by specifying path-based predicate evaluation in a path-based query operator |
FR2950714B1 (en) | 2009-09-25 | 2011-11-18 | Bull Sas | SYSTEM AND METHOD FOR MANAGING THE INTERLEAVED EXECUTION OF INSTRUCTION WIRES |
US8712984B2 (en) * | 2010-03-04 | 2014-04-29 | Microsoft Corporation | Buffer pool extension for database server |
US9195657B2 (en) * | 2010-03-08 | 2015-11-24 | Microsoft Technology Licensing, Llc | Columnar storage of a database index |
US8886631B2 (en) * | 2010-06-04 | 2014-11-11 | Yale University | Query execution systems and methods |
US8639671B2 (en) * | 2010-06-29 | 2014-01-28 | Teradata Us, Inc. | Database compression |
CA2802219A1 (en) * | 2010-07-21 | 2012-01-26 | Mosaid Technologies Incorporated | Multipage program scheme for flash memory |
CN101916261B (en) | 2010-07-28 | 2013-07-17 | 北京播思软件技术有限公司 | Data partitioning method for distributed parallel database system |
US20120036146A1 (en) * | 2010-10-26 | 2012-02-09 | ParElastic Corporation | Apparatus for elastic database processing with heterogeneous data |
WO2012082792A2 (en) | 2010-12-13 | 2012-06-21 | Fusion-Io, Inc. | Apparatus, system, and method for auto-commit memory |
US8745329B2 (en) * | 2011-01-20 | 2014-06-03 | Google Inc. | Storing data across a plurality of storage nodes |
CN102737033B (en) | 2011-03-31 | 2015-02-04 | 国际商业机器公司 | Data processing equipment and data processing method thereof |
US20140059278A1 (en) * | 2011-11-14 | 2014-02-27 | Lsi Corporation | Storage device firmware and manufacturing software |
CN104054073B (en) * | 2011-11-15 | 2018-10-30 | 起元科技有限公司 | Data divide group, segmentation and parallelization |
US9285992B2 (en) * | 2011-12-16 | 2016-03-15 | Netapp, Inc. | System and method for optimally creating storage objects in a storage system |
US8762378B2 (en) * | 2011-12-23 | 2014-06-24 | Sap Ag | Independent table nodes in parallelized database environments |
US20130166502A1 (en) * | 2011-12-23 | 2013-06-27 | Stephen Gregory WALKAUSKAS | Segmented storage for database clustering |
US9171031B2 (en) * | 2012-03-02 | 2015-10-27 | Cleversafe, Inc. | Merging index nodes of a hierarchical dispersed storage index |
US9081855B1 (en) * | 2012-05-31 | 2015-07-14 | Integrity Applications Incorporated | Systems and methods for video archive and data extraction |
US8965921B2 (en) | 2012-06-06 | 2015-02-24 | Rackspace Us, Inc. | Data management and indexing across a distributed database |
US20140047095A1 (en) | 2012-08-07 | 2014-02-13 | Advanced Micro Devices, Inc. | System and method for tuning a cloud computing system |
US8868531B2 (en) * | 2012-09-10 | 2014-10-21 | Apple Inc. | Concurrent access methods for tree data structures |
US8862550B1 (en) * | 2012-09-29 | 2014-10-14 | Emc Corporation | Reliable in-place bootstrap metadata transformation in a shared data store |
US8972378B2 (en) * | 2012-10-22 | 2015-03-03 | Microsoft Corporation | Formulating global statistics for distributed databases |
US10114908B2 (en) | 2012-11-13 | 2018-10-30 | International Business Machines Corporation | Hybrid table implementation by using buffer pool as permanent in-memory storage for memory-resident data |
US9311354B2 (en) | 2012-12-29 | 2016-04-12 | Futurewei Technologies, Inc. | Method for two-stage query optimization in massively parallel processing database clusters |
US9819727B2 (en) * | 2013-02-28 | 2017-11-14 | Amazon Technologies, Inc. | Computing infrastructure for configurable-quality random data |
US9483431B2 (en) | 2013-04-17 | 2016-11-01 | Apeiron Data Systems | Method and apparatus for accessing multiple storage devices from multiple hosts without use of remote direct memory access (RDMA) |
US9641614B2 (en) * | 2013-05-29 | 2017-05-02 | Microsoft Technology Licensing, Llc | Distributed storage defense in a cluster |
US9417903B2 (en) * | 2013-06-21 | 2016-08-16 | International Business Machines Corporation | Storage management for a cluster of integrated computing systems comprising integrated resource infrastructure using storage resource agents and synchronized inter-system storage priority map |
US8688718B1 (en) * | 2013-07-31 | 2014-04-01 | Linkedin Corporation | Management of data segments for analytics queries |
US10223326B2 (en) | 2013-07-31 | 2019-03-05 | Oracle International Corporation | Direct access persistent memory shared storage |
US9654532B2 (en) * | 2013-09-23 | 2017-05-16 | Spotify Ab | System and method for sharing file portions between peers with different capabilities |
CA2867589A1 (en) * | 2013-10-15 | 2015-04-15 | Coho Data Inc. | Systems, methods and devices for implementing data management in a distributed data storage system |
US9170746B2 (en) * | 2014-01-07 | 2015-10-27 | Netapp, Inc. | Clustered raid assimilation management |
US9921847B2 (en) | 2014-01-21 | 2018-03-20 | Nvidia Corporation | Tree-based thread management |
US20150244804A1 (en) | 2014-02-21 | 2015-08-27 | Coho Data, Inc. | Methods, systems and devices for parallel network interface data structures with differential data storage service capabilities |
US9202178B2 (en) * | 2014-03-11 | 2015-12-01 | Sas Institute Inc. | Computerized cluster analysis framework for decorrelated cluster identification in datasets |
US9779128B2 (en) | 2014-04-10 | 2017-10-03 | Futurewei Technologies, Inc. | System and method for massively parallel processing database |
US9454558B2 (en) | 2014-04-23 | 2016-09-27 | International Business Machines Corporation | Managing an index of a table of a database |
US20150312366A1 (en) * | 2014-04-24 | 2015-10-29 | Strato Scale Ltd. | Unified caching of storage blocks and memory pages in a compute-node cluster |
US9218244B1 (en) * | 2014-06-04 | 2015-12-22 | Pure Storage, Inc. | Rebuilding data across storage nodes |
US8850108B1 (en) * | 2014-06-04 | 2014-09-30 | Pure Storage, Inc. | Storage cluster |
US10169433B2 (en) | 2014-07-29 | 2019-01-01 | Microsoft Technology Licensing, Llc | Systems and methods for an SQL-driven distributed operating system |
US10983859B2 (en) * | 2014-08-07 | 2021-04-20 | Pure Storage, Inc. | Adjustable error correction based on memory health in a storage unit |
US9830238B2 (en) * | 2014-08-29 | 2017-11-28 | Netapp, Inc. | Techniques for maintaining communications sessions among nodes in a storage cluster system |
US9767149B2 (en) * | 2014-10-10 | 2017-09-19 | International Business Machines Corporation | Joining data across a parallel database and a distributed processing system |
US20160246842A1 (en) * | 2015-02-25 | 2016-08-25 | Futurewei Technologies, Inc. | Query optimization adaptive to system memory load for parallel database systems |
CN107077453B (en) * | 2015-03-24 | 2020-11-06 | 华为技术有限公司 | System and method for parallel optimization of database queries using cluster caches |
WO2017196315A1 (en) * | 2016-05-11 | 2017-11-16 | Hitachi, Ltd. | Data storage system and process for reducing read and write amplifications |
US10885062B2 (en) * | 2016-06-23 | 2021-01-05 | Sap Se | Providing database storage to facilitate the aging of database-accessible data |
-
2017
- 2017-10-02 WO PCT/US2017/054784 patent/WO2018067471A1/en active Application Filing
- 2017-10-02 US US15/722,968 patent/US10713276B2/en active Active
- 2017-10-02 US US15/722,794 patent/US20180095914A1/en not_active Abandoned
- 2017-10-02 WO PCT/US2017/054773 patent/WO2018067467A1/en active Application Filing
- 2017-10-02 US US15/722,900 patent/US11586647B2/en active Active
- 2017-10-02 US US15/722,687 patent/US20180095996A1/en not_active Abandoned
-
2020
- 2020-07-09 US US16/924,458 patent/US11294932B2/en active Active
-
2022
- 2022-02-23 US US17/652,261 patent/US11934423B2/en active Active
-
2023
- 2023-02-16 US US18/170,225 patent/US20230195749A1/en active Pending
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180150220A1 (en) * | 2016-11-25 | 2018-05-31 | Samsung Electronics Co., Ltd. | System and method for improving storage device i/o performance |
US11240178B2 (en) * | 2019-04-16 | 2022-02-01 | Realtek Semiconductor Corporation | Data transmission method and data transmission system |
Also Published As
Publication number | Publication date |
---|---|
US11294932B2 (en) | 2022-04-05 |
US20180095914A1 (en) | 2018-04-05 |
US11586647B2 (en) | 2023-02-21 |
US10713276B2 (en) | 2020-07-14 |
WO2018067471A1 (en) | 2018-04-12 |
WO2018067467A1 (en) | 2018-04-12 |
US20200342001A1 (en) | 2020-10-29 |
US11934423B2 (en) | 2024-03-19 |
US20180096049A1 (en) | 2018-04-05 |
US20220179878A1 (en) | 2022-06-09 |
US20180096048A1 (en) | 2018-04-05 |
US20230195749A1 (en) | 2023-06-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20180095996A1 (en) | Database system utilizing forced memory aligned access | |
US11860813B2 (en) | High level instructions with lower-level assembly code style primitives within a memory appliance for accessing memory | |
US11340794B2 (en) | Multiprocessor system with independent direct access to bulk solid state memory resources | |
US10331655B2 (en) | System-wide checkpoint avoidance for distributed database systems | |
US9720822B2 (en) | NVRAM caching and logging in a storage system | |
US10042751B1 (en) | Method and system for multi-tier all-flash array | |
US10180948B2 (en) | Data storage with a distributed virtual array | |
KR101914019B1 (en) | Fast crash recovery for distributed database systems | |
US20210117441A1 (en) | Data replication system | |
US10360149B2 (en) | Data structure store in persistent memory | |
US8868487B2 (en) | Event processing in a flash memory-based object store | |
CN106462510B (en) | Multiprocessor system with independent direct access to large amounts of solid-state storage resources | |
US11163452B2 (en) | Workload based device access | |
US10162686B2 (en) | Cache affinity and processor utilization technique | |
CN103873559A (en) | Database all-in-one machine capable of realizing high-speed storage | |
US10185681B2 (en) | Hybrid message-based scheduling technique | |
US9870386B1 (en) | Reducing I/O operations for on-demand demand data page generation | |
US11550479B1 (en) | Metadata management in storage systems | |
Nowoczynski et al. | Zest checkpoint storage system for large supercomputers | |
US10089228B2 (en) | I/O blender countermeasures | |
Taleb | Optimizing Distributed In-memory Storage Systems: Fault-tolerance, Performance, Energy Efficiency | |
US11630773B1 (en) | Utilizing a persistent write cache as a redo log | |
US11971825B2 (en) | Managing granularity of a metadata structure for a storage system | |
US20240020236A1 (en) | Managing granularity of a metadata structure for a storage system | |
US20200019343A1 (en) | Method and apparatus for a durable low latency system architecture |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: OCIENT, LLC, ILLINOIS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KONDILES, GEORGE;STARR, RHETT COLIN;JABLONSKI, JOSEPH;REEL/FRAME:043758/0054 Effective date: 20171002 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: OCIENT, INC., ILLINOIS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:OCIENT LLC;REEL/FRAME:045111/0181 Effective date: 20180228 |
|
STCB | Information on status: application discontinuation |
Free format text: EXPRESSLY ABANDONED -- DURING EXAMINATION |