US8768927B2 - Hybrid database table stored as both row and column store - Google Patents

Hybrid database table stored as both row and column store Download PDF

Info

Publication number
US8768927B2
US8768927B2 US13/334,669 US201113334669A US8768927B2 US 8768927 B2 US8768927 B2 US 8768927B2 US 201113334669 A US201113334669 A US 201113334669A US 8768927 B2 US8768927 B2 US 8768927B2
Authority
US
United States
Prior art keywords
row
column
store
partitioned store
search
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US13/334,669
Other versions
US20130166553A1 (en
Inventor
Yongsik Yoon
Chanho Jeong
Juchang Lee
Chang Bin Song
Yong Sik Kwon
Sang Kyun Cha
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SAP SE
Original Assignee
SAP SE
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SAP SE filed Critical SAP SE
Priority to US13/334,669 priority Critical patent/US8768927B2/en
Assigned to SAP AG reassignment SAP AG ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: LEE, JUCHANG, CHA, SANG KYUN, JEONG, CHANHO, KWON, YONG SIK, SONG, CHANG BIN, YOON, YONGSIK
Priority to EP12006519.8A priority patent/EP2608071A1/en
Priority to CN201210399140.1A priority patent/CN103177055B/en
Publication of US20130166553A1 publication Critical patent/US20130166553A1/en
Priority to US14/272,769 priority patent/US20140244628A1/en
Application granted granted Critical
Publication of US8768927B2 publication Critical patent/US8768927B2/en
Assigned to SAP SE reassignment SAP SE CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: SAP AG
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2365Ensuring data consistency and integrity
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/278Data partitioning, e.g. horizontal or vertical partitioning

Definitions

  • the present invention relates to storage of data in databases, and in particular, to the storage of database information in a hybrid table format.
  • a database is an electronic filing system that stores data in a structured way.
  • the primary storage structure in a database is a table.
  • a database may contain multiple tables and each table may hold information of a specific type.
  • Database tables store and organize data in horizontal rows and vertical columns. Rows typically correspond to real-world entities or relationships that represent individual records in a table. Columns may denote specific attributes of those entities or relationships, such as “name,” “address” or “phone number.”
  • Company X may have a database containing a “customer” table listing the names, addresses and phone numbers of its customers. Each row may represent a single customer and the columns may represent each customer's name, address and phone number.
  • Databases are generally stored in computer memory that is one-dimensional. Two-dimensional database tables must therefore be mapped onto a one-dimensional data structure to be stored within a database.
  • One mapping approach involves storing a table in a database row-by-row (i.e., a row-oriented storage model). This approach keeps information about a single entity together. For example, row-by-row storage may store all information about a first customer first, then all information about a second customer and so on.
  • a table may be stored in a database column-by-column (i.e., a column-oriented storage model). This approach keeps like attributes of different entities together. For example, column-by-column storage may store all customer names first, then all customer addresses and so on.
  • Data must generally be accessed from a table in the same manner that it was stored. That is, conventional computer storage techniques require dedicated query operators that can access specific types of storage models. For example, row query operators are used to process data stored in a database in row-formatted storage models and column query operators are used to process data stored in column-formatted storage models.
  • Row-oriented storage models are commonly well-suited for transactional queries.
  • the row-store format emphasizes the database row as the manipulable element, and is typically used for On-Line Transaction Processing (OLTP) involving a relatively large number of on-line transactions (rows), with each transaction characterized by relatively larger data types (columns).
  • OTP On-Line Transaction Processing
  • column-oriented storage models are generally well-suited for analytical queries.
  • the column-store format emphasizes the database column as the manipulable element, and is typically used for On-Line Analytical Processing (OLAP) of a subset of a total number of transactions (rows) over a fewer number of data types (columns) that may include aggregations of basic data types.
  • OLAP On-Line Analytical Processing
  • a database table in the column-store format is typically used for interrogation and analysis of the raw data for purposes of problem-solving and planning that form a part of Business Intelligence (BI) efforts.
  • BI Business Intelligence
  • row store may be useful for retrieving individual records having many columns with a primary key condition.
  • Column store may be useful for performing more complex functions such as aggregation/join over a relatively small number of columns.
  • a database having certain data stored in a column-formatted storage model may be asked to handle a transactional query relating to that data, or a database having certain data stored in a row-formatted storage model may be asked to handle an analytical query relating to that data.
  • a database having data stored in a row-formatted storage model may receive a mixed set of queries requiring transactional and analytical processing of that data.
  • row-store and column-store database table formats offer various benefits.
  • the row-store format offers ready scalability for data, as it is expected that more and more transactions will require storage in additional rows.
  • the row-store table format is, however, relatively memory intensive for analytic queries (e.g. aggregation, join) as it scans a table vertically, incurring cache misses as the data is stored horizontally.
  • the column-store format offers flexibility in allowing complex manipulation of data involving table joins and aggregation, as well as relatively low memory consumption by allowing compression within data types across multiple entries by dictionary encoding.
  • the column-store database format typically does not allow ready manipulation of the same volumes of data as the row-store table.
  • a row-store table is more effective to serve row-wise record access such as single record selection with primary key lookup.
  • a column-store table is better to serve column-wise record access such as single column aggregation.
  • column-store table becomes memory-intensive, because the data format is organized vertically in column-store so that cache misses occur while accessing record values horizontally.
  • row-store table becomes memory-intensive because the data format is organized horizontally in row-store, so that cache misses occur while reading specific column values.
  • a table can generally only be in row- or column-store at any point in time of a business life cycle. Accordingly, the present disclosure addresses this and other issues with systems and methods for implementing a hybrid database table stored as both a row and a column store.
  • a hybrid database table is stored as both a row and a column store.
  • One or more techniques may be employed alone or in combination to enhance performance of the hybrid table by regulating access to, and/or the size of, the processing-intensive column store data. For example during an insert operation, the column store data may be searched for a uniqueness violation only after certain filtering and/or boundary conditions have been considered.
  • a hybrid table manager may control movement of data to the column store based upon considerations such as frequency of access, or underlying business logic.
  • querying of the hybrid table may result in a search of the column store data only after an initial search of row store data fails to return a result.
  • An embodiment of a computer-implemented method comprises, providing in a non-transitory computer readable storage medium, a database created in an application level language and comprising row partitioned store and column partitioned store.
  • a new record is received to be inserted into the database.
  • a row engine is caused to insert the new record into the row partitioned store.
  • An optimization is performed based upon additional information to check for a uniqueness violation only if there are uniqueness constraints.
  • a result of processing a query to the database including the new record, is displayed.
  • An embodiment of a non-transitory computer readable storage medium embodying a computer program for performing a method comprises a method of receiving a new record to be inserted into a database created in an application level language in a non-transitory storage medium and comprising row partitioned store and column partitioned store.
  • a row engine to is caused insert the new record into the row partitioned store.
  • An optimization is performed based upon additional information to check for a uniqueness violation only if there are uniqueness constraints.
  • a result of processing a query to the database including the new record is displayed.
  • An embodiment of a computer system comprises, one or more processors and a software program, executable on said computer system.
  • the software program is configured to receive a new record to be inserted into a database created in an application level language in a non-transitory storage medium and comprising row partitioned store and column partitioned store.
  • the computer program is configured to cause a row engine to insert the new record into the row partitioned store, and to perform an optimization based upon additional information to check for a uniqueness violation only if there are uniqueness constraints.
  • the computer program is configured to display a result of processing a query to the database including the new record.
  • the optimization comprises maintaining min/max values of unique columns in the column partitioned store with a latch-free structure to minimize contention.
  • the optimization comprises maintaining a filter for unique columns.
  • the filter comprises a Bloom filter.
  • the filter is scalable.
  • Certain embodiments may further comprise updating or rehashing when data is moved from the row partitioned store to the column partitioned store.
  • FIG. 1 shows a simplified schematic view of the structure and operation of a hybrid table according to an embodiment.
  • FIG. 2 shows a simplified schematic view of the insertion of a new record into an embodiment of a hybrid table.
  • FIG. 2A shows a simplified view of a process flow for the insertion of a new record into an embodiment of a hybrid table.
  • FIG. 3A plots performance of a Bloom filter as bits/element vs. number of hash functions.
  • FIG. 3B plots memory size versus number of records, for a scalable Bloom filter.
  • FIG. 4 is a bar chart showing throughput in transactions per second (TPS) during insert handling with different numbers of threads.
  • FIG. 5 shows a simplified view of controlling data movement in a hybrid table according to an embodiment.
  • FIG. 5A shows a simplified view of a process flow for the movement of records between row partition store and column partition store in an embodiment of a hybrid table.
  • FIG. 6 shows a simplified schematic view of implementation of an access-based data movement policy based upon page-level access statistics management.
  • FIG. 7 shows a simplified example of a business-aware data movement policy according to an embodiment.
  • FIG. 8 shows a simplified schematic view of a search operation on a hybrid database according to an embodiment.
  • FIG. 8A shows a simplified view of a process flow for searching an embodiment of a hybrid table.
  • FIG. 9 shows a simplified schematic view of a simple search conducted on a single hybrid table.
  • FIG. 10 is a simplified flow diagram showing the resort to the column search if necessary, followed by merger of the row search results and column search results.
  • FIG. 11 shows a simplified schematic view of a conversion-based join operation.
  • FIG. 12 shows a simplified schematic view of a native mixed join operation.
  • FIG. 13 shows another view of a native mixed join operation.
  • FIG. 14 illustrates hardware of a special purpose computing machine which may be configured to implement a hybrid database table.
  • FIG. 15 shows an example of a computer system.
  • FIG. 16 plots throughput versus number of connections for search performance on conventional row and column store tables, as well as a corresponding hybrid table.
  • Described herein are techniques for providing a hybrid database table stored as both a row-store and column-store. Certain techniques, employed alone or in combination, enhance hybrid table performance by limiting access to, and/or the size of, the processing-intensive column store data.
  • FIG. 1 shows a simplified schematic view of a hybrid table according to an embodiment.
  • the hybrid table 100 comprises a first data store 102 comprising data stored according to an in-memory row partition.
  • the hybrid table also comprises a second data store 106 comprising data stored according to an in-memory column partition.
  • accessing data in the column store orientation is more processing-intensive than accessing data in the row store orientation.
  • a query processor 108 is in communication with the hybrid table to perform row searching and column searching, respectively, utilizing row engine 107 and column engine 109 .
  • the row search result 110 and the column search result 112 are returned to the query processor, and a merged query search result 114 is returned to the user.
  • the arrow 122 of FIG. 1 shows the function of inserting a new record into the hybrid table. As described in detail below, this insert function can be optimized to enhance performance of the hybrid table.
  • FIG. 1 also shows that data in the hybrid table may be moved from the relatively update-intensive row partition orientation, to the relatively read-intensive column partition orientation.
  • Data manager 120 is responsible for this data movement, which under certain circumstances may take the form of an asynchronous transactional bulk data move to the column table.
  • the asynchronous data movement is designed to exploit favorable attributes of both row- and column-partition. From a performance perspective, it may be desirable to place records that are recently inserted, frequently updated, and/or searched with primary key condition, in row-partition. It is desirable to place records that are not frequently updated but sometimes aggregated, in column-partition.
  • embodiments of hybrid tables may seek to avoid sacrificing OLTP performance (maintaining relevant data in row store), and to improve aggregation performance (maintaining stable, aged, for-aggregation data, in column store), while maintaining reasonable memory consumption (e.g. through row/column boundary management).
  • a hybrid table according to embodiments of the present invention may offer certain potential benefits.
  • the presence of the data stored in the row store orientation offers fast access.
  • Data stored in the row store orientation is amenable to single table select with search conditions (e.g. exact match/range).
  • search conditions e.g. exact match/range.
  • fast “Simple Select” queries may be used for access to the row partitioned data store.
  • the row partitioned data store also affords scalable update and search performance.
  • the presence of the row partitioned data store allows concurrent workload handling with a lean stack. Performance of functions such as insert, update, and delete, as well as simple searching (exact match/range) are possible with relatively low consumption of system resources. And, execution in the row engine may be pipelined.
  • memory consumption may be reduced by dictionary encoding of the column table.
  • the column table also affords rapid OLAP processing by the column engine.
  • Embodiments of the present invention may seek to maintain performance of the hybrid table, by limiting access to, or the size of, the processing-intensive column store data. One example of this can occur during an insert operation, as shown in FIG. 2 .
  • FIG. 2 shows a simplified schematic view of the insertion of a new record into a hybrid table 200 .
  • a new record 202 is inserted into unique row store (RS) indexes 204 associated with the in-memory row partition data store 206 .
  • RS unique row store
  • the hybrid table before resorting to search each value in the column partitioned data store 208 (a processing-intensive prospect), in order to check for uniqueness violations the hybrid table first performs an optimization 209 based on additional information.
  • FIG. 2A shows a simplified view of a process flow 250 for the insertion of a new record into an embodiment of a hybrid table.
  • a first step 252 comprises providing in a non-transitory computer readable storage medium, a database created in an application level language and comprising row partitioned store and column partitioned store.
  • a second step 252 comprises receiving a new record to be inserted into the database.
  • a third step 256 comprises causing a row engine to insert the new record into the row partitioned store according to a row store index.
  • a fourth step 258 comprises performing an optimization based upon additional information to check for a uniqueness violation.
  • a fifth step 260 comprises checking uniqueness violation in the column partitioned store, only if warranted by the optimization.
  • a sixth step 262 comprises displaying a result of processing a query to the database including the new record.
  • a first optimization approach is to maintain min/max values of unique columns on column partition with latch-free structure to minimize contention. This approach may possibly result in many false hits for multi-column unique keys.
  • a second optimization approach is to maintain a filter (such as a Bloom filter) for unique columns.
  • This filtering approach 210 is beneficial in that it requires updating or rehashing only when data is moved from the row store orientation to the data store orientation.
  • a Bloom filter is a data structure to effectively filter out any element that does not belong to a set.
  • the Bloom filter comprises a bit vector and a set of hash functions. If the Bloom filter returns FALSE, then the element is definitely not a member of the set. If the Bloom filter returns TRUE, then the element is probably in the set.
  • FIG. 3A plots performance of a Bloom filter as bits/element vs. number of hash functions, with a False Positive Rate (FPR) of 0.1.
  • the FPR depends on the number of hash functions and the number of element/(size of bit vector).
  • hybrid table performance during insert handing may be enhanced through the use of a Bloom filter.
  • values in the column partition data store need be searched only if the Bloom filter returns TRUE.
  • a standard Bloom filter for this task may consume large amounts of memory. For example, a standard Bloom filter needs about 60 MB to keep FPR ⁇ 0.1 with 100 million records.
  • certain embodiments may employ a Scalable Bloom Filter (SBF) in order to perform optimization of insert handing to a hybrid table.
  • SBF Scalable Bloom Filter
  • a SBF is a variant of the Bloom filter that uses space proportional to the number of stored elements.
  • the query time for a SBF obeys the following relation:
  • n number of records.
  • Optimized insert handling utilizing a SBF can result in enhanced performance of a hybrid table. This is discussed in connection with the following example.
  • a data population of 800,000 records was configured in six (6) different table types: row oriented (row, A); column oriented (column, B); hybrid without filtering and all 800,000 records in row store (C); hybrid without filtering and all 800,000 records in column store (D); hybrid with SBF and all 800,000 records in row store (E); and hybrid with SBF and all 800,000 records in column store (F).
  • FIG. 4 is a bar graph showing throughput in transactions per second (TPS) during insert handling with different numbers of threads.
  • TPS transactions per second
  • FIG. 4 indicates that without filtering (C, D), the insert handing performance of the hybrid table is only about 1 ⁇ 3 that of the row table (A).
  • the hybrid table shows similar insert handing performance as a row table (A), notwithstanding the fact that the populated data of the hybrid table is located in the column partitioned data store. This represents a significant enhancement, rendering performance of the hybrid database table akin to that of the conventional row-store approach.
  • FIG. 5 shows a simplified view of controlling data movement in a hybrid table according to an embodiment.
  • a hybrid table manager 500 is responsible for controlling the movement of data from an in-memory row partition store 502 , to in-memory column partition store 504 , according to a data movement policy. By intelligently partitioning data, selective data movement may allow fast OLTP processing avoiding column partition access.
  • FIG. 5A shows a simplified view of a process flow 550 for the movement of records between row partition store and column partition store in an embodiment of a hybrid table.
  • a first step 552 comprises providing in a non-transitory computer readable storage medium, a database created in an application level language and comprising a row partitioned store and a column partitioned store.
  • a second step 554 comprises providing a data movement manager configured to execute a data move from the row partitioned store to the column partitioned store according to a data movement policy.
  • a third step 556 comprises causing the data movement manager to automatically move a record from the row partitioned store to the column partitioned store upon satisfaction of a condition of the data movement policy.
  • a search result of the row partitioned store is returned to the user.
  • One example of a data movement policy is based upon access. As this data movement policy does not rely upon knowledge of business rules (see below), it may serve as the default.
  • the hybrid table manager moves some portion of records on the row partition to the column partition, when the number of records on row partition exceeds a defined threshold.
  • This access-based data movement can be based upon statistics maintained for accessed records, such that when a movement is triggered those records having a lower frequency of access are moved. Access statistics may be reset whenever a data movement is finished.
  • FIG. 6 shows a simplified schematic view of implementation of an access-based data movement policy 600 based upon page-level access statistics management.
  • Each of a plurality of pages 602 in a row partition store 604 include an ongoing access count 606 . Based upon relatively low numbers of this access count, pages having a least access frequency can be moved out to the column partition data store.
  • frequently accessed records can possibly be moved out of the column partitioned store due to coarse granular access statistics management.
  • some embodiments could perform page level movement instead of record level management, where a page includes multiple records, some of which are frequently accessed whereas others are not. Such an approach is possible, although an unintended data move may occur for frequently-accessed records.
  • An alternative data movement policy may be based upon business concepts instead of access frequency. Such a data movement policy may offer the best results when business semantics are known.
  • business-aware data movement can take the form of condition-based movement.
  • records that satisfy certain business conditions can be moved out to the column partitioned store.
  • business-aware data movement can take the form of reference-based movement.
  • records are moved out to the column partitioned store when related records (in referenced table) are moved out.
  • FIG. 7 shows a simplified embodiment of a business-aware data movement policy 700 .
  • closed sales documents are moved to the column partitioned data store.
  • GBSTK ‘C’
  • Closed-document-related records in SD item (VBAP) table and Document header (VBUK) table can be moved out accordingly by specifying a relationship between tables as move out condition.
  • MANDT and VBELN columns are used to specify this relationship.
  • FIG. 9 shows a simplified schematic view of a simple search conducted on a single hybrid table. Specifically, as part of a project a query 900 is posed to the hybrid table 902 . The union 904 of the search results of the row store (RS) 906 and column store (CS) 908 , are returned to the user.
  • RS row store
  • CS column store
  • FIG. 8 shows a simplified schematic view of a search operation on a hybrid database 800 according to an embodiment.
  • a search 802 can initially be performed on the in-memory row partitioned data store 804 . Only once the result of that initial search 802 has been returned, is a decision made whether or not to conduct a further, search 806 of the values stored in the in-memory column partition 808 .
  • FIG. 8A shows a simplified view of a process flow 850 for searching an embodiment of a hybrid table.
  • a first step 852 comprises providing in a non-transitory computer readable storage medium, a database created in an application level language and comprising a row partitioned store and a column partitioned store.
  • a second step 854 comprises receiving a query from a user.
  • a third step 856 comprises searching the row partitioned store based upon the query.
  • a fourth step 858 comprises performing an optimization based upon a search of the row partitioned store.
  • a fifth step 860 comprises searching the column partitioned store based upon the query, only if warranted by the optimization.
  • a search result is returned to the user based on the query.
  • search of the column store data can occur in the background, transparent to the user. For example searches involving a primary key (PK) or a unique-key can be run initially only on the less processing-intensive row store data. If a result is returned from this initial searching, then there is no need to resort to the processing-intensive task of also searching the column store data.
  • the simplified flow diagram of FIG. 10 shows an embodiment of a hybrid database table search 1000 , with resort to the column search 1002 , followed by merger 1004 of the row search results 1006 and column search results, if necessary.
  • Filtering according to boundary conditions can also be employed as optimization technique in hybrid table searching.
  • a column dictionary can maintain a minimum and maximum value of each column partition. Searching of the actual column store data could then be limited to only the partition that satisfies a filter condition (e.g. a search condition).
  • a boundary of column partition may be maintained to allow search optimization (limiting column partition access) if a given filter condition does not require column partition access.
  • FIG. 16 plots throughput (in threads per second—TPS) versus number of connections, to evaluate performance of a PK exact match search conducted on conventional row and column store tables, and also conducted on a corresponding hybrid table according to an embodiment.
  • FIG. 16 shows that searching a hybrid table with the PK pruning optimization, results in a throughput for the hybrid table that is higher than for the conventional column store table.
  • More complex searching may be appropriate over multiple hybrid database tables.
  • such complex searching may involve table join operations.
  • FIG. 11 shows a simplified schematic view of a conversion-based join operation 1100 .
  • this approach leverages a column join 1102 for a split table.
  • the row table serves as a partition.
  • This approach employs on-the-fly dictionary generation, and dictionary caching.
  • FIG. 12 shows a simplified schematic view of a native mixed join operation 1200 .
  • This approach utilizes a column dictionary as an index. Pruning of the join operation is achieved based upon filtering using the column dictionary index as a data boundary.
  • FIG. 13 shows another view of a native mixed join operation 1300 , which uses the column dictionary as an index.
  • First records are iterated in the row table.
  • the next step involves column dictionary lookup for each join condition.
  • matched records are returned.
  • FIG. 14 illustrates hardware of a special purpose computing machine which may be configured to implement a hybrid database table according to certain embodiments.
  • computer system 1400 comprises a processor 1402 that is in electronic communication with a non-transitory computer-readable storage medium 1403 .
  • This computer-readable storage medium has stored thereon code 1405 corresponding to the in-memory row partition.
  • Code 1404 corresponds to an in-memory column partition, both of which may be accessed as part of the hybrid table according to embodiments as has been described above.
  • the system may further comprise code corresponding to the data movement manager, as has been shown and described above.
  • the apparatuses, methods, and techniques described herein may be implemented as a computer program (software) executing on one or more computers.
  • the computer program may further be stored on a non-transitory computer readable medium.
  • the non-transitory computer readable medium may include instructions for performing the processes described.
  • the computer system may comprise a software server.
  • a number of software servers together may form a cluster, or logical network of computer systems programmed with software programs that communicate with each other and work together to process requests.
  • Computer system 1510 includes a bus 1505 or other communication mechanism for communicating information, and a processor 1501 coupled with bus 1505 for processing information.
  • Computer system 1510 also includes a memory 1502 coupled to bus 1505 for storing information and instructions to be executed by processor 1501 , including information and instructions for performing the techniques described above, for example.
  • This memory may also be used for storing variables or other intermediate information during execution of instructions to be executed by processor 1501 . Possible implementations of this memory may be, but are not limited to, random access memory (RAM), read only memory (ROM), or both.
  • a storage device 1503 is also provided for storing information and instructions.
  • Common forms of storage devices include, for example, a hard drive, a magnetic disk, an optical disk, a CD-ROM, a DVD, a flash memory, a USB memory card, or any other medium from which a computer can read.
  • Storage device 1503 may include source code, binary code, or software files for performing the techniques above, for example.
  • Storage device and memory are both examples of computer readable media.
  • Computer system 1510 may be coupled via bus 1505 to a display 1512 , such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user.
  • a display 1512 such as a cathode ray tube (CRT) or liquid crystal display (LCD)
  • An input device 1511 such as a keyboard and/or mouse is coupled to bus 1505 for communicating information and command selections from the user to processor 1501 .
  • the combination of these components allows the user to communicate with the system.
  • bus 1505 may be divided into multiple specialized buses.
  • Computer system 1510 also includes a network interface 1504 coupled with bus 1505 .
  • Network interface 1504 may provide two-way data communication between computer system 1510 and the local network 1520 .
  • the network interface 1504 may be a digital subscriber line (DSL) or a modem to provide data communication connection over a telephone line, for example.
  • DSL digital subscriber line
  • Another example of the network interface is a local area network (LAN) card to provide a data communication connection to a compatible LAN.
  • LAN local area network
  • Wireless links are another example.
  • network interface 1404 sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information.
  • Computer system 1510 can send and receive information, including messages or other interface actions, through the network interface 1504 across a local network 1520 , an Intranet, or the Internet 1530 .
  • computer system 1510 may communicate with a plurality of other computer machines, such as server 1515 .
  • server 1515 may form a cloud computing network, which may be programmed with processes described herein.
  • software components or services may reside on multiple different computer systems 1510 or servers 1531 - 1535 across the network.
  • the processes described above may be implemented on one or more servers, for example.
  • a server 1531 may transmit actions or messages from one component, through Internet 1530 , local network 1520 , and network interface 1504 to a component on computer system 1510 .
  • the software components and processes described above may be implemented on any computer system and send and/or receive information across a network, for example.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A hybrid database table is stored as both a row and a column store. One or more techniques may be employed alone or in combination to enhance performance of the hybrid table by regulating access to, and/or the size of, the processing-intensive column store data. For example during an insert operation, the column store data may be searched for a uniqueness violation only after certain filtering and/or boundary conditions have been considered. In another technique, a hybrid table manager may control movement of data to the column store based upon considerations such as frequency of access, or underlying business logic. In still another technique, querying of the hybrid table may result in a search of the column store data only after an initial search of row store data fails to return a result.

Description

BACKGROUND
The present invention relates to storage of data in databases, and in particular, to the storage of database information in a hybrid table format.
Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.
A database is an electronic filing system that stores data in a structured way. The primary storage structure in a database is a table. A database may contain multiple tables and each table may hold information of a specific type. Database tables store and organize data in horizontal rows and vertical columns. Rows typically correspond to real-world entities or relationships that represent individual records in a table. Columns may denote specific attributes of those entities or relationships, such as “name,” “address” or “phone number.” For example, Company X may have a database containing a “customer” table listing the names, addresses and phone numbers of its customers. Each row may represent a single customer and the columns may represent each customer's name, address and phone number.
Databases are generally stored in computer memory that is one-dimensional. Two-dimensional database tables must therefore be mapped onto a one-dimensional data structure to be stored within a database. One mapping approach involves storing a table in a database row-by-row (i.e., a row-oriented storage model). This approach keeps information about a single entity together. For example, row-by-row storage may store all information about a first customer first, then all information about a second customer and so on. Alternatively, a table may be stored in a database column-by-column (i.e., a column-oriented storage model). This approach keeps like attributes of different entities together. For example, column-by-column storage may store all customer names first, then all customer addresses and so on.
Data must generally be accessed from a table in the same manner that it was stored. That is, conventional computer storage techniques require dedicated query operators that can access specific types of storage models. For example, row query operators are used to process data stored in a database in row-formatted storage models and column query operators are used to process data stored in column-formatted storage models.
Choosing which storage model to use thus often depends on how data will be used. Row-oriented storage models are commonly well-suited for transactional queries. The row-store format emphasizes the database row as the manipulable element, and is typically used for On-Line Transaction Processing (OLTP) involving a relatively large number of on-line transactions (rows), with each transaction characterized by relatively larger data types (columns).
By contrast, column-oriented storage models are generally well-suited for analytical queries. The column-store format emphasizes the database column as the manipulable element, and is typically used for On-Line Analytical Processing (OLAP) of a subset of a total number of transactions (rows) over a fewer number of data types (columns) that may include aggregations of basic data types. A database table in the column-store format is typically used for interrogation and analysis of the raw data for purposes of problem-solving and planning that form a part of Business Intelligence (BI) efforts.
In summary, row store may be useful for retrieving individual records having many columns with a primary key condition. Column store may be useful for performing more complex functions such as aggregation/join over a relatively small number of columns.
Accordingly, conventional query processing schemes are bound to the underlying storage model of the database being queried. In reality, however, a database having certain data stored in a column-formatted storage model may be asked to handle a transactional query relating to that data, or a database having certain data stored in a row-formatted storage model may be asked to handle an analytical query relating to that data. For example, a database having data stored in a row-formatted storage model may receive a mixed set of queries requiring transactional and analytical processing of that data.
Both the row-store and column-store database table formats offer various benefits. For example, the row-store format offers ready scalability for data, as it is expected that more and more transactions will require storage in additional rows. The row-store table format is, however, relatively memory intensive for analytic queries (e.g. aggregation, join) as it scans a table vertically, incurring cache misses as the data is stored horizontally.
Conversely, the column-store format offers flexibility in allowing complex manipulation of data involving table joins and aggregation, as well as relatively low memory consumption by allowing compression within data types across multiple entries by dictionary encoding. The column-store database format, however, typically does not allow ready manipulation of the same volumes of data as the row-store table.
Thus, a row-store table is more effective to serve row-wise record access such as single record selection with primary key lookup. A column-store table is better to serve column-wise record access such as single column aggregation. For row-wise record access, column-store table becomes memory-intensive, because the data format is organized vertically in column-store so that cache misses occur while accessing record values horizontally. For column-wise record access, row-store table becomes memory-intensive because the data format is organized horizontally in row-store, so that cache misses occur while reading specific column values.
Despite the various advantages of the various database table types, conventionally a table can generally only be in row- or column-store at any point in time of a business life cycle. Accordingly, the present disclosure addresses this and other issues with systems and methods for implementing a hybrid database table stored as both a row and a column store.
SUMMARY
A hybrid database table is stored as both a row and a column store. One or more techniques may be employed alone or in combination to enhance performance of the hybrid table by regulating access to, and/or the size of, the processing-intensive column store data. For example during an insert operation, the column store data may be searched for a uniqueness violation only after certain filtering and/or boundary conditions have been considered. In another technique, a hybrid table manager may control movement of data to the column store based upon considerations such as frequency of access, or underlying business logic. According to still another technique, querying of the hybrid table may result in a search of the column store data only after an initial search of row store data fails to return a result.
An embodiment of a computer-implemented method comprises, providing in a non-transitory computer readable storage medium, a database created in an application level language and comprising row partitioned store and column partitioned store. A new record is received to be inserted into the database. A row engine is caused to insert the new record into the row partitioned store. An optimization is performed based upon additional information to check for a uniqueness violation only if there are uniqueness constraints. A result of processing a query to the database including the new record, is displayed.
An embodiment of a non-transitory computer readable storage medium embodying a computer program for performing a method, comprises a method of receiving a new record to be inserted into a database created in an application level language in a non-transitory storage medium and comprising row partitioned store and column partitioned store. A row engine to is caused insert the new record into the row partitioned store. An optimization is performed based upon additional information to check for a uniqueness violation only if there are uniqueness constraints. A result of processing a query to the database including the new record, is displayed.
An embodiment of a computer system, comprises, one or more processors and a software program, executable on said computer system. The software program is configured to receive a new record to be inserted into a database created in an application level language in a non-transitory storage medium and comprising row partitioned store and column partitioned store. The computer program is configured to cause a row engine to insert the new record into the row partitioned store, and to perform an optimization based upon additional information to check for a uniqueness violation only if there are uniqueness constraints. The computer program is configured to display a result of processing a query to the database including the new record.
In certain embodiments, the optimization comprises maintaining min/max values of unique columns in the column partitioned store with a latch-free structure to minimize contention.
According to some embodiments, the optimization comprises maintaining a filter for unique columns. In particular embodiments, the filter comprises a Bloom filter. In certain embodiments the filter is scalable.
Certain embodiments may further comprise updating or rehashing when data is moved from the row partitioned store to the column partitioned store.
The following detailed description and accompanying drawings provide a better understanding of the nature and advantages of the present invention.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 shows a simplified schematic view of the structure and operation of a hybrid table according to an embodiment.
FIG. 2 shows a simplified schematic view of the insertion of a new record into an embodiment of a hybrid table.
FIG. 2A shows a simplified view of a process flow for the insertion of a new record into an embodiment of a hybrid table.
FIG. 3A plots performance of a Bloom filter as bits/element vs. number of hash functions.
FIG. 3B plots memory size versus number of records, for a scalable Bloom filter.
FIG. 4 is a bar chart showing throughput in transactions per second (TPS) during insert handling with different numbers of threads.
FIG. 5 shows a simplified view of controlling data movement in a hybrid table according to an embodiment.
FIG. 5A shows a simplified view of a process flow for the movement of records between row partition store and column partition store in an embodiment of a hybrid table.
FIG. 6 shows a simplified schematic view of implementation of an access-based data movement policy based upon page-level access statistics management.
FIG. 7 shows a simplified example of a business-aware data movement policy according to an embodiment.
FIG. 8 shows a simplified schematic view of a search operation on a hybrid database according to an embodiment.
FIG. 8A shows a simplified view of a process flow for searching an embodiment of a hybrid table.
FIG. 9 shows a simplified schematic view of a simple search conducted on a single hybrid table.
FIG. 10 is a simplified flow diagram showing the resort to the column search if necessary, followed by merger of the row search results and column search results.
FIG. 11 shows a simplified schematic view of a conversion-based join operation.
FIG. 12 shows a simplified schematic view of a native mixed join operation.
FIG. 13 shows another view of a native mixed join operation.
FIG. 14 illustrates hardware of a special purpose computing machine which may be configured to implement a hybrid database table.
FIG. 15 shows an example of a computer system.
FIG. 16 plots throughput versus number of connections for search performance on conventional row and column store tables, as well as a corresponding hybrid table.
DETAILED DESCRIPTION
Described herein are techniques for providing a hybrid database table stored as both a row-store and column-store. Certain techniques, employed alone or in combination, enhance hybrid table performance by limiting access to, and/or the size of, the processing-intensive column store data.
In the following description, for purposes of explanation, examples and specific details are set forth in order to provide a thorough understanding of various embodiments. It will be evident, however, to one skilled in the art that the present invention as defined by the claims may include some or all of the features in these examples alone or in combination with other features described below, and may further include modifications and equivalents of the features and concepts described herein.
FIG. 1 shows a simplified schematic view of a hybrid table according to an embodiment. The hybrid table 100 comprises a first data store 102 comprising data stored according to an in-memory row partition. The hybrid table also comprises a second data store 106 comprising data stored according to an in-memory column partition. As mentioned above, accessing data in the column store orientation is more processing-intensive than accessing data in the row store orientation.
A query processor 108 is in communication with the hybrid table to perform row searching and column searching, respectively, utilizing row engine 107 and column engine 109. The row search result 110 and the column search result 112 are returned to the query processor, and a merged query search result 114 is returned to the user.
The arrow 122 of FIG. 1 shows the function of inserting a new record into the hybrid table. As described in detail below, this insert function can be optimized to enhance performance of the hybrid table.
FIG. 1 also shows that data in the hybrid table may be moved from the relatively update-intensive row partition orientation, to the relatively read-intensive column partition orientation. Data manager 120 is responsible for this data movement, which under certain circumstances may take the form of an asynchronous transactional bulk data move to the column table.
The asynchronous data movement is designed to exploit favorable attributes of both row- and column-partition. From a performance perspective, it may be desirable to place records that are recently inserted, frequently updated, and/or searched with primary key condition, in row-partition. It is desirable to place records that are not frequently updated but sometimes aggregated, in column-partition.
From a memory consumption perspective, it may be desirable to maintain row partition as small as possible, because as row-partition is not compressed. Thus, the memory consumed for a same amount of records is usually larger than for column-partition.
Accordingly, embodiments of hybrid tables may seek to avoid sacrificing OLTP performance (maintaining relevant data in row store), and to improve aggregation performance (maintaining stable, aged, for-aggregation data, in column store), while maintaining reasonable memory consumption (e.g. through row/column boundary management).
A hybrid table according to embodiments of the present invention may offer certain potential benefits. In particular, the presence of the data stored in the row store orientation offers fast access. Data stored in the row store orientation is amenable to single table select with search conditions (e.g. exact match/range). Thus fast “Simple Select” queries may be used for access to the row partitioned data store. The row partitioned data store also affords scalable update and search performance.
Moreover, the presence of the row partitioned data store allows concurrent workload handling with a lean stack. Performance of functions such as insert, update, and delete, as well as simple searching (exact match/range) are possible with relatively low consumption of system resources. And, execution in the row engine may be pipelined.
Regarding the column store data, memory consumption may be reduced by dictionary encoding of the column table. The column table also affords rapid OLAP processing by the column engine.
Embodiments of the present invention may seek to maintain performance of the hybrid table, by limiting access to, or the size of, the processing-intensive column store data. One example of this can occur during an insert operation, as shown in FIG. 2.
In particular, FIG. 2 shows a simplified schematic view of the insertion of a new record into a hybrid table 200. First, a new record 202 is inserted into unique row store (RS) indexes 204 associated with the in-memory row partition data store 206.
According to particular embodiments, before resorting to search each value in the column partitioned data store 208 (a processing-intensive prospect), in order to check for uniqueness violations the hybrid table first performs an optimization 209 based on additional information.
FIG. 2A shows a simplified view of a process flow 250 for the insertion of a new record into an embodiment of a hybrid table. A first step 252 comprises providing in a non-transitory computer readable storage medium, a database created in an application level language and comprising row partitioned store and column partitioned store. A second step 252 comprises receiving a new record to be inserted into the database. A third step 256 comprises causing a row engine to insert the new record into the row partitioned store according to a row store index. A fourth step 258 comprises performing an optimization based upon additional information to check for a uniqueness violation. A fifth step 260 comprises checking uniqueness violation in the column partitioned store, only if warranted by the optimization. A sixth step 262 comprises displaying a result of processing a query to the database including the new record.
A first optimization approach is to maintain min/max values of unique columns on column partition with latch-free structure to minimize contention. This approach may possibly result in many false hits for multi-column unique keys.
A second optimization approach is to maintain a filter (such as a Bloom filter) for unique columns. This filtering approach 210, is beneficial in that it requires updating or rehashing only when data is moved from the row store orientation to the data store orientation.
In particular, a Bloom filter is a data structure to effectively filter out any element that does not belong to a set. The Bloom filter comprises a bit vector and a set of hash functions. If the Bloom filter returns FALSE, then the element is definitely not a member of the set. If the Bloom filter returns TRUE, then the element is probably in the set.
FIG. 3A plots performance of a Bloom filter as bits/element vs. number of hash functions, with a False Positive Rate (FPR) of 0.1. The FPR depends on the number of hash functions and the number of element/(size of bit vector).
In certain embodiments, hybrid table performance during insert handing may be enhanced through the use of a Bloom filter. In particular, values in the column partition data store need be searched only if the Bloom filter returns TRUE.
Use of a standard Bloom filter for this task may consume large amounts of memory. For example, a standard Bloom filter needs about 60 MB to keep FPR≦0.1 with 100 million records.
Accordingly, certain embodiments may employ a Scalable Bloom Filter (SBF) in order to perform optimization of insert handing to a hybrid table. A SBF is a variant of the Bloom filter that uses space proportional to the number of stored elements. The query time for a SBF obeys the following relation:
Query time=O(k log n), where:
k=number of hash functions; and
n=number of records.
FIG. 3B plots memory size versus number of records, for a SBF having a FPR=0.1 and k=3.
Optimized insert handling utilizing a SBF, can result in enhanced performance of a hybrid table. This is discussed in connection with the following example.
EXAMPLE
A data population of 800,000 records was configured in six (6) different table types: row oriented (row, A); column oriented (column, B); hybrid without filtering and all 800,000 records in row store (C); hybrid without filtering and all 800,000 records in column store (D); hybrid with SBF and all 800,000 records in row store (E); and hybrid with SBF and all 800,000 records in column store (F).
FIG. 4 is a bar graph showing throughput in transactions per second (TPS) during insert handling with different numbers of threads. The performance of a hybrid table can be gauged by comparison with a table having data stored in a row orientation (the least processing-intensive format).
FIG. 4 indicates that without filtering (C, D), the insert handing performance of the hybrid table is only about ⅓ that of the row table (A). By contrast where the SBF is used (E, F), the hybrid table shows similar insert handing performance as a row table (A), notwithstanding the fact that the populated data of the hybrid table is located in the column partitioned data store. This represents a significant enhancement, rendering performance of the hybrid database table akin to that of the conventional row-store approach.
While the above discussion has focused upon the use of a scalable Bloom filter for optimization of hybrid table performance, the present invention is not limited to this particular filter type. Different types of index variations for fast value existence checking may be employed. Examples include but are not limited to hash index, b+-tree index, csb+-tree index, and others.
Another technique which may be employed to enhance performance of a hybrid table, is to selectively move records to and from the column partitioned data store in order to control its size. FIG. 5 shows a simplified view of controlling data movement in a hybrid table according to an embodiment.
In particular, a hybrid table manager 500 is responsible for controlling the movement of data from an in-memory row partition store 502, to in-memory column partition store 504, according to a data movement policy. By intelligently partitioning data, selective data movement may allow fast OLTP processing avoiding column partition access.
FIG. 5A shows a simplified view of a process flow 550 for the movement of records between row partition store and column partition store in an embodiment of a hybrid table. A first step 552 comprises providing in a non-transitory computer readable storage medium, a database created in an application level language and comprising a row partitioned store and a column partitioned store. A second step 554 comprises providing a data movement manager configured to execute a data move from the row partitioned store to the column partitioned store according to a data movement policy. A third step 556 comprises causing the data movement manager to automatically move a record from the row partitioned store to the column partitioned store upon satisfaction of a condition of the data movement policy. In an optional fourth step 558, a search result of the row partitioned store is returned to the user.
One example of a data movement policy is based upon access. As this data movement policy does not rely upon knowledge of business rules (see below), it may serve as the default.
According to an access-based data movement policy, the hybrid table manager moves some portion of records on the row partition to the column partition, when the number of records on row partition exceeds a defined threshold. This access-based data movement can be based upon statistics maintained for accessed records, such that when a movement is triggered those records having a lower frequency of access are moved. Access statistics may be reset whenever a data movement is finished.
FIG. 6 shows a simplified schematic view of implementation of an access-based data movement policy 600 based upon page-level access statistics management. Each of a plurality of pages 602 in a row partition store 604 include an ongoing access count 606. Based upon relatively low numbers of this access count, pages having a least access frequency can be moved out to the column partition data store.
In certain embodiments, frequently accessed records can possibly be moved out of the column partitioned store due to coarse granular access statistics management. For example particular some embodiments could perform page level movement instead of record level management, where a page includes multiple records, some of which are frequently accessed whereas others are not. Such an approach is possible, although an unintended data move may occur for frequently-accessed records.
An alternative data movement policy may be based upon business concepts instead of access frequency. Such a data movement policy may offer the best results when business semantics are known.
According to certain embodiments, business-aware data movement can take the form of condition-based movement. In such an approach, records that satisfy certain business conditions can be moved out to the column partitioned store.
In other embodiments, business-aware data movement can take the form of reference-based movement. In such approaches, records are moved out to the column partitioned store when related records (in referenced table) are moved out.
FIG. 7 shows a simplified embodiment of a business-aware data movement policy 700. In this example, closed sales documents are moved to the column partitioned data store.
This in the example of FIG. 7, for the sales document table VBAK, if the value of GBSTK column is ‘C’, then it means closed document. So, the move out condition can be specified as GBSTK=‘C’ to move out only closed documents. Closed-document-related records in SD item (VBAP) table and Document header (VBUK) table, can be moved out accordingly by specifying a relationship between tables as move out condition. Here, MANDT and VBELN columns are used to specify this relationship.
Performance of a hybrid table may also be enhanced through optimization of the search function. FIG. 9 shows a simplified schematic view of a simple search conducted on a single hybrid table. Specifically, as part of a project a query 900 is posed to the hybrid table 902. The union 904 of the search results of the row store (RS) 906 and column store (CS) 908, are returned to the user.
According to particular embodiments of the present invention, information such as boundary conditions can be referenced to control access to the column stored data, thereby minimizing column store access for simple queries. FIG. 8 shows a simplified schematic view of a search operation on a hybrid database 800 according to an embodiment.
As described further in detail below, a search 802 can initially be performed on the in-memory row partitioned data store 804. Only once the result of that initial search 802 has been returned, is a decision made whether or not to conduct a further, search 806 of the values stored in the in-memory column partition 808.
FIG. 8A shows a simplified view of a process flow 850 for searching an embodiment of a hybrid table. A first step 852 comprises providing in a non-transitory computer readable storage medium, a database created in an application level language and comprising a row partitioned store and a column partitioned store. A second step 854 comprises receiving a query from a user. A third step 856 comprises searching the row partitioned store based upon the query. A fourth step 858 comprises performing an optimization based upon a search of the row partitioned store. A fifth step 860 comprises searching the column partitioned store based upon the query, only if warranted by the optimization. In sixth step 862, a search result is returned to the user based on the query.
If warranted by the optimization, search of the column store data can occur in the background, transparent to the user. For example searches involving a primary key (PK) or a unique-key can be run initially only on the less processing-intensive row store data. If a result is returned from this initial searching, then there is no need to resort to the processing-intensive task of also searching the column store data. The simplified flow diagram of FIG. 10 shows an embodiment of a hybrid database table search 1000, with resort to the column search 1002, followed by merger 1004 of the row search results 1006 and column search results, if necessary.
Filtering according to boundary conditions can also be employed as optimization technique in hybrid table searching. For example, to control access to the column store data, a column dictionary can maintain a minimum and maximum value of each column partition. Searching of the actual column store data could then be limited to only the partition that satisfies a filter condition (e.g. a search condition). A boundary of column partition may be maintained to allow search optimization (limiting column partition access) if a given filter condition does not require column partition access.
FIG. 16 plots throughput (in threads per second—TPS) versus number of connections, to evaluate performance of a PK exact match search conducted on conventional row and column store tables, and also conducted on a corresponding hybrid table according to an embodiment. FIG. 16 shows that searching a hybrid table with the PK pruning optimization, results in a throughput for the hybrid table that is higher than for the conventional column store table.
More complex searching may be appropriate over multiple hybrid database tables. In particular, such complex searching may involve table join operations.
One technique which may be used for complex searching of multiple hybrid tables, is a conversion-based join. Such a conversion-based join is discussed in detail in U.S. patent application Ser. No. 12/982,673 titled “Processing Database Queries Using Format Conversion”, filed Dec. 30, 2010 and incorporated by reference in its entirety herein for all purposes.
FIG. 11 shows a simplified schematic view of a conversion-based join operation 1100. In particular, this approach leverages a column join 1102 for a split table. The row table serves as a partition. This approach employs on-the-fly dictionary generation, and dictionary caching.
An alternative technique which may be used for complex searching of multiple hybrid tables, is a mixed join operation which accesses data stored in its native row- or column-store orientation. Such a native mixed join approach is discussed in detail in U.S. patent application Ser. No. 13/323,530 titled “Mixed Join of Row and Column Database Tables in Native Orientation” filed Dec. 12, 2011, and incorporated by reference in its entirety herein for all purposes.
FIG. 12 shows a simplified schematic view of a native mixed join operation 1200. This approach utilizes a column dictionary as an index. Pruning of the join operation is achieved based upon filtering using the column dictionary index as a data boundary.
FIG. 13 shows another view of a native mixed join operation 1300, which uses the column dictionary as an index. First, records are iterated in the row table. The next step involves column dictionary lookup for each join condition. In the third step, matched records are returned.
By employing column dictionary lookup while iterating the row table according to a native mixed join approach, no result materialization is achieved. This also allows pipelined execution using the row identifier.
For multi-column join handling, a full key match may be achieved if a concatenated attribute or an external key exist. Column-by-column matching is possible.
FIG. 14 illustrates hardware of a special purpose computing machine which may be configured to implement a hybrid database table according to certain embodiments.
In particular, computer system 1400 comprises a processor 1402 that is in electronic communication with a non-transitory computer-readable storage medium 1403. This computer-readable storage medium has stored thereon code 1405 corresponding to the in-memory row partition. Code 1404 corresponds to an in-memory column partition, both of which may be accessed as part of the hybrid table according to embodiments as has been described above. The system may further comprise code corresponding to the data movement manager, as has been shown and described above.
The apparatuses, methods, and techniques described herein may be implemented as a computer program (software) executing on one or more computers. The computer program may further be stored on a non-transitory computer readable medium. The non-transitory computer readable medium may include instructions for performing the processes described.
The computer system may comprise a software server. A number of software servers together may form a cluster, or logical network of computer systems programmed with software programs that communicate with each other and work together to process requests.
An example computer system 1510 is illustrated in FIG. 15. Computer system 1510 includes a bus 1505 or other communication mechanism for communicating information, and a processor 1501 coupled with bus 1505 for processing information.
Computer system 1510 also includes a memory 1502 coupled to bus 1505 for storing information and instructions to be executed by processor 1501, including information and instructions for performing the techniques described above, for example. This memory may also be used for storing variables or other intermediate information during execution of instructions to be executed by processor 1501. Possible implementations of this memory may be, but are not limited to, random access memory (RAM), read only memory (ROM), or both.
A storage device 1503 is also provided for storing information and instructions. Common forms of storage devices include, for example, a hard drive, a magnetic disk, an optical disk, a CD-ROM, a DVD, a flash memory, a USB memory card, or any other medium from which a computer can read.
Storage device 1503 may include source code, binary code, or software files for performing the techniques above, for example. Storage device and memory are both examples of computer readable media.
Computer system 1510 may be coupled via bus 1505 to a display 1512, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. An input device 1511 such as a keyboard and/or mouse is coupled to bus 1505 for communicating information and command selections from the user to processor 1501. The combination of these components allows the user to communicate with the system. In some systems, bus 1505 may be divided into multiple specialized buses.
Computer system 1510 also includes a network interface 1504 coupled with bus 1505. Network interface 1504 may provide two-way data communication between computer system 1510 and the local network 1520. The network interface 1504 may be a digital subscriber line (DSL) or a modem to provide data communication connection over a telephone line, for example. Another example of the network interface is a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links are another example. In any such implementation, network interface 1404 sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information.
Computer system 1510 can send and receive information, including messages or other interface actions, through the network interface 1504 across a local network 1520, an Intranet, or the Internet 1530. For a local network, computer system 1510 may communicate with a plurality of other computer machines, such as server 1515. Accordingly, computer system 1510 and server computer systems represented by server 1515 may form a cloud computing network, which may be programmed with processes described herein.
In an example involving the Internet, software components or services may reside on multiple different computer systems 1510 or servers 1531-1535 across the network. The processes described above may be implemented on one or more servers, for example. A server 1531 may transmit actions or messages from one component, through Internet 1530, local network 1520, and network interface 1504 to a component on computer system 1510. The software components and processes described above may be implemented on any computer system and send and/or receive information across a network, for example.
The above description illustrates various embodiments of the present invention along with examples of how aspects of the present invention may be implemented. The above examples and embodiments should not be deemed to be the only embodiments, and are presented to illustrate the flexibility and advantages of the present invention as defined by the following claims. Based on the above disclosure and the following claims, other arrangements, embodiments, implementations and equivalents will be evident to those skilled in the art and may be employed without departing from the spirit and scope of the invention as defined by the claims.

Claims (17)

What is claimed is:
1. A computer-implemented method comprising:
providing in a non-transitory computer readable storage medium, a database created in an application level language and comprising row partitioned store and column partitioned store;
causing a row engine to insert a record into the row partitioned store;
performing an optimization based upon additional information to determine that a uniqueness violation may exist with respect to the record;
when it is determined that a uniqueness violation may exist, performing a search of the column partitioned store to determine whether a uniqueness violation exists with respect to the record;
receiving a query on the database;
performing a search on the row partitioned store of the database using the query;
selectively searching the column partitioned store using the query depending on an analysis of the search performed on the row partitioned store; and
returning a search result.
2. The computer-implemented method of claim 1 wherein the optimization comprises maintaining min/max values of unique columns in the column partitioned store with a latch-free structure to minimize contention.
3. The computer-implemented method of claim 1 wherein the optimization comprises maintaining a filter for unique columns.
4. The computer-implemented method of claim 3 wherein the filter comprises a Bloom filter.
5. The computer-implemented method of claim 3 wherein the filter is scalable.
6. The computer-implemented method of claim 3 further comprising updating or rehashing when data is moved from the row partitioned store to the column partitioned store.
7. A non-transitory computer readable storage medium embodying a computer program for performing a method, said method comprising:
receiving records to be inserted into a database created in an application level language in a non-transitory storage medium and comprising row partitioned store and column partitioned store;
causing a row engine to insert a record into the row partitioned store;
performing an optimization based upon additional information to determine that a uniqueness violation may exist with respect to the record;
when it is determined that a uniqueness violation may exist, performing a search of the column partitioned store to determine whether a uniqueness violation exists with respect to the record;
receiving a query on the database;
performing a search on the row partitioned store of the database using the query;
selectively searching the column partitioned store using the query depending on an analysis of the search performed on the row partitioned store; and
returning a search result.
8. The non-transitory computer readable storage medium of claim 7 wherein the optimization comprises maintaining min/max values of unique columns in the column partitioned store with a latch-free structure to minimize contention.
9. The non-transitory computer readable storage medium of claim 7 wherein the optimization comprises maintaining a filter for unique columns.
10. The non-transitory computer readable storage medium of claim 9 wherein the filter comprises a Bloom filter.
11. The non-transitory computer readable storage medium of claim 9 wherein the filter is scalable.
12. The non-transitory computer readable storage medium of claim 9 wherein the method further comprises updating or rehashing when data is moved from the row partitioned store to the column partitioned store.
13. A computer system comprising:
one or more processors;
a software program, executable on said computer system, the software program configured to:
receive records to be inserted into a database created in an application level language in a non-transitory storage medium and comprising row partitioned store and column partitioned store;
cause a row engine to insert a record into the row partitioned store;
perform an optimization based upon additional information to determine that a uniqueness violation may exist with respect to the record;
when it is determined that a uniqueness violation may exist, performing a search of the column partitioned store to determine whether a uniqueness violation exists with respect to the record;
receive a query on the database;
perform a search on the row partitioned store of the database using the query;
selectively search the column partitioned store using the query depending on an analysis of the search performed on the row partitioned store; and
return a search result.
14. The computer system of claim 13 wherein the optimization comprises maintaining min/max values of unique columns in the column partitioned store with a latch-free structure to minimize contention.
15. The computer system of claim 13 wherein the optimization comprises maintaining a filter for unique columns.
16. The computer system of claim 15 wherein the filter comprises a Bloom filter.
17. The computer system of claim 15 wherein the filter is scalable.
US13/334,669 2011-12-22 2011-12-22 Hybrid database table stored as both row and column store Active US8768927B2 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US13/334,669 US8768927B2 (en) 2011-12-22 2011-12-22 Hybrid database table stored as both row and column store
EP12006519.8A EP2608071A1 (en) 2011-12-22 2012-09-17 Hybrid database table stored as both row and column store
CN201210399140.1A CN103177055B (en) 2011-12-22 2012-10-19 It is stored as row storage and row stores the hybrid database table of the two
US14/272,769 US20140244628A1 (en) 2011-12-22 2014-05-08 Hybrid Database Table Stored as Both Row and Column Store

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US13/334,669 US8768927B2 (en) 2011-12-22 2011-12-22 Hybrid database table stored as both row and column store

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US14/272,769 Continuation US20140244628A1 (en) 2011-12-22 2014-05-08 Hybrid Database Table Stored as Both Row and Column Store

Publications (2)

Publication Number Publication Date
US20130166553A1 US20130166553A1 (en) 2013-06-27
US8768927B2 true US8768927B2 (en) 2014-07-01

Family

ID=47178442

Family Applications (2)

Application Number Title Priority Date Filing Date
US13/334,669 Active US8768927B2 (en) 2011-12-22 2011-12-22 Hybrid database table stored as both row and column store
US14/272,769 Abandoned US20140244628A1 (en) 2011-12-22 2014-05-08 Hybrid Database Table Stored as Both Row and Column Store

Family Applications After (1)

Application Number Title Priority Date Filing Date
US14/272,769 Abandoned US20140244628A1 (en) 2011-12-22 2014-05-08 Hybrid Database Table Stored as Both Row and Column Store

Country Status (3)

Country Link
US (2) US8768927B2 (en)
EP (1) EP2608071A1 (en)
CN (1) CN103177055B (en)

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9292560B2 (en) 2013-01-30 2016-03-22 International Business Machines Corporation Reducing collisions within a hash table
US9311359B2 (en) 2013-01-30 2016-04-12 International Business Machines Corporation Join operation partitioning
US9317517B2 (en) 2013-06-14 2016-04-19 International Business Machines Corporation Hashing scheme using compact array tables
US9405858B2 (en) 2013-06-14 2016-08-02 International Business Machines Corporation On-the-fly encoding method for efficient grouping and aggregation
US9672248B2 (en) 2014-10-08 2017-06-06 International Business Machines Corporation Embracing and exploiting data skew during a join or groupby
US9922064B2 (en) 2015-03-20 2018-03-20 International Business Machines Corporation Parallel build of non-partitioned join hash tables and non-enforced N:1 join hash tables
US10095764B2 (en) 2015-06-19 2018-10-09 Sap Se Multi-replica asynchronous table replication
US10108653B2 (en) 2015-03-27 2018-10-23 International Business Machines Corporation Concurrent reads and inserts into a data structure without latching or waiting by readers
US10210280B2 (en) 2014-10-23 2019-02-19 Sap Se In-memory database search optimization using graph community structure
US10235440B2 (en) 2015-12-21 2019-03-19 Sap Se Decentralized transaction commit protocol
US10268743B2 (en) 2015-06-19 2019-04-23 Sap Se Distributed database transaction protocol
US10298702B2 (en) 2016-07-05 2019-05-21 Sap Se Parallelized replay of captured database workload
US10303791B2 (en) 2015-03-20 2019-05-28 International Business Machines Corporation Efficient join on dynamically compressed inner for improved fit into cache hierarchy
US10353924B2 (en) 2015-11-19 2019-07-16 International Business Machines Corporation Data warehouse single-row operation optimization
US10552413B2 (en) 2016-05-09 2020-02-04 Sap Se Database workload capture and replay
US10572510B2 (en) 2015-12-21 2020-02-25 Sap Se Distributed database transaction protocol
US10585873B2 (en) 2017-05-08 2020-03-10 Sap Se Atomic processing of compound database transactions that modify a metadata entity
US10592528B2 (en) 2017-02-27 2020-03-17 Sap Se Workload capture and replay for replicated database systems
US10650011B2 (en) 2015-03-20 2020-05-12 International Business Machines Corporation Efficient performance of insert and point query operations in a column store
US10698892B2 (en) 2018-04-10 2020-06-30 Sap Se Order-independent multi-record hash generation and data filtering
US10769123B2 (en) 2016-09-30 2020-09-08 Microsoft Technology Licensing, Llc Workload-driven recommendations for Columnstore and Rowstore indexes in relational databases
US10795881B2 (en) 2015-12-18 2020-10-06 Sap Se Table replication in a database environment
US10831736B2 (en) 2015-03-27 2020-11-10 International Business Machines Corporation Fast multi-tier indexing supporting dynamic update
US10936578B2 (en) 2017-06-01 2021-03-02 Sap Se Client-driven commit of distributed write transactions in a database environment
US10977227B2 (en) 2017-06-06 2021-04-13 Sap Se Dynamic snapshot isolation protocol selection
US11347705B2 (en) 2019-04-02 2022-05-31 Sap Se Supporting scalable distributed secondary index using replication engine for high-performance distributed database systems
US11573947B2 (en) 2017-05-08 2023-02-07 Sap Se Adaptive query routing in a replicated database environment
US11615012B2 (en) 2020-04-03 2023-03-28 Sap Se Preprocessing in database system workload capture and replay
US11709752B2 (en) 2020-04-02 2023-07-25 Sap Se Pause and resume in database system workload capture and replay
US12061708B1 (en) * 2019-09-27 2024-08-13 Amazon Technologies, Inc. Remote tracking and identification of key access patterns for databases

Families Citing this family (50)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8918436B2 (en) * 2011-12-22 2014-12-23 Sap Ag Hybrid database table stored as both row and column store
US10885001B2 (en) * 2013-01-17 2021-01-05 International Business Machines Corporation System and method for assigning data to columnar storage in an online transactional system
US9632944B2 (en) 2013-04-22 2017-04-25 Sap Se Enhanced transactional cache
US9477609B2 (en) 2013-04-22 2016-10-25 Sap Se Enhanced transactional cache with bulk operation
CN103440245A (en) * 2013-07-15 2013-12-11 西北工业大学 Line and column hybrid storage method of database system
CN104424287B (en) * 2013-08-30 2019-06-07 深圳市腾讯计算机系统有限公司 Data query method and apparatus
US10311154B2 (en) * 2013-09-21 2019-06-04 Oracle International Corporation Combined row and columnar storage for in-memory databases for OLTP and analytics workloads
US10838926B2 (en) * 2013-10-01 2020-11-17 Sap Se Transparent access to multi-temperature data
US10360206B2 (en) 2014-01-16 2019-07-23 Hewlett Packard Enterprise Development Lp Latch-free concurrent searching
US9342544B2 (en) 2014-01-30 2016-05-17 International Business Machines Corporation Parallel load in a column-store database
US9697242B2 (en) * 2014-01-30 2017-07-04 International Business Machines Corporation Buffering inserts into a column store database
EP3093772B1 (en) * 2014-02-10 2018-07-18 Huawei Technologies Co., Ltd. Data processing method, data writing method, and related apparatus
US9547681B2 (en) 2014-05-30 2017-01-17 International Business Machines Corporation Combining row based and column based tables to form mixed-mode tables
KR101679011B1 (en) * 2014-06-26 2016-11-24 주식회사 알티베이스 Method and Apparatus for moving data in DBMS
US10671594B2 (en) * 2014-09-17 2020-06-02 Futurewei Technologies, Inc. Statement based migration for adaptively building and updating a column store database from a row store database based on query demands using disparate database systems
US20160078085A1 (en) * 2014-09-17 2016-03-17 Futurewei Technologies, Inc. Method and system for adaptively building and updating a column store database from a row store database based on query demands
US9836507B2 (en) * 2014-09-17 2017-12-05 Futurewei Technologies, Inc. Method and system for adaptively building a column store database from a temporal row store database based on query demands
CN105354323A (en) * 2015-11-16 2016-02-24 天津南大通用数据技术股份有限公司 Method and device for increasing precise inquiry speed of columnar storage database by using two-stage filtration
CN105426466A (en) * 2015-11-16 2016-03-23 天津南大通用数据技术股份有限公司 Method and apparatus for increasing accurate query speed of packing database
CN108140050B (en) * 2016-04-25 2020-08-25 华为技术有限公司 Method and device for filtering files by using bloom filter
US20180074967A1 (en) * 2016-09-09 2018-03-15 Sap Se Paging Mechanism for In-Memory Data Management System
CN106777027B (en) * 2016-12-08 2021-03-09 北京中电普华信息技术有限公司 Large-scale parallel processing row-column mixed data storage device and storage and query method
CN107092624B (en) * 2016-12-28 2022-08-30 北京星选科技有限公司 Data storage method, device and system
US10761946B2 (en) 2017-02-10 2020-09-01 Sap Se Transaction commit protocol with recoverable commit identifier
US10803039B2 (en) 2017-05-26 2020-10-13 Oracle International Corporation Method for efficient primary key based queries using atomic RDMA reads on cache friendly in-memory hash index
US10719446B2 (en) 2017-08-31 2020-07-21 Oracle International Corporation Directly mapped buffer cache on non-volatile memory
US10802766B2 (en) 2017-09-29 2020-10-13 Oracle International Corporation Database with NVDIMM as persistent storage
US11086876B2 (en) 2017-09-29 2021-08-10 Oracle International Corporation Storing derived summaries on persistent memory of a storage device
US10732836B2 (en) 2017-09-29 2020-08-04 Oracle International Corporation Remote one-sided persistent writes
US10956335B2 (en) 2017-09-29 2021-03-23 Oracle International Corporation Non-volatile cache access using RDMA
US11675761B2 (en) 2017-09-30 2023-06-13 Oracle International Corporation Performing in-memory columnar analytic queries on externally resident data
CN109033271B (en) * 2018-07-10 2021-03-02 上海达梦数据库有限公司 Data insertion method and device based on column storage, server and storage medium
US11449499B1 (en) * 2018-08-16 2022-09-20 Yellowbrick Data, Inc. System and method for retrieving data
US11170002B2 (en) 2018-10-19 2021-11-09 Oracle International Corporation Integrating Kafka data-in-motion with data-at-rest tables
US11016975B1 (en) 2019-12-26 2021-05-25 Snowflake Inc. Scan set pruning for pattern matching queries
US12050605B2 (en) 2019-12-26 2024-07-30 Snowflake Inc. Indexed geospatial predicate search
US10769150B1 (en) * 2019-12-26 2020-09-08 Snowflake Inc. Pruning indexes to enhance database query processing
US11681708B2 (en) 2019-12-26 2023-06-20 Snowflake Inc. Indexed regular expression search with N-grams
US11567939B2 (en) 2019-12-26 2023-01-31 Snowflake Inc. Lazy reassembling of semi-structured data
US11308090B2 (en) 2019-12-26 2022-04-19 Snowflake Inc. Pruning index to support semi-structured data types
US11372860B2 (en) 2019-12-26 2022-06-28 Snowflake Inc. Processing techniques for queries where predicate values are unknown until runtime
US10997179B1 (en) 2019-12-26 2021-05-04 Snowflake Inc. Pruning index for optimization of pattern matching queries
US11914589B2 (en) * 2020-02-28 2024-02-27 Sap Se Efficient computation of order by, order by with limit, min, and max in column-oriented databases
WO2021218280A1 (en) 2020-04-28 2021-11-04 Huawei Technologies Co., Ltd. Data storage and data retrieval methods and devices
US20230072930A1 (en) * 2021-09-09 2023-03-09 Servicenow, Inc. Database query splitting
US20230409601A1 (en) * 2022-06-09 2023-12-21 Futurewei Technologies, Inc. Apparatus and Method for Managing Storage of a Primary Database and a Replica Database
US12079360B2 (en) * 2022-06-30 2024-09-03 Atlassian Pty Ltd. Enabling limited database access using randomized limited access pointers
US11880369B1 (en) 2022-11-21 2024-01-23 Snowflake Inc. Pruning data based on state of top K operator
US20240193142A1 (en) * 2022-12-07 2024-06-13 Singlestore, Inc. Method of processing data in a database
CN117216068A (en) * 2023-11-07 2023-12-12 本原数据(北京)信息技术有限公司 Data storage method, device and storage medium supporting database mixed load

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5806074A (en) * 1996-03-19 1998-09-08 Oracle Corporation Configurable conflict resolution in a computer implemented distributed database
US6804667B1 (en) 1999-11-30 2004-10-12 Ncr Corporation Filter for checking for duplicate entries in database
US20050289129A1 (en) * 2004-06-23 2005-12-29 Winfried Schmitt Data processing systems and methods
US20090193006A1 (en) * 2008-01-07 2009-07-30 Ori Herrnstadt Multiple dimenisioned database architecture
US20090240663A1 (en) * 2007-09-21 2009-09-24 Hasso-Plattner-Institute Fur Softwaresystemtechnik Gmbh ETL-Less Zero-Redundancy System and Method for Reporting OLTP Data
US20100328115A1 (en) * 2009-06-28 2010-12-30 Carsten Binnig Dictionary-based order-preserving string compression for main memory column stores
US20110219020A1 (en) * 2010-03-08 2011-09-08 Oks Artem A Columnar storage of a database index
US20120084287A1 (en) * 2010-09-30 2012-04-05 Choudur Lakshminarayan Estimation of unique database values

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7024414B2 (en) * 2001-08-06 2006-04-04 Sensage, Inc. Storage of row-column data
CN101089846A (en) * 2006-06-16 2007-12-19 国际商业机器公司 Data analysis method, equipment and data analysis auxiliary method
CN101216821B (en) * 2007-01-05 2010-06-16 中兴通讯股份有限公司 Data acquisition system storage management method
CN101178732A (en) * 2007-12-12 2008-05-14 江苏省电力公司 Method for quick-speed realizing data store house process based on metadata
US8583692B2 (en) * 2009-04-30 2013-11-12 Oracle International Corporation DDL and DML support for hybrid columnar compressed tables
US20130198168A1 (en) * 2011-08-01 2013-08-01 Wei Huang Data storage combining row-oriented and column-oriented tables
US20130124483A1 (en) * 2011-11-10 2013-05-16 Treasure Data, Inc. System and method for operating a big-data platform

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5806074A (en) * 1996-03-19 1998-09-08 Oracle Corporation Configurable conflict resolution in a computer implemented distributed database
US6804667B1 (en) 1999-11-30 2004-10-12 Ncr Corporation Filter for checking for duplicate entries in database
US20050289129A1 (en) * 2004-06-23 2005-12-29 Winfried Schmitt Data processing systems and methods
US20090240663A1 (en) * 2007-09-21 2009-09-24 Hasso-Plattner-Institute Fur Softwaresystemtechnik Gmbh ETL-Less Zero-Redundancy System and Method for Reporting OLTP Data
US20090193006A1 (en) * 2008-01-07 2009-07-30 Ori Herrnstadt Multiple dimenisioned database architecture
US20100328115A1 (en) * 2009-06-28 2010-12-30 Carsten Binnig Dictionary-based order-preserving string compression for main memory column stores
US20110219020A1 (en) * 2010-03-08 2011-09-08 Oks Artem A Columnar storage of a database index
US20120084287A1 (en) * 2010-09-30 2012-04-05 Choudur Lakshminarayan Estimation of unique database values

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
A scalable bloom filter for membership queries by Kun Xie (hereafter Xie), Global Telecommunications Conference, 2007. GLOBECOM '07. IEEE, Nov. 26-30, 2007, pp. 543-547. *
Extended European Search Report (from a corresponding foreign application), EP 12006519.8, mailed Apr. 11, 2013.
Jens Dittrich et al., "Towards a One Size Fits All Databases Architecture," Proceedings of the 5th Biennial Conference on Innovative Data Systems Research (CIDR 2011), Jan. 6, 2011.
Oracle Database Performance Tuning Guide 11g Release 1 (11.1) B28274-02, Jul. 2008, chapter 14, p. 337-348. *
Ravishankar Ramamurthy et al., "A Case for Fractured Mirrors," Proceedings of the International Conference on Very Large Data Bases, VLDB Conference, Rome, Italy, Sep. 11-14, 2001.

Cited By (55)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9292560B2 (en) 2013-01-30 2016-03-22 International Business Machines Corporation Reducing collisions within a hash table
US9311359B2 (en) 2013-01-30 2016-04-12 International Business Machines Corporation Join operation partitioning
US9317548B2 (en) 2013-01-30 2016-04-19 International Business Machines Corporation Reducing collisions within a hash table
US9665624B2 (en) 2013-01-30 2017-05-30 International Business Machines Corporation Join operation partitioning
US9317517B2 (en) 2013-06-14 2016-04-19 International Business Machines Corporation Hashing scheme using compact array tables
US9367556B2 (en) 2013-06-14 2016-06-14 International Business Machines Corporation Hashing scheme using compact array tables
US9405858B2 (en) 2013-06-14 2016-08-02 International Business Machines Corporation On-the-fly encoding method for efficient grouping and aggregation
US9471710B2 (en) 2013-06-14 2016-10-18 International Business Machines Corporation On-the-fly encoding method for efficient grouping and aggregation
US10592556B2 (en) 2013-06-14 2020-03-17 International Business Machines Corporation On-the-fly encoding method for efficient grouping and aggregation
US9672248B2 (en) 2014-10-08 2017-06-06 International Business Machines Corporation Embracing and exploiting data skew during a join or groupby
US10489403B2 (en) 2014-10-08 2019-11-26 International Business Machines Corporation Embracing and exploiting data skew during a join or groupby
US10210280B2 (en) 2014-10-23 2019-02-19 Sap Se In-memory database search optimization using graph community structure
US10394783B2 (en) 2015-03-20 2019-08-27 International Business Machines Corporation Parallel build of non-partitioned join hash tables and non-enforced N:1 join hash tables
US10387397B2 (en) 2015-03-20 2019-08-20 International Business Machines Corporation Parallel build of non-partitioned join hash tables and non-enforced n:1 join hash tables
US11061878B2 (en) 2015-03-20 2021-07-13 International Business Machines Corporation Parallel build of non-partitioned join hash tables and non-enforced N:1 join hash tables
US9922064B2 (en) 2015-03-20 2018-03-20 International Business Machines Corporation Parallel build of non-partitioned join hash tables and non-enforced N:1 join hash tables
US10650011B2 (en) 2015-03-20 2020-05-12 International Business Machines Corporation Efficient performance of insert and point query operations in a column store
US10303791B2 (en) 2015-03-20 2019-05-28 International Business Machines Corporation Efficient join on dynamically compressed inner for improved fit into cache hierarchy
US11080260B2 (en) 2015-03-27 2021-08-03 International Business Machines Corporation Concurrent reads and inserts into a data structure without latching or waiting by readers
US10831736B2 (en) 2015-03-27 2020-11-10 International Business Machines Corporation Fast multi-tier indexing supporting dynamic update
US10108653B2 (en) 2015-03-27 2018-10-23 International Business Machines Corporation Concurrent reads and inserts into a data structure without latching or waiting by readers
US10990610B2 (en) 2015-06-19 2021-04-27 Sap Se Synchronization on reactivation of asynchronous table replication
US10095764B2 (en) 2015-06-19 2018-10-09 Sap Se Multi-replica asynchronous table replication
US10169439B2 (en) 2015-06-19 2019-01-01 Sap Se Multi-source asynchronous table replication
US10296632B2 (en) 2015-06-19 2019-05-21 Sap Se Synchronization on reactivation of asynchronous table replication
US11003689B2 (en) 2015-06-19 2021-05-11 Sap Se Distributed database transaction protocol
US10866967B2 (en) 2015-06-19 2020-12-15 Sap Se Multi-replica asynchronous table replication
US10268743B2 (en) 2015-06-19 2019-04-23 Sap Se Distributed database transaction protocol
US10353924B2 (en) 2015-11-19 2019-07-16 International Business Machines Corporation Data warehouse single-row operation optimization
US10929434B2 (en) 2015-11-19 2021-02-23 International Business Machines Corporation Data warehouse single-row operation optimization
US10795881B2 (en) 2015-12-18 2020-10-06 Sap Se Table replication in a database environment
US11327958B2 (en) 2015-12-18 2022-05-10 Sap Se Table replication in a database environment
US11372890B2 (en) 2015-12-21 2022-06-28 Sap Se Distributed database transaction protocol
US10235440B2 (en) 2015-12-21 2019-03-19 Sap Se Decentralized transaction commit protocol
US10572510B2 (en) 2015-12-21 2020-02-25 Sap Se Distributed database transaction protocol
US11294897B2 (en) 2016-05-09 2022-04-05 Sap Se Database workload capture and replay
US11829360B2 (en) 2016-05-09 2023-11-28 Sap Se Database workload capture and replay
US10552413B2 (en) 2016-05-09 2020-02-04 Sap Se Database workload capture and replay
US10298702B2 (en) 2016-07-05 2019-05-21 Sap Se Parallelized replay of captured database workload
US10554771B2 (en) 2016-07-05 2020-02-04 Sap Se Parallelized replay of captured database workload
US10769123B2 (en) 2016-09-30 2020-09-08 Microsoft Technology Licensing, Llc Workload-driven recommendations for Columnstore and Rowstore indexes in relational databases
US10592528B2 (en) 2017-02-27 2020-03-17 Sap Se Workload capture and replay for replicated database systems
US11314716B2 (en) 2017-05-08 2022-04-26 Sap Se Atomic processing of compound database transactions that modify a metadata entity
US10585873B2 (en) 2017-05-08 2020-03-10 Sap Se Atomic processing of compound database transactions that modify a metadata entity
US11573947B2 (en) 2017-05-08 2023-02-07 Sap Se Adaptive query routing in a replicated database environment
US11914572B2 (en) 2017-05-08 2024-02-27 Sap Se Adaptive query routing in a replicated database environment
US10936578B2 (en) 2017-06-01 2021-03-02 Sap Se Client-driven commit of distributed write transactions in a database environment
US11681684B2 (en) 2017-06-01 2023-06-20 Sap Se Client-driven commit of distributed write transactions in a database environment
US10977227B2 (en) 2017-06-06 2021-04-13 Sap Se Dynamic snapshot isolation protocol selection
US11468062B2 (en) 2018-04-10 2022-10-11 Sap Se Order-independent multi-record hash generation and data filtering
US10698892B2 (en) 2018-04-10 2020-06-30 Sap Se Order-independent multi-record hash generation and data filtering
US11347705B2 (en) 2019-04-02 2022-05-31 Sap Se Supporting scalable distributed secondary index using replication engine for high-performance distributed database systems
US12061708B1 (en) * 2019-09-27 2024-08-13 Amazon Technologies, Inc. Remote tracking and identification of key access patterns for databases
US11709752B2 (en) 2020-04-02 2023-07-25 Sap Se Pause and resume in database system workload capture and replay
US11615012B2 (en) 2020-04-03 2023-03-28 Sap Se Preprocessing in database system workload capture and replay

Also Published As

Publication number Publication date
US20140244628A1 (en) 2014-08-28
CN103177055A (en) 2013-06-26
EP2608071A1 (en) 2013-06-26
CN103177055B (en) 2017-09-05
US20130166553A1 (en) 2013-06-27

Similar Documents

Publication Publication Date Title
US10664497B2 (en) Hybrid database table stored as both row and column store
US10346383B2 (en) Hybrid database table stored as both row and column store
US8768927B2 (en) Hybrid database table stored as both row and column store
US11238039B2 (en) Materializing internal computations in-memory to improve query performance
US10204135B2 (en) Materializing expressions within in-memory virtual column units to accelerate analytic queries
US9513811B2 (en) Materializing data from an in-memory array to an on-disk page structure
US10474648B2 (en) Migration of unified table metadata graph nodes
EP2605158B1 (en) Mixed join of row and column database tables in native orientation
EP2811792B1 (en) A method for operating a mobile telecommunication device
US8924373B2 (en) Query plans with parameter markers in place of object identifiers
US9218394B2 (en) Reading rows from memory prior to reading rows from secondary storage
US8843436B2 (en) Systems and methods for performing direct reporting access to transaction databases
US20160078089A1 (en) Method and system for adaptively building a column store database from a temporal row store database based on query demands
CN113704300A (en) Data imprinting technique for use with data retrieval methods
KR20230096107A (en) Column-Based Techniques for Big Metadata Management
Dwivedi et al. Performance analysis of column oriented database vs row oriented database
Bach et al. Hybrid column/row-oriented DBMS
US11556540B1 (en) Transmission format cache for database reads

Legal Events

Date Code Title Description
AS Assignment

Owner name: SAP AG, GERMANY

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YOON, YONGSIK;JEONG, CHANHO;LEE, JUCHANG;AND OTHERS;SIGNING DATES FROM 20111220 TO 20111221;REEL/FRAME:027437/0805

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCF Information on status: patent grant

Free format text: PATENTED CASE

AS Assignment

Owner name: SAP SE, GERMANY

Free format text: CHANGE OF NAME;ASSIGNOR:SAP AG;REEL/FRAME:033625/0334

Effective date: 20140707

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551)

Year of fee payment: 4

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8