EP4535185A2 - Datensatzzusammenfassungsmetadaten zur bereitstellung verbesserter abfrageleistung - Google Patents

Datensatzzusammenfassungsmetadaten zur bereitstellung verbesserter abfrageleistung Download PDF

Info

Publication number
EP4535185A2
EP4535185A2 EP24203968.3A EP24203968A EP4535185A2 EP 4535185 A2 EP4535185 A2 EP 4535185A2 EP 24203968 A EP24203968 A EP 24203968A EP 4535185 A2 EP4535185 A2 EP 4535185A2
Authority
EP
European Patent Office
Prior art keywords
values
gap
value
sorted
stream
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP24203968.3A
Other languages
English (en)
French (fr)
Other versions
EP4535185A3 (de
Inventor
Gunes Aluc
Bojana Bislimovska
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SAP SE
Original Assignee
SAP SE
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SAP SE filed Critical SAP SE
Publication of EP4535185A2 publication Critical patent/EP4535185A2/de
Publication of EP4535185A3 publication Critical patent/EP4535185A3/de
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G06F16/24534—Query rewriting; Transformation
    • G06F16/24537—Query rewriting; Transformation of operators
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2462—Approximate or statistical queries
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/907—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/908—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2457—Query processing with adaptation to user needs
    • G06F16/24573—Query processing with adaptation to user needs using data annotations, e.g. user-defined metadata
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/40—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
    • H03M7/4031—Fixed length to variable length coding
    • H03M7/4037—Prefix coding
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/40—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
    • H03M7/42—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code using table look-up for the coding or decoding process, e.g. using read-only memory

Definitions

  • the present disclosure generally relates to providing summary metadata that can be used to help determine whether a summarized dataset contains data satisfying selection criteria.
  • Modern database systems are often tasked with processing huge amounts of data.
  • enterprise level database systems commonly have tables with hundreds of individual fields and often millions of rows, or more.
  • Loading and processing datasets can use large amounts of computing resources, as well as time. It can be useful to determine whether a particular dataset includes, or likely includes, relevant information before loading and processing a dataset.
  • the present disclosure provides a process of defining a gap filter and using the gap filter to determine whether a specified value may be present in a dataset for which the gap filter is defined.
  • At set of sorted unique values in a dataset sorted according to a sorting criterion is received.
  • For values of the set of unique values a plurality of gaps between at least a portion of the values of the set of unique values are determined.
  • the set of unique values is represented as a gap filter, the gap filter includes: (1) a starting value of the set of unique values, (2) a set of gaps lengths for the plurality of gaps; and (3) identifiers indicating a number of unique values between respective gaps of the plurality of gaps.
  • Summary metadata for the dataset is defined using the sorted set of values.
  • a request is received to determine whether a specified value is in the dataset. From the summary metadata, it is determined whether the specified value may be present in the dataset.
  • the set of values is sorted to provide a sorted set of values sorted according to a sorting criterion.
  • the sorting includes, for a second element in the set of unsorted values, determining that the second element is out of order compared with a first, preceding element in the set of unsorted values. It is determined whether the second element would be sorted compared with the first element if a current offset value was added to the second element. The current offset value is added to the second element. An intersection between the sorted set and another set of values is determined.
  • the present disclosure also includes computing systems and tangible, non-transitory computer-readable storage media configured to carry out, or includes instructions for carrying out an above-described method. As described herein, a variety of other features and advantages can be incorporated into the technologies as desired.
  • Modern database systems are often tasked with processing huge amounts of data.
  • enterprise level database systems commonly have tables with hundreds of individual fields and often millions of rows, or more.
  • Loading and processing datasets can use large amounts of computing resources, as well as time. It can be useful to determine whether a particular dataset includes, or likely includes, relevant information before loading and processing a dataset.
  • the numerical dataset may have a maximum value represented in the dataset and a minimum value represented in the dataset.
  • the maximum and minimum values thus provide summary information for the dataset.
  • the summary information for a dataset can be analyzed to determine whether the particular value is less than or equal to the maximum value or greater than or equal to the minimum value (that is, the query value is between the maximum value and the minimum value). If the query value is not within the maximum and minimum values, it can be determined that the query value is not represented in the dataset.
  • the summary information may not establish whether a particular data value is present in the dataset.
  • a value to be searched may be within the maximum and minimum values, but still not be present in the dataset.
  • the summary information can be used as a filter, where false positives can occur (an indication that a particular value is consistent with the summary information, but is not actually present in the dataset), but false negatives do not occur (if the particular value is not consistent with the summary information, then the particular value is definitely not in the dataset).
  • an initially constructed gap filter identifies all unique values present in the dataset and gaps between such values.
  • a given gap filter is analyzed to determine whether one or both of dictionary compression and run length encoding can be employed.
  • gaps can be removed from the gap filter. That is, a set of consecutive values can be considered together as a unit, even though one or more of such values are not present in the dataset.
  • a plurality of data streams can be defined, and data can be sorted into a different data stream whenever it is not possible to place a data value in the correct order in an existing data stream.
  • the number of possible data streams can be limited, and the offset value approach described above applied once the allowed number of data streams have been exhausted or before they are exhausted. When all of the data has been processed, data from the data streams can be merged.
  • FIG. 1 illustrates an example database environment 100.
  • the database environment 100 can include a client 104. Although a single client 104 is shown, the client 104 can represent multiple clients.
  • the client or clients 104 may be OLAP clients, OLTP clients, or a combination thereof.
  • the client 104 is in communication with a database server 106.
  • the database server 106 can process requests for database operations, such as requests to store, read, or manipulate data (i.e., CRUD operations).
  • a session manager component 108 can be responsible for managing connections between the client 104 and the database server 106, such as clients communicating with the database server using a database programming interface, such as Java Database Connectivity (JDBC), Open Database Connectivity (ODBC), or Database Shared Library (DBSL).
  • JDBC Java Database Connectivity
  • ODBC Open Database Connectivity
  • DBSL Database Shared Library
  • the session manager 108 can simultaneously manage connections with multiple clients 104.
  • the session manager 108 can carry out functions such as creating a new session for a client request, assigning a client request to an existing session, and authenticating access to the database server 106.
  • the session manager 108 can maintain a context that stores a set of parameters related to the session, such as settings related to committing database transactions or the transaction isolation level (such as statement level isolation
  • the client can interface with an application manager component 110.
  • the application manager 110 can be located outside of, but in communication with, the database server 106.
  • the application manager 110 can initiate new database sessions with the database server 106, and carry out other functions, in a similar manner to the session manager 108.
  • the application manager 110 can determine the type of application making a request for a database operation and mediate execution of the request at the database server 106, such as by invoking or executing procedure calls, generating query language statements, or converting data between formats useable by the client 104 and the database server 106.
  • the application manager 110 receives requests for database operations from a client 104, but does not store information, such as state information, related to the requests.
  • a connection is established between the client 104 and the database server 106, including when established through the application manager 110, execution of client requests is usually carried out using a query language, such as the structured query language (SQL).
  • SQL structured query language
  • the session manager 108 and application manager 110 may communicate with a query interface 112.
  • the query interface 112 can be responsible for creating connections with appropriate execution components of the database server 106.
  • the query interface 112 can also be responsible for determining whether a request is associated with a previously cached statement or a stored procedure, and calling the stored procedure or associating the previously cached statement with the request.
  • At least certain types of requests for database operations can be associated with a transaction context.
  • each new session can be assigned to a transaction.
  • Transactions can be managed by a transaction manager component 114.
  • the transaction manager component 114 can be responsible for operations such as coordinating transactions, managing transaction isolation, tracking running and closed transactions, and managing the commit or rollback of transactions. In carrying out these operations, the transaction manager 114 can communicate with other components of the database server 106.
  • the query interface 112 can communicate with a query language processor 116, such as a structured query language processor. For example, the query interface 112 may forward to the query language processor 116 query language statements or other database operation requests from the client 104.
  • the query language processor 116 can include a query language executor 120, such as a SQL executor, which can include a thread pool 124. Some requests for database operations, or components thereof, can be executed directly by the query language processor 116. Other requests, or components thereof, can be forwarded by the query language processor 116 to another component of the database server 106. For example, transaction control statements (such as commit or rollback operations) can be forwarded by the query language processor 116 to the transaction manager 114.
  • the query language processor 116 is responsible for carrying out operations that retrieve or manipulate data (e.g., SELECT, UPDATE, DELETE). Other types of operations, such as queries, can be sent by the query language processor 116 to other components of the database server 106.
  • the query interface 112, and the session manager 108, can maintain and manage context information associated with requests for database operation. In particular implementations, the query interface 112 can maintain and manage context information for requests received through the application manager 110.
  • a client request such as a query
  • a thread of the thread pool 124 can be assigned to a thread of the thread pool 124, such as using the query interface 112.
  • a thread is associated with a context for executing a processing activity.
  • the thread can be managed by an operating system of the database server 106, or by, or in combination with, another component of the database server.
  • the thread pool 124 contains a plurality of threads.
  • the number of threads in the thread pool 124 can be dynamically adjusted, such in response to a level of activity at the database server 106.
  • Each thread of the thread pool 124 in particular aspects, can be assigned to a plurality of different sessions.
  • the session manager 108 or the application manager 110 can determine whether an execution plan for the query already exists, such as in a plan cache 136. If a query execution plan exists, the cached execution plan can be retrieved and forwarded to the query language executor 120, such as using the query interface 112. For example, the query can be sent to an execution thread of the thread pool 124 determined by the session manager 108 or the application manager 110. In a particular example, the query plan is implemented as an abstract data type.
  • the query can be parsed using a query language parser 128.
  • the query language parser 128 can, for example, check query language statements of the query to make sure they have correct syntax, and confirm that the statements are otherwise valid. For example, the query language parser 128 can check to see if tables and records recited in the query language statements are defined in the database server 106.
  • the query can also be optimized using a query language optimizer 132.
  • the query language optimizer 132 can manipulate elements of the query language statement to allow the query to be processed more efficiently. For example, the query language optimizer 132 may perform operations such as unnesting queries or determining an optimized execution order for various operations in the query, such as operations within a statement.
  • an execution plan can be generated, or compiled, for the query. In at least some cases, the execution plan can be cached, such as in the plan cache 136, which can be retrieved (such as by the session manager 108 or the application manager 110) if the query is received again.
  • one task that can be performed by the query language optimizer 132 is determining a location where a request for a database operation, or a portion thereof, should be performed.
  • a complex query may be submitted that reads data from multiple data sources.
  • At least one of the data sources may be a virtual table, and the request can be performed on a replica table of the database server 106, or the request can be sent to a remote system via data federation, and results returned to the database server.
  • the query language executor 120 can oversee the execution of an execution plan for the query. For example, the query language executor 120 can invoke appropriate subcomponents of the database server 106.
  • the query language executor 120 can call a query processor 140, which can include one or more query processing engines.
  • the query processing engines can include, for example, an OLAP engine 142, a join engine 144, an attribute engine 146, or a calculation engine 148.
  • the OLAP engine 142 can, for example, apply rules to create an optimized execution plan for an OLAP query.
  • the join engine 144 can be used to implement relational operators, typically for non-OLAP queries, such as join and aggregation operations.
  • the attribute engine 146 can implement column data structures and access operations. For example, the attribute engine 146 can implement merge functions and query processing functions, such as scanning columns.
  • the query processing engines of the query processor 140 can access data stored in the database server 106.
  • Data can be stored in a row-wise format in a row store 162, or in a column-wise format in a column store 164. In at least some cases, data can be transformed between a row-wise format and a column-wise format.
  • a particular operation carried out by the query processor 140 may access or manipulate data in the row store 162, the column store 164, or, at least for certain types of operations (such a join, merge, and subquery), both the row store 162 and the column store 164.
  • the row store 162 and the column store 164 can be maintained in main memory.
  • a persistence layer 168 can be in communication with the row store 162 and the column store 164.
  • the persistence layer 168 can be responsible for actions such as committing write transaction, storing redo log entries, rolling back transactions, and periodically writing data to storage to provided persisted data 172.
  • the database server 106 may need to access information stored at another location, such as another database server.
  • the database server 106 may include a communication manager 180 component to manage such communications.
  • the communication manger 180 can also mediate communications between the database server 106 and the client 104 or the application manager 110, when the application manager is located outside of the database server.
  • the database server 106 can include components to coordinate data processing operations that involve remote data sources.
  • the database server 106 includes a data federation component 190 that at least in part processes requests to access data maintained at remote system.
  • the data federation component 190 can include one or more adapters 192, where an adapter can include logic, settings, or connection information usable in communicating with remote systems, such as in obtaining information to help generate virtual parameterized data objects or to execute requests for data using virtual parameterized data objects (such as issuing a request to a remote system for data accessed using a corresponding parameterized data object of the remote system).
  • adapters include "connectors" as implemented in technologies available from SAP SE, of Walldorf, Germany.
  • disclosed techniques can use technologies underlying data federation techniques such as Smart Data Access (SDA) and Smart Data Integration (SDI) of SAP SE.
  • SDA Smart Data Access
  • SDI Smart Data Integration
  • gaps filters Just as data within a database is dynamic and evolves over time, the associated gap filters, zone maps, or bloom filters need are typically maintained updated. Such updates can be handled by dedicated components within the database server 106 or can be integrated into existing components like the persistence layer 168.
  • summary information 222 (which can also be referred to as summary metadata - metadata that summarizes the data in one of the data subsets) for the data subsets 214, 216, 218.
  • the summary information 222 can be used to help identify what information is stored in a given data subset 214, 216, 218 so that it can be determined whether such data subset should be retrieved and further processed.
  • the summary information 222 can correspond to an index or filter, including the gap filters of the present disclosure. Without some kind of summary information 222, each data subset 214, 216, 218 may need to be loaded and analyzed to determine whether it contains relevant information. Thus, while having data subsets 214, 216, 218 may be useful for other reasons, the summary information 222 allows for decisions to be made as to what data subsets should be loaded and further processed.
  • FIG. 2 also illustrates a process 250 for analyzing the data subsets 214, 216, 218.
  • a query is received, such a database query that requests data satisfying one or more particular selection criteria reflected in a WHERE clause. It can be determined at 258 whether data subsets 214, 216, 218 in the dataset 208 remain to be analyzed. If not, the process 250 can end at 262. Otherwise, the process 250 can proceed to 266 where a next data subset 214, 216, 218 is selected and its summary information 222 retrieved. In some cases, including with particular implementations of disclosed gap filters, the summary information 222 may be compressed, and, if so, the summary information can be decompressed at 270.
  • the summary information 222 is analyzed at 274. At 278, it is determined whether the is a probability of relevant data being in the data subset. For example, a gap filter can be consulted to determine whether a particular value falls within a gap, is present as a unique value, or is located within a sequence of consecutive values. As noted above, the results of analyzing the gap filter, when configured as a true filter, will not result in a false negative result, but may provide a false positive. If the data subset 214, 216, 218 was not identified as having relevant data using the summary information 222, the process 250 can return to 258. Otherwise, the data subset 214, 216, 218 can be loaded and analyzed according to a particular use scenario at 282, and then the process 250 can return to 258. Again, at least certain summary information 222 can be associated with false positive results, and so it may be determined at 282 from the actual data in the data subset 214, 216, 218 that the data subset does not have data relevant to the query received at 254.
  • Example 4 Example Gap Filter Definition and Compression
  • FIG. 3 provides a flowchart of general operations in a process 300 of constructing a gap filter according to the present disclosure.
  • unique values in a particular data subset are identified. Note that while typically gap filters are used with multiple data subsets of a dataset, the techniques of the present disclosure, including the process 300, can be used with a single dataset, which can correspond to the data subset of the operations in the process.
  • Sorting can be accomplished using any suitable sorting technique, and the data can be fully sorted or partially sorted.
  • the data in the particular data subset of the process 300 can itself be divided into different subsets, where data in each subset is ordered with respect to the data in that subset.
  • Suitable sorting techniques include bubble sort, selection sort, insertion sort, merge sort, or quick sort.
  • the lazy sorting and stream sorting techniques described herein can also be used.
  • a representation of the sorted unique values is created at 312.
  • the representation identifies gaps between unique values in the data subset, and thus can be used to also identify unique values that are (or at least may be) present in the data subset. In some cases, information is stored both about gaps in the data subset and a number of consecutive unique values that may be present in the data subset.
  • An example gap filter representation is provided in FIG. 4A , and the creation and use of this example representation will be described in detail as the disclosure proceeds.
  • the representation of unique values is compressed at 316.
  • the representation can be compressed using one or both of dictionary compression or run length encoding.
  • the run length encoding information can be further encoded, such as according to a dictionary compression schema, at 320.
  • the compressed representation is stored as summary information at 324.
  • Example 5 Example Gap Filter Manipulation for Improved Compression
  • FIGS. 4A-4D illustrate an example representation of a gap filter of the present disclosure and example operations that can be performed to enhance the compressibility of the gap filter.
  • a set of unique values 404 is determined. That is, a given value of the unique values 404 occurs at least once in the dataset or may occur multiple times. However, even when the unique value is included in the dataset multiple times, it is only represented once in the set of unique values 404.
  • the set of unique values 404 is shown as unsorted.
  • the set of unique values 404 can be sorted to provide a sorted set of unique values 408. Although the sorted set of unique values 408 is shown as fully sorted, in at least some cases, including as will be further described, the sorted set of unique values is not truly sorted.
  • the data in the set 408 may not include all of the original unique values of the set 404, but rather may include modified unique values where the modified unique values and unmodified unique values are sorted in the set 408. That is, the sorting process may produce a list of values that is sorted, but the sorting is "superficial," in a way, in that the underlying values themselves are not fully sorted.
  • a first representation 412 of the sorted set of unique values 408 can be created.
  • the first representation 412 includes a number of elements 414, 416.
  • the elements 414 correspond to unique values of the set of sorted unique values 408 and are placed in the first representation 412 according to their sorted order.
  • Elements 416 are shown as having a value of "" where the value can, depending on the scenario, indicate a gap or can indicate a series of consecutive unique values of the set of sorted unique values 408. In other cases, different values can be used to represent gaps and sequences of consecutive unique values.
  • a unique value in the representation need not correspond to a unique value in the set 408. That is, unique values in the gap filter representation 412 include values of the set 408, but may also include one or more unique values that are not in the set. Introduction of unique values that are not in the set 408 correspond to false positive results, which decrease the accuracy of the gap filter, but can improve its compressibility.
  • the first representation 412 includes a first set of values 422 above the elements 416, representing gaps, and a second set of values 424 underneath the elements 414, representing a number of consecutive unique elements when an element 416 serves this purpose, as opposed to indicating a gap between unique values.
  • the values 424 can be referred to as "nSet" values (for number in set).
  • a value 422 above an element 414 indicates that the element corresponds to a gap, and the value 422 is the length of a gap.
  • a value 424 greater than one underneath an element 414 indicates that the following element 416 represents a number of consecutive unique values, rather than representing a gap length.
  • the remove gap operation 432 results in the insertion of a value into a gap, so that the gap is eliminated. Since the value inserted into the gap is not a value of the original set of unique values 404, it introduces a possibility, in this case a single possibility, of a false positive value being provided by the gap filter of the second representation 430.
  • the outcome of the remove gap operation 432 is reflected in the removal of the element 422b, the update of the value of the element 424a from “1" to "3" (indicating a sequence of three consecutive unique values) and the removal of the element 424c. That is, the updated value of the element 424a, and the removal of elements 422b and 424c, indicate that the element 414a represents a consecutive sequence of the unique values, starting at "49" and ending at "51.”
  • the operation 448 involves analyzing instances of the different gap sizes for the third representation 442 to determine whether a gap size can be decreased such that instead of being represented by a unique value in the set of gap sizes, it can be represented as an instance of another gap size that is already in the set of gap sizes. Note that at least in some cases the operation 448 does not involve increasing a gap size, as that could lead to false negative results.
  • the code 458 checks to see if it would reduce the number of distinct gap sizes. If so, the gap g is reduced, otherwise the code 458 continues if there are additional, smaller gap sizes to be analyzed.
  • the operation types 432, 448, 482 described in the discussion FIGS. 4A-4C can be applied in a particular order, including as shown in the code 496 of FIG. 4D .
  • the code 496 processes gaps from the smallest gap size to the largest gap size. In at least some cases, if there are multiple gaps of the same size, the code 496 processes gaps of this size from left to right. As long as gaps remain, the code 496 tries to apply the operations 432, 448, 482 in the listed order. That is, the code 496 first tries to reduce the number of distinct gap values for a representation of the set of sorted unique values 408, then tries to create longer consecutive runs of a particular gap size, and finally checks to see if any gaps can be removed. Again, because of the use of a budget variable in the operations 432, 448, 482, the code 496 only executes so long as there is remaining budget, allowing for a balance between compressibility and a number of false positive results.
  • FIGS. 5A and 5B provide a more complete analysis of the first representation 412 of the sorted set of unique values 408, which is shown as representation 504 of FIG. 5A . It can be seen that initially a budget for manipulating the sorted set of unique values 408 is 21, and that the first representation 504 has a total of 7 unique gap values. It is determined that a gap 508 can be removed at the cost of one false positive, resulting in a second representation 512.
  • a gap 516 can be removed in a similar manner as the gap 508 to provide a third representation 518.
  • multiple operations are considered for the third representation 518.
  • a gap 520 it can be seen that the gap length of 1 cannot be reduced to eliminate a discrete gap length, and that the gap length cannot be reduced to create a longer run of a common gap length. However, the gap 520 can be removed. Gaps 522 and 524 can also be removed in the remove gap operation, at a cost of 11 false positives. The next smallest gap, gap 526 can then be analyzed.
  • the gap 524 cannot be shortened to reduce the number of unique gap lengths (since the value of 1 was already removed as an element as a result of the preceding operations), nor can its length be reduced to increase a run length of a particular gap size.
  • the gap can be removed at a cost of two false positives.
  • a gap 530 with a length of 11, is now the smallest gap, and is analyzed next. In this case, it can be seen that gap 530 is the only gap having a gap length of 11. Thus, if the length of gap 530 can be reduced to a value of another gap, the gap length of 11 can be removed from the set of unique gap lengths. However, there are no smaller gaps in the set of unique gap lengths, and so this operation cannot be performed. In a similar manner, the size of the gap 530 cannot be reduced to create a longer run of a particular gap value. Since the size of the gap 530 exceeds the available budget, it cannot be removed. Thus, no action can be taken for the gap 530, and so the analysis process proceeds to analyze a gap 536, which is the leftmost occurrence of the next smallest gap size of 12.
  • the analysis can proceed to the next instance of gap size 12, gap 538.
  • the analysis of the gap 538 is similar to that of the gap 536 and no operations can be performed.
  • a gap 540 having the next largest gap size of 13 is analyzed.
  • analyzing the gap 540 indicates that reducing the size of the gap 13 will not result in the removal of a unique gap length.
  • reducing the size of the gap from 13 to 12 will create a longer consecutive run of gaps having length 12. Accordingly, this operation is performed to provide a fifth representation 544 of the sorted set of unique values 408. The performance of this operation reduces the budget from 6 to 5, as the reduction of the length of the gap 540 by one results in the introduction of another false positive result.
  • gap 546 in the representation 544, it is determined that reducing the size of the gap from 17 to 12 would remove a distinct gap value. Since this introduces 5 false positive results, and the remaining budget is five, this operation can be performed to provide a sixth representation 560 of the sorted set of unique values 408, shown in FIG. 5B . Since gap 546 was the last remaining gap to be analyzed, the analysis of the initial set of sorted unique values 408 is complete.
  • FIG. 5B also illustrates a technique for compressing/encoding a representation of a set of sorted unique values, including after any processing to try and improve the compression of such set.
  • the compression process is illustrated using the sixth representation 560.
  • FIG. 5B illustrates a dictionary 564 that can be used for dictionary compression.
  • the dictionary provides 564 values to be used to indicate a number of consecutive unique values and values used to indicate gap lengths.
  • the same dictionary 564 is used for both purposes, and so a given value is included in the dictionary a single time whether it is included in one set or both sets, and regardless of the number of times it is present in any set.
  • the set of values to be included in the dictionary includes six elements (1, 4, 6, 11, 12, and 20).
  • the dictionary 564 will include an additional element indicating that run length encoding is used for a particular gap (or for a particular nSet value) in addition to dictionary encoding.
  • values are encoded in binary (four bits). A value with a leading 0 is used to indicate run length encoding, where the remaining three bits indicate the length of a run. Note that this particular encoding scheme provides that the maximum run length that can be represented by a single value is 10. As shown, a minimum run length for dictionary compression is three, since a value of 0 does not make sense, a value of 1 indicates a single occurrence of a value, and not a run of multiple occurrences of the value. In this implementation, runs of only two consecutive occurrences of a value are also not compressed using run length encoding, as they do not result in significant compression.
  • the dictionary 564 shows that a value that corresponds to the trailing bits of the dictionary value is the number encoded by the three trailing bits, plus three. So, "0000" is initially interpreted as run length encoding with a value of 0 indicated by the three trailing bits. However, when decoded, three is added to the 0, to provide a final run length of 3.
  • Bit representations of the values 8-115 are used to represent lengths of consecutive unique values or particular gap sizes (either an isolated gap or a gap that is part of a run of a given gap size).
  • dictionary or run length compression may be performed/encoded in another manner.
  • the disclosed binary representation is beneficial because it easily allows for both run length encoding and dictionary compression in a single representation, and because binary values can be more compressible than other values.
  • the set of values 570 corresponds to the values 572, where each value represents a number of consecutive unique values in a particular set of consecutive unique values (nSet values), followed by the values 574 representing the gap sizes.
  • nSet values a number of consecutive unique values in a particular set of consecutive unique values
  • the actual values in a set of ordered, unique values can be determined using the values 572, the values 574, and the starting/smallest value of the set of ordered, unique values (in this particular example, 5).
  • the set 570 includes a single run of multiple values, in this case four consecutive occurrences of the value 12, all of which correspond to gap lengths 574.
  • the dictionary 564 is used to indicate the use of run length encoding, and the length of the run. As explained above, a leading bit value of 0 is used to indicate run length encoding, and the remaining bits are used to indicate the length of the run.
  • a set of decompressed values 624 resulting from the sequence 616 can be used to create a representation 632 of the gap filter encoded by the values 604, corresponding to the gap filter 560 of FIG. 5B .
  • the values in the set of value 604, and thus the values in the set of decompressed values 624 are ordered by having values representing a number of consecutive unique values between gaps, ordered from left to right, followed by values representing gaps, where the gaps values are also ordered from left to right (in this particular example, from the smallest unique value to the largest unique value in the originally provided sorted set of unique values, such as the set 604).
  • Example 8 Example Use of Gap Filter
  • FIGS. 7A and 7B illustrate examples of how gap filters of the present disclosure can be used.
  • the examples use the gap filter representation 632 of FIG. 6 .
  • the present disclosure provides various sorting techniques that can be used in defining the disclosed gap filters, as well as in other applications.
  • the sorting techniques can be used with other type of information, such as summary information, used to improve query evaluation, including using indexes or other types of filters, such as in database systems, including relational database systems.
  • incrementing of an offset value may need to performed multiple times before a current element is sorted. That is, when the increment value is selected as described above, a single incrementation of the offset value will result in the new offset value placing the current element in sorted order. However, if the increment value is less than the difference between the maximum and minimum elements in the list of values, then multiple incrementations may need to be performed before the current element plus the offset value is in sorted order. This is shown by the alternative path in FIG. 8 , where the process 800 can proceed from 828 to 820.
  • the next value in the set 908, 18, is analyzed. It can be seen that placing the value of 18 after the value of 36 in the set 912 would cause that set to no longer be sorted, as shown at step 924. Accordingly, a current offset value is updated, in this case from a starting value of 0 to a value of 100, and the value of 100 is added to the value of 18 to provide an updated value of 118. As shown at step 928, the set 912 is ordered with the manipulation of the value of 18, since 118 is larger than 36.
  • Steps 940, 942, and 944, for the values of 60, 65, and 67 are analogous to step 936. That is, these elements would be unsorted if placed in the set 912 without alteration, but are sorted with the addition of the current offset value of 100. However, now consider the element with the value of 49 in the set 904. Placing the element in the set 912 would result in the set being out of order, since 49 is less than 167, as shown at 946. However, the element is still out of order even with the addition of the current offset value of 100, since 149 is still less than 167. Accordingly, the current offset value is increased from 100 to 200, and then added to provide a value of 249. Placing 249 in the set 912 results in the set being in order, as shown at step 948.
  • the next value in the set 904, 96 is in order after the addition of the current offset value of 200, as shown at step 952.
  • the next value in the set 904, 63 is not in order with respect to the set 912, and would not be in order even with the addition of the current offset value, since 263 is still less than 296. Accordingly, the current offset value is updated from 200 to 300, added to 63 and the resultant value of 363 is placed in the set 912, as shown at 954.
  • the 912 is in order, since 363 is greater than 296.
  • the offset value is increased once again from 300 to 400, added to the value of 5, which is then added to the set 912 as shown at step 956.
  • the set 912 remains ordered, since 405 is greater than 363.
  • Step 960 shows the results of the next operation, where adding the current offset value of 400 to the next element of the set 904, 93, results in the set 912 being ordered.
  • the last element of the set 904, having a value of 81 the element would not provide an ordered set 912 even after adding the current offset value of 400, since 481 is less than 493. So, the offset value is increased a final time, from 400 to 500, added to the element, and the updated element value added to the set 912, as shown in step 964.
  • the set 912 is ordered, since 581 is greater than 493.
  • an ordered set of elements 970 resulting from applying the process 800 to the set of elements provides ordered elements, superficially, but at least a subset of the elements may be unordered. That is, for example, even though values 36 and 118 appear ordered in the set 970, the actual values represented by those elements, 36 and 18, remain unordered.
  • processes using a lazily sorted set of values can be defined to account for the facts that underlying values may not be fully sorted.
  • the particular value can be searched using its ordered value, and the original value combined with each distinct offset value up to the final increment value. That is, for example, if the final increment value is 500, and the increment size is 100, it is known that offset values of 100, 200, 300, 400, and 500 are used in the set 970.
  • the set can be searched for the value, the value plus 100, and so on, up the value plus 500, the maximum off set value used based on the final increment value (and knowing the increment rate).
  • the use of the lazy sorting technique can reduce the computational expense of providing indexing or filtering functionality, such as with a gap filter. While extra computational costs may be incurred when it is desired to determine whether a particular value is present in a particular dataset, these costs can overall be lower than performing a strict sort initially.
  • performing multiple queries of a gap filter created using lazily sorted data can be performed in parallel.
  • FIG. 10 illustrates a flowchart for an example process 1000 for conducting a stream sort.
  • a list (or other type of set) of elements is received at 1004.
  • a first stream is initialized, and the first element of the set is placed in the first stream at 1012. It is determined at 1016 whether the set has additional elements that have not yet been sorted. If not, any streams that have been populated are merged at 1020, and the process 1000 ends at 1024.
  • the element is placed into the stream where the element has the smallest difference between its value and the value of the last element currently in the stream. For example, consider that a value of 65 could be placed into a stream with a last value of 57 or another stream with a last value of 34. Since the difference between 65 and 57 is smaller than the difference between 65 and 34, the value of 65 is placed in the stream with the value of 57. The process 1000 then returns to 1016.
  • the process 1000 can employ the lazy sorting technique FIG. 8 at 1044. Namely, an offset value can be added to the current element, and then the process 1000 can proceed to 1032.
  • FIG. 12 illustrates an intersection determination process 1200 using disclosed sorting techniques, as well as a concrete example of this process.
  • two or more lists are obtained. While the term list is used for convenience of presentation, it should be understood that the process 1200 can be used with other specific types of collections, such as sets.
  • the shorter list (the list having the smaller number of elements) is sorted using a disclosed technique at 1208. In particular, the shortest list is sorted using a disclosed lazy sort technique, alone or in combination with stream sorting.
  • the process 1200 can also be used with lists sorted with the disclosed stream sorting process, where the sorting does not also use the lazy sort technique. In this case, operations of the process 1200 related to false positive values or offset values can be omitted.
  • a binary search process is initiated at 1212.
  • the sorted, shorter list is probed using values in the longer list.
  • the sorted nature of the sorted list allows the binary search technique to be performed.
  • the process 1200 is described for a binary search algorithm, disclosed intersection determination processes and sorting techniques can be used with other types of search processes, including interpolation search, exponential search, Fibonacci search, jump search, ternary search, hash table lookups, range queries, and interval tree searches.
  • the offset value is subtracted from the value in the candidate intersection at 1248. Also at 1248, the resultant values from removing the offset values are compared with the larger, probe list at 1252. False positives are removed from the candidate intersection at 1256. That is, if there is no longer a match between the probe (larger) list and the shorter list after the removal of the offset value, the element corresponds to a false positive, since the "real"/"true" value is not both lists. The updated list then corresponds to the intersection, which is returned at 1244.
  • Data set 1268 corresponds to a partially sorted version of the smaller data set 1262.
  • the first three elements of the data set 1262 are sorted in the data set 1268.
  • the offset value of 100 is added to the value to provide an incremented value of 103, which is then in sorted order in the data set 1268.
  • the sorting process continues in a similar manner to provide the final, sorted version 1272 of the smaller list 1262. It can be seen that the offset value was further incremented from 100 to 200 during sorting.
  • FIG. 12 depicts a set 1274 of possible offset values.
  • the value of 65 will be used in the probe as well as the values 165 and 265, as shown at 1276, since the value of 65 could appear as the original value or as the combined with either of the two offset values as a result of the lazy sort process. Similar probes are conducted for the remaining values of the larger set 1260.
  • the values in the candidate intersection 1280 are compared with the values in the sorted set 1272 and the values in the larger set 1260 to determine whether the values in fact match. In the case of 78, both sets contain the value and so it is a true match. Looking at the value of 103, it can be determined that the value of 103 in the sorted set 1272 actually represents the value of 3. Since the sorted set 1272 does not contain the actual value of 103, 103 is a false positive and is removed from the candidate intersection 1280.
  • 3 is a true match with the sorted set 1272, since the value of 103 in the sorted set is a representation of the value of 3. So, 3 remains in the candidate intersection 1280.
  • the value of 236 is considered. While the sorted set 1272 includes the value of 236, it is really a representation of the value 36. Since the sorted set 1272 does not include an actual value of 236, that value is a false positive, and is removed from the candidate intersection 1280. The process results in the final, true intersection 1288.
  • a request is received at 1316 to determine whether a specified value is present in the dataset.
  • the set of values is sorted at 1406 to provide a sorted set of values sorted according to a sorting criterion.
  • the sorting includes, at 1408, for a second element in the set of unsorted values, determining that the second element is out of order compared with a first, preceding element in the set of unsorted values.
  • the current offset value is added to the second element at 1412.
  • Summary metadata for the dataset is defined at 1414 using the sorted set of values.
  • a request is received to determine whether a specified value is in the dataset. From the summary metadata, it is determined at 1418 whether the specified value may be present in the dataset.
  • FIG. 14B provides a flowchart of a process 1430 of defining summary information useable to determine whether a particular value may be present in a data set, including sorting a set of values for use in preparing the summary information by sorting elements of the set of values into data streams.
  • a set of values is received that is at least partially unsorted, the set of values representing values in a dataset.
  • the set of values is sorted at 1436 to provide a sorted set of values sorted according to a sorting criterion.
  • the sorting includes, at 1438, for an element of the set of unique values, determining whether the element is in a sorted order if placed in a first stream of processed values of the set of values.
  • Summary metadata for the dataset is defined at 1446 using the sorted set of values.
  • a request is received to determine whether a specified value is in the dataset. From the summary metadata, it is determined at 1450 whether the specified value may be present in the dataset.
  • FIG. 14C provides a flowchart of a process 1460 of sorting a dataset for use in determining an intersection with another dataset.
  • a set of values is received that is at least partially unsorted, the set of values representing values in a dataset.
  • the set of values is sorted at 1466 to provide a sorted set of values sorted according to a sorting criterion.
  • the sorting includes, at 1468, for a second element in the set of unsorted values, determining that the second element is out of order compared with a first, preceding element in the set of unsorted values.
  • the current offset value is added to the second element at 1472.
  • An intersection between the sorted set and another set of values is determined at 1474.
  • the computing system 1500 includes one or more processing units 1510, 1515 and memory 1520, 1525.
  • the processing units 1510, 1515 execute computer-executable instructions, such as for implementing a database environment, and associated methods, described in Examples 1-14.
  • a processing unit can be a general-purpose central processing unit (CPU), a processor in an application-specific integrated circuit (ASIC), or any other type of processor.
  • CPU central processing unit
  • ASIC application-specific integrated circuit
  • FIG. 15 shows a central processing unit 1510 as well as a graphics processing unit or co-processing unit 1515.
  • the tangible memory 1520, 1525 may be volatile memory (e.g., registers, cache, RAM), non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), or some combination of the two, accessible by the processing unit(s) 1510, 1515.
  • the memory 1520, 1525 stores software 1580 implementing one or more innovations described herein, in the form of computer-executable instructions suitable for execution by the processing unit(s) 1510, 1515.
  • a computing system 1500 may have additional features.
  • the computing system 1500 includes storage 1540, one or more input devices 1550, one or more output devices 1560, and one or more communication connections 1570.
  • An interconnection mechanism such as a bus, controller, or network interconnects the components of the computing system 1500.
  • operating system software provides an operating environment for other software executing in the computing system 1500, and coordinates activities of the components of the computing system 1500.
  • the tangible storage 1540 may be removable or non-removable, and includes magnetic disks, magnetic tapes or cassettes, CD-ROMs, DVDs, or any other medium which can be used to store information in a non-transitory way, and which can be accessed within the computing system 1500.
  • the storage 1540 stores instructions for the software 1580 implementing one or more innovations described herein.
  • the input device(s) 1550 may be a touch input device such as a keyboard, mouse, pen, or trackball, a voice input device, a scanning device, or another device that provides input to the computing system 1500.
  • the output device(s) 1560 may be a display, printer, speaker, CD-writer, or another device that provides output from the computing system 1500.
  • the communication connection(s) 1570 enable communication over a communication medium to another computing entity, such as another database server.
  • the communication medium conveys information such as computer-executable instructions, audio or video input or output, or other data in a modulated data signal.
  • a modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
  • communication media can use an electrical, optical, RF, or other carrier.
  • program modules or components include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types.
  • the functionality of the program modules may be combined or split between program modules as desired in various embodiments.
  • Computer-executable instructions for program modules may be executed within a local or distributed computing system.
  • system and “device” are used interchangeably herein. Unless the context clearly indicates otherwise, neither term implies any limitation on a type of computing system or computing device. In general, a computing system or computing device can be local or distributed, and can include any combination of special-purpose hardware and/or general-purpose hardware with software implementing the functionality described herein.
  • Example 16 Cloud Computing Environment
  • FIG. 16 depicts an example cloud computing environment 1600 in which the described technologies can be implemented.
  • the cloud computing environment 1600 comprises cloud computing services 1610.
  • the cloud computing services 1610 can comprise various types of cloud computing resources, such as computer servers, data storage repositories, networking resources, etc.
  • the cloud computing services 1610 can be centrally located (e.g., provided by a data center of a business or organization) or distributed (e.g., provided by various computing resources located at different locations, such as different data centers and/or located in different cities or countries).
  • the cloud computing services 1610 are utilized by various types of computing devices (e.g., client computing devices), such as computing devices 1620, 1622, and 1664.
  • any of the disclosed methods can be implemented as computer-executable instructions or a computer program product stored on one or more computer-readable storage media, such as tangible, non-transitory computer-readable storage media, and executed on a computing device (e.g., any available computing device, including smart phones or other mobile devices that include computing hardware).
  • Tangible computer-readable storage media are any available tangible media that can be accessed within a computing environment (e.g., one or more optical media discs such as DVD or CD, volatile memory components (such as DRAM or SRAM), or nonvolatile memory components (such as flash memory or hard drives)).
  • computer-readable storage media include memory 1520 and 1525, and storage 1540.
  • the term computer-readable storage media does not include signals and carrier waves.
  • the term computer-readable storage media does not include communication connections (e.g., 1570).
  • any of the computer-executable instructions for implementing the disclosed techniques, as well as any data created and used during implementation of the disclosed embodiments, can be stored on one or more computer-readable storage media.
  • the computer-executable instructions can be part of, for example, a dedicated software application or a software application that is accessed or downloaded via a web browser or other software application (such as a remote computing application).
  • Such software can be executed, for example, on a single local computer (e.g., any suitable commercially available computer) or in a network environment (e.g., via the Internet, a wide-area network, a local-area network, a client-server network (such as a cloud computing network), or other such network) using one or more network computers.
  • the disclosed technology is not limited to any specific computer language or program.
  • the disclosed technology can be implemented by software written in C++, Java, Perl, JavaScript, Python, Ruby, ABAP, Structured Query Language, Adobe Flash, or any other suitable programming language, or, in some examples, markup languages such as html or XML, or combinations of suitable programming languages and markup languages.
  • the disclosed technology is not limited to any particular computer or type of hardware. Certain details of suitable computers and hardware are well known and need not be set forth in detail in this disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Library & Information Science (AREA)
  • Probability & Statistics with Applications (AREA)
  • Software Systems (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
EP24203968.3A 2023-10-03 2024-10-01 Datensatzzusammenfassungsmetadaten zur bereitstellung verbesserter abfrageleistung Pending EP4535185A3 (de)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US18/376,369 US12265535B1 (en) 2023-10-03 2023-10-03 Dataset summary metadata providing improved query performance

Publications (2)

Publication Number Publication Date
EP4535185A2 true EP4535185A2 (de) 2025-04-09
EP4535185A3 EP4535185A3 (de) 2025-04-30

Family

ID=92966800

Family Applications (1)

Application Number Title Priority Date Filing Date
EP24203968.3A Pending EP4535185A3 (de) 2023-10-03 2024-10-01 Datensatzzusammenfassungsmetadaten zur bereitstellung verbesserter abfrageleistung

Country Status (2)

Country Link
US (1) US12265535B1 (de)
EP (1) EP4535185A3 (de)

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU5688199A (en) * 1998-08-20 2000-03-14 Raycer, Inc. System, apparatus and method for spatially sorting image data in a three-dimensional graphics pipeline
US7630986B1 (en) * 1999-10-27 2009-12-08 Pinpoint, Incorporated Secure data interchange
CN100428633C (zh) * 2001-06-29 2008-10-22 日本胜利株式会社 数字信号的调制方法及调制装置
US7765221B2 (en) * 2004-09-30 2010-07-27 Sap Ag Normalization of a multi-dimensional set object
US20080059492A1 (en) * 2006-08-31 2008-03-06 Tarin Stephen A Systems, methods, and storage structures for cached databases
US8374359B2 (en) * 2007-02-16 2013-02-12 Bose Corporation Kick gate
US8645309B2 (en) * 2009-11-30 2014-02-04 At&T Intellectual Property I. L.P. Processing data using sequential dependencies
US9727594B2 (en) * 2013-01-10 2017-08-08 Microsoft Technology Licensing, Llc Adaptive range filters for range and point queries
KR102006029B1 (ko) * 2013-07-24 2019-08-01 매그나칩 반도체 유한회사 방위각 계산장치 및 그 방법
US10108633B2 (en) * 2014-12-27 2018-10-23 Ascava, Inc. Using a distributed prime data sieve for efficient lossless reduction, search, and retrieval of data
US10467293B2 (en) * 2017-05-18 2019-11-05 Aetna Inc. Scalable distributed computing system for determining exact median and other quantiles in big data applications
US11500868B2 (en) 2021-01-29 2022-11-15 Oracle International Corporation Efficient identification of vertices and edges for graph indexes in an RDBMS
US11727008B1 (en) * 2022-06-30 2023-08-15 Vast Data Ltd. Database table with a minimum-maximum filter for multiple ranges
US12353291B2 (en) * 2022-12-20 2025-07-08 Rubrik, Inc. Obtaining full snapshots for subsets of objects over time

Also Published As

Publication number Publication date
EP4535185A3 (de) 2025-04-30
US20250110955A1 (en) 2025-04-03
US12265535B1 (en) 2025-04-01

Similar Documents

Publication Publication Date Title
Yang et al. Pase: Postgresql ultra-high-dimensional approximate nearest neighbor search extension
Zoumpatianos et al. ADS: the adaptive data series index
US9507825B2 (en) Techniques for partition pruning based on aggregated zone map information
US10380269B2 (en) Sideways information passing
US12047098B2 (en) Data compression techniques
CN107092627B (zh) 记录的列状存储表示
EP3014488B1 (de) Inkrementelle pflege von bereichspartitionierten statistiken zur abfrageoptimierung
US20160055191A1 (en) Executing constant time relational queries against structured and semi-structured data
US20160364421A1 (en) Database index for constructing large scale data level of details
US11520763B2 (en) Automated optimization for in-memory data structures of column store databases
EP3913497B1 (de) Datenabdrucktechniken zur verwendung mit datenabrufverfahren
Kvet Relational data index consolidation
US12265541B2 (en) Systems and methods for executing queries on tensor datasets
Sunuwar et al. Comparative analysis of relational and graph databases for data provenance: Performance, queries, and security considerations
CN120086273A (zh) 多模融合数据处理方法、装置、介质、电子设备和程序产品
CN115809268B (zh) 一种基于分片索引的自适应查询方法和装置
Kvet et al. Master index access as a data tuple and block locator
CN116975098A (zh) 查询计划构建方法、装置、电子设备和存储介质
EP4546154A1 (de) Relationale unterbaumanpassung für verbesserte abfrageleistung
US12265535B1 (en) Dataset summary metadata providing improved query performance
EP4535186A1 (de) Effiziente sortiertechniken zur erleichterung der erzeugung und verwendung von datensatzzusammenfassungsmetadaten
Costa et al. Partitioning and bucketing in hive-based big data warehouses
WO2017019883A1 (en) Locality-sensitive hashing for algebraic expressions
CN116401285B (zh) 一种关系型数据库中对orc文件的索引方法及装置
EP4660830A1 (de) Abfragesprachendarstellungen von einschränkungen, die zum verursachen von abfragefehlern verwendbar sind

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR

RIC1 Information provided on ipc code assigned before grant

Ipc: H04L 69/04 20220101ALI20250326BHEP

Ipc: H03M 7/30 20060101ALI20250326BHEP

Ipc: G06F 16/908 20190101ALI20250326BHEP

Ipc: G06F 16/2458 20190101ALI20250326BHEP

Ipc: G06F 16/22 20190101AFI20250326BHEP

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20250925