CN113468181B - Parallel Hash connection acceleration method and system based on FPGA - Google Patents
Parallel Hash connection acceleration method and system based on FPGA Download PDFInfo
- Publication number
- CN113468181B CN113468181B CN202110792999.8A CN202110792999A CN113468181B CN 113468181 B CN113468181 B CN 113468181B CN 202110792999 A CN202110792999 A CN 202110792999A CN 113468181 B CN113468181 B CN 113468181B
- Authority
- CN
- China
- Prior art keywords
- tuple
- data
- partition
- hash
- linked list
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/278—Data partitioning, e.g. horizontal or vertical partitioning
Abstract
The invention discloses a parallel hash connection acceleration method and a system based on FPGA, comprising the following steps: reading a plurality of tuple data from a first data table in each set period of a hash connection construction stage, calculating a partition number corresponding to each tuple data by using a pre-division function, guiding each tuple data to a corresponding partition, and reading one tuple data from each partition to construct a hash table and a linked list; reading a plurality of element group data from the second data table in each set period of the detection stage of the Hash connection, calculating the partition number corresponding to each element group data by using a pre-division function, guiding each element group data to the corresponding partition, and reading a Hash table and a linked list constructed in the element group data detection matching construction stage from each partition, wherein the construction stage and the detection stage of the Hash connection are both completed in the FPGA. The invention can ensure that the access of the tuple among the partitions to the hash table has no address conflict and realizes the capability of processing a plurality of tuples in parallel.
Description
Technical Field
The invention relates to the technical field of databases, in particular to a parallel hash connection acceleration method and system based on an FPGA.
Background
The join operation is a common operation in a database, the performance of which often affects the overall performance of a database system, and the common join operation includes three types, namely nested loop join, sort merge join, and hash join. The hash connection is considered to be an efficient connection operation implementation manner, but the conventional hash connection has only linear time complexity, but still occupies more CPU computing resources.
In order to solve the problem that the conventional hash connection occupies more CPU computing resources, an FPGA (Field Programmable Gate Array) is introduced into a database system at present, and the hash connection operation is implemented in an FPGA circuit manner, so as to achieve the purpose of acceleration. However, in the process of constructing and detecting the hash table, when a plurality of tuple data are processed at the same time, a plurality of read-write requests for accessing the hash table need to be processed, if the accesses need to access the same memory bank, address access conflict can be caused, and at this time, one tuple needs to be processed before another tuple can be processed, so that the parallelism of the hash connection operation implemented in the existing FPGA circuit mode is low, and only one tuple can be processed in one clock cycle, which results in low performance.
Disclosure of Invention
In order to solve part or all of the technical problems in the prior art, the invention provides a parallel hash connection acceleration method and system based on an FPGA.
The technical scheme of the invention is as follows:
in a first aspect, a parallel hash connection acceleration method based on an FPGA is provided, where the method includes:
reading a plurality of element group data from a first data table in each set period of a Hash connection construction stage, calculating a partition number corresponding to each element group data by using a preset pre-dividing function, guiding each element group data to a partition corresponding to the partition number, reading one element group data from each partition to construct a Hash table and a linked list until all element group data of the first data table are processed;
reading a plurality of tuple data from a second data table in each set period of a detection stage of Hash connection, calculating a partition number corresponding to each tuple data by using a preset pre-division function, guiding each tuple data to a partition corresponding to the partition number, reading one tuple data from each partition, detecting the Hash table and a linked list constructed in a matching construction stage, and if matching exists, performing connection operation and outputting a connection result until all tuple data of the second data table are processed;
the first data table and the second data table are two data tables to be subjected to Hash connection, a partition number corresponds to a partition, the partition number is not smaller than the element group data number processed simultaneously, the Hash table and the linked list are organized in a partitioned mode, the Hash table comprises a plurality of different Hash table partitions, the linked list comprises a plurality of different linked list partitions, different Hash table partitions and different linked list partitions are stored in different storage bodies arranged inside the FPGA, different storage bodies are provided with independent read-write ports, and the construction stage and the detection stage of the Hash connection are completed inside the FPGA.
In one possible implementation manner, the pre-partition function is implemented by using a hash function, and the hash function is used for mapping the tuple data to the partition number corresponding to the partition.
In a possible implementation manner, each partition is provided with a partition queue, and each tuple data is guided into the partition queue of the corresponding partition according to the partition number corresponding to the tuple data, and the partition queue is used for buffering the tuple data.
In one possible implementation, reading a tuple data from a partition to construct a hash table and a linked list includes:
reading tuple data from a partition queue, calculating a hash value of the currently read tuple data by using a hash function, determining a hash entry address of a hash table to be accessed by the currently read tuple data by using the hash value and a partition number, reading an item corresponding to the hash entry address, generating a new item according to the currently read tuple data, writing the new item into the position of the hash entry address to cover the item corresponding to the hash entry address, wherein each item of the hash table is a first node of a linked list and comprises a tuple and a next tuple pointer, the next tuple pointer indicates a position for pointing to the storage of a next tuple with the same hash value in the linked list, the tuple in the new item is the currently read tuple data, and the next tuple pointer in the new item is generated in a preset address allocation mode;
and writing the item corresponding to the read hash entry address into a linked list, wherein the address of the written linked list is an address pointed by a next tuple pointer in a new item, a tuple in the linked list corresponds to a tuple in the item corresponding to the hash entry address, the next tuple pointer in the linked list is the next tuple pointer in the item corresponding to the hash entry address, each item of the linked list is a middle node or a tail node of the linked list and comprises the tuple and the next tuple pointer, the next tuple pointer points to the position where the next tuple with the same hash value is stored, and if one node is the tail node, the data stored in the tuple position in the current node is invalid.
In a possible implementation manner, reading a hash table and a linked list constructed in a tuple data detection matching construction stage from a partition, and if there is a match, performing a connection operation and outputting a connection result, including:
step S210, reading a tuple data from a partition queue, calculating a hash value of the currently read tuple data by using a hash function, determining a hash entry address of a hash table to be accessed by the currently read tuple data by using the hash value and a partition number, reading an item corresponding to the hash entry address, matching and verifying the item corresponding to the hash entry address and the currently read tuple data, if a next tuple pointer in the item corresponding to the hash entry address is 'END', ending the matching and verifying of the currently read tuple data, if the next tuple pointer in the item corresponding to the hash entry address is not 'END', verifying whether a tuple in the item corresponding to the hash entry address matches the currently read tuple data, if so, performing a connection operation on the tuple in the item corresponding to the hash entry address and the currently read tuple data, outputting a connection result and executing step S220, and if not, executing step S220;
step S220, according to the partition indicated by the next tuple pointer in the item corresponding to the hash entry address, guiding the currently read tuple data and the next tuple pointer in the item corresponding to the hash entry address to the detection queue of the corresponding linked list partition for caching, reading a group of data from the corresponding detection queue, accessing the item corresponding to the next tuple pointer in the linked list partition according to the next tuple pointer in the read data, performing matching verification on the item corresponding to the next tuple pointer in the linked list partition and the currently read tuple data, if the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is 'END', ending the matching verification on the currently read tuple data, if the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is not 'END', verifying whether the tuple in the item corresponding to the next tuple pointer in the linked list partition is matched with the currently read tuple data, if the tuple pointer in the item corresponding to the next tuple pointer in the linked list partition and the currently read tuple pointer in the linked list partition are not matched, and performing an independent detection operation on each detection queue, wherein the step S230 is performed;
step S230, according to a partition indicated by a next tuple pointer in data read from a detection queue, guiding currently read tuple data and the next tuple pointer in the data read from the detection queue to the detection queue of a corresponding linked list partition for caching, reading a group of data from the corresponding detection queue again, accessing an item corresponding to the next tuple pointer in the linked list partition according to the next tuple pointer in the read data, performing matching verification on the item corresponding to the next tuple pointer in the linked list partition and the currently read tuple data, if the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is 'END', ending the matching verification on the currently read tuple data, if the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is not 'END', verifying whether a tuple in the item corresponding to the next tuple pointer in the linked list partition is matched with the currently read tuple data, if matching, connecting a tuple in the item corresponding to the next tuple pointer in the linked list partition and the currently read tuple pointer in the linked list partition with the tuple pointer in the linked list partition and the tuple pointer in the linked list partition are not matched, and outputting a result, and step S240, if the next tuple pointer in the linked list is not matched;
step S240, repeatedly executing step S230 until the matching verification of the currently read tuple data is finished;
wherein, 'END' is a mnemonic for specifying that a corresponding address in the linked list is not used for storing data, if a next tuple pointer in an entry of the hash table is 'END', it indicates that the linked list with a corresponding node as a head node is empty, and if the next tuple pointer in the entry of the linked list is 'END', it indicates that the corresponding node is a tail node of a current linked list, and the next tuple pointer in each entry of the hash table is initialized to 'END'.
In a second aspect, a parallel hash join acceleration system based on an FPGA is provided, which includes:
the reading module is used for reading a plurality of tuple data from the first data table in each set period of the construction stage of the Hash connection, and reading a plurality of tuple data from the second data table in each set period of the detection stage of the Hash connection, and sending the read tuple data to the pre-division calculating module;
the pre-division calculating module is used for calculating the partition number corresponding to each received metafile data by using a preset pre-division function and sending each metafile data and the corresponding partition number to the pre-division module;
the pre-partitioning module is used for guiding each received tuple data to a partition corresponding to the partition number;
the hash table construction module is used for reading one tuple data from each partition in each set period of the hash connection construction stage to construct a hash table and a linked list until all tuple data of the first data table are processed;
the hash table detection module is used for reading one tuple data from each partition in each set period of the hash connection construction stage, detecting the hash table and the linked list constructed in the matching construction stage, performing connection operation according to the existence of matching and outputting a connection result until all tuple data of the second data table are processed;
the storage body is used for storing the hash table and the linked list in a partition mode, wherein the hash table comprises a plurality of hash table partitions, and the linked list comprises a plurality of linked list partitions;
the system completes construction in an FPGA, and the reading module, the pre-division calculating module, the pre-division module, the hash table constructing module, the hash table detecting module and the memory bank are all realized by utilizing an FPGA internal circuit.
In some possible implementations, each partition is provided with a partition queue, and the partition queue is used for buffering tuple data.
In some possible implementations, the hash table construction module includes:
the hash table updating submodule is used for reading element group data from a partition queue in each set period, calculating the hash value of the read element group data by using a hash function, determining the hash entry address of the hash table to be accessed by the read element group data by using the hash value and a partition number, reading an item corresponding to the hash entry address, generating a new item according to the read element group data, writing the new item into the position of the hash entry address, and sending the item corresponding to the hash entry address and a written linked list address to the network building module, wherein the written linked list address is the address pointed by a next element group pointer in the new item, each partition is provided with a hash table updating submodule, and the hash table updating submodule reads the element group data from the corresponding partition queue;
the linked list address calculation submodule is used for generating a next tuple pointer in a new item in a preset address distribution mode;
the network building submodule is used for sending an item corresponding to the hash entry address and the written linked list address to a linked list updating submodule corresponding to the partition according to the partition indicated by the written linked list address, wherein each partition is provided with the linked list updating submodule;
and the linked list updating submodule is used for writing the item corresponding to the hash entry address into the corresponding linked list partition and writing the position pointed by the linked list address.
In some possible implementations, the hash table probing module includes:
the hash table detection submodule is used for reading one tuple data from the partition queue in each set period, calculating the hash value of the currently read tuple data by using a hash function, determining the hash entry address of the hash table to be accessed by the currently read tuple data by using the hash value and the partition number, reading the item corresponding to the hash entry address, and sending the item corresponding to the hash entry address and the currently read tuple data to the verification submodule, wherein each partition is provided with the hash table detection submodule, and the hash table detection submodule reads the tuple data from the corresponding partition queue;
the verification submodule is used for performing matching verification on the item corresponding to the received hash entry address and the currently read tuple data, or the item corresponding to the next tuple pointer in the received linked list partition and the currently read tuple data, if the next tuple pointer in the item corresponding to the hash entry address or the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is 'END', ending the matching verification of the currently read tuple data, if the next tuple pointer in the item corresponding to the hash entry address or the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is not 'END', verifying whether the tuple in the item corresponding to the hash entry address and the currently read tuple data, or the tuple in the item corresponding to the next tuple pointer in the linked list partition is matched with the currently read tuple data, and if the tuple in the item corresponding to the hash entry address and the currently read tuple data, or the tuple in the item corresponding to the next tuple pointer in the linked list partition and the currently read tuple data in the linked list partition are connected to the sub-module; and the number of the first and second groups,
the next tuple pointer in the item corresponding to the currently read tuple data and the hash entry address or the next tuple pointer in the currently read tuple data and the data read from the detection queue is sent to the detection network submodule;
the detection network submodule is used for sending the received tuple data and the next tuple pointer to the detection queues of the corresponding linked list partitions for caching according to the partition number in the received next tuple pointer, wherein each linked list partition is provided with an independent detection queue;
the linked list detection submodule is used for reading a group of data from the corresponding detection queue in each set period, accessing an item corresponding to a next unary group pointer in a linked list partition according to the next unary group pointer in the read data, and sending the item corresponding to the next unary group pointer in the linked list partition to the verification submodule, wherein each partition is provided with the linked list detection submodule;
and the connection submodule is used for performing connection operation on the received tuple and outputting a connection result.
In some possible implementation manners, when the hash table probing submodule and the linked list probing submodule send data to the verification submodule at the same time, the verification submodule selects the data of the linked list probing submodule as input.
The technical scheme of the invention has the following main advantages:
the parallel hash connection acceleration method and system based on the FPGA pre-divide tuple data in the hash connection construction stage and the hash connection detection stage, and guide the tuple data to the corresponding partitions, so that address conflict does not exist in the access of tuples among the partitions to a hash table; moreover, the hash table is constructed by using the tuple data of the plurality of different partitions in the construction stage, and the hash table is simultaneously detected by using the tuple data of the plurality of different partitions in the detection stage, so that the parallel processing of the plurality of tuples can be realized.
Drawings
The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
fig. 1 is a flowchart of an FPGA-based parallel hash join acceleration method according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating an organization of a hash table according to an embodiment of the invention;
fig. 3 is a schematic diagram illustrating a principle of a construction stage of a parallel hash connection acceleration method based on an FPGA according to an embodiment of the present invention;
fig. 4 is a schematic diagram illustrating a detection phase principle of the parallel hash connection acceleration method based on the FPGA according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the specific embodiments of the present invention and the accompanying drawings. It is to be understood that the described embodiments are only some of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present invention. It will be understood by those skilled in the art that the present invention may be practiced without some of these specific details. In some instances, methods, procedures, components, and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present invention.
The technical solution provided by an embodiment of the present invention is described in detail below with reference to the accompanying drawings.
In a first aspect, as shown in fig. 1, an embodiment of the present invention provides a parallel hash join acceleration method based on an FPGA, including:
step S100, reading a plurality of metadata from a first data table in each set period of a hash connection construction stage, calculating a partition number corresponding to each metadata by using a preset pre-partitioning function, guiding each metadata to a partition corresponding to the partition number, reading one metadata from each partition to construct a hash table and a linked list until all metadata of the first data table are processed;
step S200, in each set period of a detection stage of Hash connection, reading a plurality of metadata from a second data table, calculating a partition number corresponding to each metadata by using a preset pre-division function, guiding each metadata to a partition corresponding to the partition number, reading one Hash table and a linked list constructed in a metadata detection matching construction stage from each partition, and if matching exists, performing connection operation and outputting a connection result until all metadata of the second data table is processed;
the first data table and the second data table are two data tables to be subjected to Hash connection, a partition number corresponds to a partition, the partition number is not smaller than the element group data number processed simultaneously, the Hash table and the linked list are organized in a partitioned mode, the Hash table comprises a plurality of different Hash table partitions, the linked list comprises a plurality of different linked list partitions, different Hash table partitions and different linked list partitions are stored in different storage bodies arranged inside the FPGA, different storage bodies are provided with independent read-write ports, and the construction stage and the detection stage of the Hash connection are completed inside the FPGA.
The parallel hash connection acceleration method based on the FPGA provided by the embodiment of the invention pre-divides tuple data in a hash connection construction stage and a hash connection detection stage, and guides the tuple data to corresponding partitions, so that address conflict does not exist in the access of tuples among the partitions to a hash table; moreover, the hash table is constructed by using the tuple data of the plurality of different partitions in the construction stage, and the hash table is simultaneously detected by using the tuple data of the plurality of different partitions in the detection stage, so that the parallel processing of the plurality of tuples can be realized.
The parallel hash connection acceleration method based on FPGA according to an embodiment of the present invention is further described below.
Fig. 2 is a schematic diagram illustrating an organization form of a hash table according to an embodiment of the present invention. As shown in fig. 2, in an embodiment of the present invention, the hash table is organized in a form of partitions, and includes a plurality of different hash table partitions, where the different hash table partitions are stored in different storage banks; the linked list is organized in a partition mode and comprises a plurality of different linked list partitions, and the different linked list partitions are stored in different storage bodies arranged in the FPGA; meanwhile, in order to ensure that read-write among different memory banks has no conflict, each memory bank has an independent read-write port; in order to ensure that the requirement of simultaneously processing a plurality of tuple data can be met, the number of the hash table partitions and the number of the linked list partitions are not less than the number of the tuple data required to be simultaneously processed, and the invention is specifically described below with the number of the tuple data required to be simultaneously processed being N.
Each item of the hash table is the first node of a linked list and comprises a tuple and a next tuple pointer; the next tuple pointer indicates the position of the next tuple storage with the same hash value in the pointing linked list, and if the next tuple pointer is 'END', the linked list with the node as the head node is empty; each item of the linked list is a middle node or a tail node of the linked list and comprises a tuple and a next tuple pointer; and if the next tuple pointer is 'END', the node is a tail node of the current linked list, otherwise, the node is a middle node of the current linked list, and if one node is a tail node, the data stored in the tuple position in the node is invalid. The 'END' is a mnemonic, and the specific implementation can use the combination of the appointed line number value and the appointed partition number value, for example, the maximum line number and the maximum partition number are used as the 'END', and the address in the linked list is not used for storing data.
The next tuple pointer in each item of the hash table and the next tuple pointer in each item of the linked list have the same format and can be formed by a pair consisting of a linked list row number and a linked list partition number, and the next tuple pointer in each item of the hash table is initialized to 'END'.
Fig. 3 is a schematic diagram illustrating a principle of a construction phase of the parallel hash join acceleration method based on FPGA according to an embodiment of the present invention. As shown in fig. 3, in step S100, in each setting period of the hash connection building stage, N tuple data in the first data table are read from the input end, a partition number corresponding to each tuple data is calculated by using a preset pre-partition function, each tuple data is guided to a corresponding partition according to the partition number corresponding to the tuple data, and one tuple data is read from each partition to build the hash table and the linked list until all tuple data of the first data table is processed.
And N is more than 1, each partition is provided with a partition queue, each tuple data is routed to the partition queue of the corresponding partition according to the partition number corresponding to the tuple data, and the partition queue caches the tuple data.
Because multiple tuple data read in the same set period may have the same partition number, and at this time, multiple tuple data with the same partition number need to be buffered in the same partition queue, the partition queue can be set as a multiple-input-one-output queue; specifically, when N sets of metadata are read every set period, the partition queue may be set as an N-input-one-output queue, that is, the partition queue can write N data at the input end at most and read one data at most to the output end at most every set period.
The pre-partition function may be implemented by using a hash function, and the hash function maps the tuple data to the partition number of the partition.
Further, in step S100, reading a tuple data from the partition to construct a hash table and a linked list, which may specifically include:
step S110, reading a tuple data from a partition queue, calculating a hash value of the currently read tuple data by using a hash function, determining a hash entry address of a hash table to be accessed by the currently read tuple data by using the hash value and a partition number, reading an item corresponding to the hash entry address, generating a new item according to the currently read tuple data, and writing the new item into the position of the hash entry address to cover the item corresponding to the hash entry address;
step S120, writing the read entry corresponding to the hash entry address into a linked list, where the written linked list address is an address pointed by a next tuple pointer in a new entry, a tuple in the linked list corresponds to a tuple in an entry corresponding to the hash entry address, and the next tuple in the linked list refers to the next tuple pointer in an entry corresponding to the hash entry address.
In an embodiment of the present invention, based on the set next tuple pointer format, the address allocation manner may be a manner of allocating in sequence according to a row number and a partition number, and the manner of allocating in sequence according to a row number and a partition number includes: when address allocation is carried out, the partition numbers are increased firstly, when the partition numbers reach the maximum value, the partition numbers return to the minimum value, and the line numbers are increased; for example, the number of partitions is 8, the partition numbers are 0, 1,2, 3, 4, 5, 6, and 7, respectively, and when the next tuple pointer is allocated to 8 tuples, if the currently allocated address starts from (k, 6), the addresses should be allocated in the order of (k, 6), (k, 7), (k +1,0), (k +1,1), (k +1,2), (k +1,3), (k +1,4), and (k +1,5); the N linked list updates generated by address allocation according to the address allocation mode respectively fall into N linked list partitions of the linked list, and different linked list partitions are stored in different memory banks, so that address conflict does not exist in the N linked list update operations.
Further, fig. 4 shows a schematic diagram of a detection phase principle of the parallel hash connection acceleration method based on FPGA according to an embodiment of the present invention. As shown in fig. 4, in step S200, in each setting period of the hash connection detection stage, N tuple data in the second data table are read from the input end, a partition number corresponding to each tuple data is calculated by using a preset pre-partition function, each tuple data is guided to a partition corresponding to the partition number, one tuple data detection matching hash table and linked list constructed in the construction stage are read from each partition, and if matching exists, a connection operation is performed and a connection result is output until all tuple data in the second data table are processed.
The pre-division function used in the detection stage is the same as the pre-division function used in the construction stage, each tuple data is routed to the partition queue of the corresponding partition according to the partition number corresponding to the tuple data, and the partition queue caches the tuple data.
Further, in step S200, a hash table and a linked list constructed in the metadata detection matching construction stage are read from the partition, and if there is a match, a connection operation is performed and a connection result is output, which may specifically include:
step S210, reading a tuple data from the partition queue, calculating a hash value of the currently read tuple data by using a hash function, determining a hash entry address of a hash table to be accessed by the currently read tuple data by using the hash value and the partition number, reading out an entry corresponding to the hash entry address, matching and verifying the entry corresponding to the hash entry address with the currently read tuple data, if a next tuple pointer in the entry corresponding to the hash entry address is 'END', ending the matching and verifying of the currently read tuple data, if the next tuple pointer in the entry corresponding to the hash entry address is not 'END', verifying whether a tuple in the entry corresponding to the hash entry address matches with the currently read tuple data, if so, performing a join operation between a tuple in the entry corresponding to the hash entry address and the currently read tuple data, outputting a join result and performing step S220, and if not, performing step S220;
step S220, according to the partition number in the next tuple pointer in the entry corresponding to the hash entry address, directing the currently read tuple data and the next tuple pointer in the entry corresponding to the hash entry address to the probe queue of the corresponding linked list partition for caching, reading a set of data from the corresponding probe queue, accessing the entry corresponding to the next tuple pointer in the linked list partition according to the next tuple pointer in the read data, performing matching validation on the entry corresponding to the next tuple pointer in the linked list partition and the currently read tuple data, if the next tuple pointer in the entry corresponding to the next tuple pointer in the linked list partition is 'END', ending the matching validation of the currently read tuple data, if the next tuple pointer in the entry corresponding to the next tuple pointer in the linked list partition is not 'END', verifying whether the tuple in the entry corresponding to the next tuple pointer in the linked list partition matches the currently read tuple data, if the next tuple pointer in the entry corresponding to the next tuple pointer in the linked list partition does not match the currently read tuple pointer, verifying whether the tuple pointer in the entry corresponding to the next tuple pointer in the linked list partition matches the currently read tuple pointer in the linked list partition and the current probe queue, and performing the step S230, wherein each step S is performed for connecting operation;
step S230, according to the partition number in the next tuple pointer in the data read from the probe queue, directing the currently read tuple data and the next tuple pointer in the data read from the probe queue to the probe queue of the corresponding linked list partition for caching, reading a set of data again from the corresponding probe queue, accessing the entry corresponding to the next tuple pointer in the linked list partition according to the next tuple pointer in the read set of data, performing matching validation on the entry corresponding to the next tuple pointer in the linked list partition and the currently read tuple data, if the next tuple pointer in the entry corresponding to the next tuple pointer in the linked list partition is 'END', ending the matching validation of the currently read tuple data, if the next tuple pointer in the entry corresponding to the next tuple pointer in the linked list partition is not 'END', verifying whether the tuple in the entry corresponding to the next tuple pointer in the linked list partition matches the currently read tuple data, if the next tuple pointer in the entry corresponding to the next tuple pointer in the linked list partition does not match the currently read tuple pointer, performing S240 operation, and performing S operation on the read tuple data;
step S240, repeatedly executing step S230 until the matching verification of the currently read tuple data is finished.
In a second aspect, an embodiment of the present invention provides an FPGA-based parallel hash join acceleration system, where the system includes:
the reading module is used for reading a plurality of metadata from the first data table in each set period of the construction stage of the Hash connection, and is used for reading a plurality of metadata from the second data table in each set period of the detection stage of the Hash connection, and sending the read metadata to the pre-division calculating module;
the pre-division calculation module is used for calculating the partition number corresponding to each received tuple data by using a preset pre-division function and sending each tuple data and the corresponding partition number to the pre-division module;
the pre-dividing module is used for guiding each received tuple data to a partition corresponding to the partition number;
the hash table construction module is used for reading one tuple data from each partition in each set period of the hash connection construction stage to construct a hash table and a linked list until all tuple data of the first data table are processed;
the hash table detection module is used for reading a hash table and a linked list constructed in the metadata detection matching construction stage from each partition in each set period of the detection stage of hash connection, performing connection operation according to the existence of matching and outputting a connection result until all metadata of the second data table are processed;
the storage body is used for storing the hash table and the linked list in a partition mode, wherein the hash table comprises a plurality of hash table partitions, and the linked list comprises a plurality of linked list partitions;
the system is built inside the FPGA, and the reading module, the pre-division calculating module, the pre-division module, the hash table building module, the hash table detecting module and the storage body are all realized by using an internal circuit of the FPGA.
Further, in an embodiment of the present invention, a plurality of different memory banks are arranged inside the FPGA, and the plurality of different memory banks are used for storing a plurality of different hash table partitions and different linked list partitions.
Optionally, the memory banks may employ storage resources internal to the FPGA, such as BRAM, for storing the different partitions.
Further, in an embodiment of the present invention, each partition is provided with a partition queue, each tuple data is routed to the partition queue of the corresponding partition according to the partition number corresponding to the tuple data, and the partition queue buffers the tuple data.
Further, in an embodiment of the present invention, the hash table constructing module may include:
the hash table updating submodule is used for reading element group data from a partition queue in each set period, calculating the hash value of the read element group data by using a hash function, determining the hash entry address of the hash table to be accessed by the read element group data by using the hash value and a partition number, reading an item corresponding to the hash entry address, generating a new item according to the read element group data, writing the new item into the position of the hash entry address, and sending the item corresponding to the hash entry address and a written linked list address to the network building module, wherein the written linked list address is the address pointed by a next element group pointer in the new item, each partition is provided with a hash table updating submodule, and the hash table updating submodule reads the element group data from the corresponding partition queue;
the linked list address calculation submodule is used for generating a next tuple pointer in a new item in a preset address distribution mode;
a network submodule is constructed and used for sending items corresponding to the Hash entry address and the written linked list address to linked list updating submodules corresponding to partitions according to partition numbers in the written linked list address, wherein each partition is provided with a linked list updating submodule;
and the linked list updating submodule is used for writing the item corresponding to the hash entry address into the position pointed by the written linked list address in the corresponding linked list partition.
In an embodiment of the present invention, the linked list address calculation submodule may provide an address written into the linked list for each hash table update submodule that needs to update the linked list, and the linked list address calculation submodule may maintain a last empty node address register for recording a base address of a currently allocated linked list address, where the base address may be initialized to (0,0), i.e., the 0 th partition of the 0 th row of the linked list; the address distribution mode of the linked list address calculation submodule can adopt a mode of distributing according to the sequence of the row number and the partition number, and the mode of distributing according to the sequence of the row number and the partition number comprises the following steps: when address allocation is carried out, the partition number is increased, when the partition number reaches the maximum value, the partition number returns to the minimum value, and the line number is increased; after the address allocation is completed, the base address in the last empty node address register is updated to the next empty address, for example, the number of the linked list partitions is 8, the partition numbers are 0, 1,2, 3, 4, 5, 6, and 7, the base address in the last empty node address register is (k, 6), and when there are 8 hash table update submodules that need to allocate the next tuple pointer to the tuple, the base addresses are allocated in the order of (k, 6), (k, 7), (k +1,0), (k +1,1), (k +1,2), (k +1,3), (k +1,4), (k +1,5), and at this time, the base address in the last empty node address register is updated to the next empty address (k +1,6).
The linked list address calculation submodule can distribute the linked list updating addresses to the linked list updating submodules in sequence, so that the addresses are mapped to different linked list partitions, namely the N linked list updating submodules access the N different linked list partitions of the linked list, and the problem of address conflict is avoided.
Further, in an embodiment of the present invention, the hash table detecting module may include:
the hash table detection submodule is used for reading one tuple data from the partition queue in each set period, calculating the hash value of the currently read tuple data by using a hash function, determining the hash entry address of the hash table to be accessed by the currently read tuple data by using the hash value and the partition number, reading an item corresponding to the hash entry address, and sending the item corresponding to the hash entry address and the currently read tuple data to the verification submodule, wherein each partition is provided with the hash table detection submodule, and the hash table detection submodule reads the tuple data from the corresponding partition queue;
a verification submodule, configured to perform matching verification on an item corresponding to the received hash entry address and currently read tuple data, or a received item corresponding to a next tuple pointer in a linked list partition and currently read tuple data, if the next tuple pointer in the item corresponding to the hash entry address, or the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is 'END', END matching verification on the currently read tuple data, if the next tuple pointer in the item corresponding to the hash entry address, or the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is not 'END', verify whether a tuple in the item corresponding to the hash entry address and currently read tuple data, or a tuple in the item corresponding to the next tuple pointer in the linked list partition matches with the currently read tuple data, if matching, send a tuple in the item corresponding to the hash entry address and currently read tuple data, or a tuple in the item corresponding to the linked list partition and currently read tuple in the currently read tuple data to the verification submodule; and the number of the first and second groups,
the next tuple pointer in the item corresponding to the currently read tuple data and the hash entry address or the next tuple pointer in the currently read tuple data and the data read from the detection queue is sent to the detection network submodule;
the detection network submodule is used for sending the received tuple data and the next tuple pointer to the detection queues of the corresponding linked list partitions for caching according to the partition number in the received next tuple pointer, wherein each linked list partition is provided with an independent detection queue;
the linked list detection submodule is used for reading a group of data from the corresponding detection queue in each set period, accessing an item corresponding to a next unary group pointer in a linked list partition according to the next unary group pointer in the read data, and sending the item corresponding to the next unary group pointer in the linked list partition to the verification submodule, wherein each partition is provided with the linked list detection submodule;
and the connection submodule is used for performing connection operation on the received tuple and outputting a connection result.
In an embodiment of the present invention, when the hash table probing submodule and the linked list probing submodule simultaneously send data to the verification submodule, the verification submodule selects the data of the linked list probing submodule as input.
Specifically, when the system is implemented in a multi-stage pipeline manner, the hash table detection submodule and the linked list detection submodule may send data to the verification submodule at the same time, and in order to ensure that the system is not deadlocked, the verification submodule preferentially selects the data of the linked list detection submodule, that is, when the hash table detection submodule and the linked list detection submodule send data to the verification submodule at the same time, the data of the linked list detection submodule is selected as input, and the data of the hash table detection submodule is stopped from being transmitted to the verification submodule.
It is noted that, in this document, relational terms such as "first" and "second," and the like, if any, are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. In addition, "front", "rear", "left", "right", "upper" and "lower" (if present) in the present document are referred to the placement states shown in the drawings.
Finally, it should be noted that: the above examples are only for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A parallel hash connection acceleration method based on FPGA is characterized by comprising the following steps:
reading a plurality of element group data from a first data table in each set period of a Hash connection construction stage, calculating a partition number corresponding to each element group data by using a preset pre-dividing function, guiding each element group data to a partition corresponding to the partition number, reading one element group data from each partition to construct a Hash table and a linked list until all element group data of the first data table are processed;
reading a plurality of element group data from a second data table in each set period of a detection stage of Hash connection, calculating a partition number corresponding to each element group data by using a preset pre-division function, guiding each element group data to a partition corresponding to the partition number, reading one element group data from each partition, detecting the Hash table and the linked list constructed in the matching construction stage, if matching exists, performing connection operation and outputting a connection result until all the element group data of the second data table are processed;
wherein, first data table and second data table are for waiting to carry out two data tables of hash connection, a partition number corresponds a subregion, the partition number is not less than the tuple data number of simultaneous processing, hash table and link table adopt the form of subregion to organize, hash table includes the hash table subregion of a plurality of differences, the link table includes the link table subregion of a plurality of differences, different hash table subregions and different link table subregion storage are in the inside different memory banks that set up of FPGA, different memory banks have independent read and write port, hash connection's the construction stage and the detection stage all accomplish inside FPGA.
2. The FPGA-based parallel hash join acceleration method of claim 1, wherein the pre-partition function is implemented using a hash function, and the hash function is used to map the tuple data to a partition number corresponding to the partition.
3. The FPGA-based parallel hash join acceleration method of claim 1 or 2, characterized in that each partition is provided with a partition queue, each tuple data is guided into the partition queue of the corresponding partition according to the partition number corresponding to the tuple data, and the partition queue is used for caching the tuple data.
4. The parallel hash join acceleration method based on FPGA of claim 3, wherein reading a tuple data from a partition to construct a hash table and a linked list comprises:
reading tuple data from a partition queue, calculating a hash value of the currently read tuple data by using a hash function, determining a hash entry address of a hash table to be accessed by the currently read tuple data by using the hash value and a partition number, reading an item corresponding to the hash entry address, generating a new item according to the currently read tuple data, writing the new item into the position of the hash entry address to cover the item corresponding to the hash entry address, wherein each item of the hash table is a first node of a linked list and comprises a tuple and a next tuple pointer, the next tuple pointer indicates a position for pointing to the storage of a next tuple with the same hash value in the linked list, the tuple in the new item is the currently read tuple data, and the next tuple pointer in the new item is generated in a preset address allocation mode;
and writing the item corresponding to the read hash entry address into a linked list, wherein the address of the written linked list is an address pointed by a next tuple pointer in a new item, a tuple in the linked list corresponds to a tuple in the item corresponding to the hash entry address, the next tuple pointer in the linked list is the next tuple pointer in the item corresponding to the hash entry address, each item of the linked list is a middle node or a tail node of the linked list and comprises the tuple and the next tuple pointer, the next tuple pointer points to the position where the next tuple with the same hash value is stored, and if one node is the tail node, the data stored in the tuple position in the current node is invalid.
5. The parallel hash connection acceleration method based on the FPGA of claim 3 or 4, wherein the hash table and the linked list constructed in the matching construction stage are read from the partition, and if there is a match, the connection operation is performed and a connection result is output, including:
step S210, reading a tuple data from a partition queue, calculating a hash value of the currently read tuple data by using a hash function, determining a hash entry address of a hash table to be accessed by the currently read tuple data by using the hash value and a partition number, reading an item corresponding to the hash entry address, matching and verifying the item corresponding to the hash entry address and the currently read tuple data, if a next tuple pointer in the item corresponding to the hash entry address is 'END', ending the matching and verifying of the currently read tuple data, if the next tuple pointer in the item corresponding to the hash entry address is not 'END', verifying whether a tuple in the item corresponding to the hash entry address matches the currently read tuple data, if so, performing a connection operation on the tuple in the item corresponding to the hash entry address and the currently read tuple data, outputting a connection result and executing step S220, and if not, executing step S220;
step S220, according to the partition indicated by the next tuple pointer in the item corresponding to the hash entry address, guiding the currently read tuple data and the next tuple pointer in the item corresponding to the hash entry address to the detection queue of the corresponding linked list partition for caching, reading a group of data from the corresponding detection queue, accessing the item corresponding to the next tuple pointer in the linked list partition according to the next tuple pointer in the read data, performing matching verification on the item corresponding to the next tuple pointer in the linked list partition and the currently read tuple data, if the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is 'END', ending the matching verification on the currently read tuple data, if the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is not 'END', verifying whether the tuple in the item corresponding to the next tuple pointer in the linked list partition is matched with the currently read tuple data, if the tuple pointer in the item corresponding to the next tuple pointer in the linked list partition and the currently read tuple pointer in the linked list partition are not matched, and performing an independent detection operation on each detection queue, wherein the step S230 is performed;
step S230, according to the partition indicated by the next tuple pointer in the data read from the detection queue, guiding the currently read tuple data and the next tuple pointer in the data read from the detection queue to the detection queue of the corresponding linked list partition for caching, reading a group of data again from the corresponding detection queue, accessing the item corresponding to the next tuple pointer in the linked list partition according to the next tuple pointer in the read data, performing matching verification on the item corresponding to the next tuple pointer in the linked list partition and the currently read tuple data, if the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is 'END', ending the matching verification of the currently read tuple data, if the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is not 'END', verifying whether the tuple in the item corresponding to the next tuple pointer in the linked list partition is matched with the currently read tuple data, if the matching is not, connecting the tuple in the item corresponding to the next tuple pointer in the linked list partition and the currently read tuple pointer in the linked list partition with the connected result, and performing a step S240;
step S240, repeatedly executing step S230 until the matching verification of the currently read tuple data is finished;
wherein, 'END' is a mnemonic for specifying that a corresponding address in the linked list is not used for storing data, if a next tuple pointer in an entry of the hash table is 'END', it indicates that the linked list with a corresponding node as a head node is empty, and if the next tuple pointer in the entry of the linked list is 'END', it indicates that the corresponding node is a tail node of a current linked list, and the next tuple pointer in each entry of the hash table is initialized to 'END'.
6. A parallel hash connection acceleration system based on FPGA is characterized by comprising:
the reading module is used for reading a plurality of metadata from the first data table in each set period of the construction stage of the Hash connection, and is used for reading a plurality of metadata from the second data table in each set period of the detection stage of the Hash connection, and sending the read metadata to the pre-division calculating module;
the pre-division calculation module is used for calculating the partition number corresponding to each received metafile data by using a preset pre-division function and sending each metafile data and the corresponding partition number to the pre-division module;
the pre-partitioning module is used for guiding each received tuple data to a partition corresponding to the partition number;
the hash table construction module is used for reading one tuple data from each partition in each set period of the hash connection construction stage to construct a hash table and a linked list until all tuple data of the first data table are processed;
the hash table detection module is used for reading one tuple data from each partition in each set period of the hash connection construction stage, detecting the hash table and the linked list constructed in the matching construction stage, performing connection operation according to the existence of matching and outputting a connection result until all tuple data of the second data table are processed;
the storage body is used for storing the hash table and the linked list in a partition mode, wherein the hash table comprises a plurality of hash table partitions, and the linked list comprises a plurality of linked list partitions;
the system completes construction in an FPGA, and the reading module, the pre-division calculating module, the pre-division module, the hash table constructing module, the hash table detecting module and the memory bank are all realized by utilizing an FPGA internal circuit.
7. The FPGA-based parallel hash join acceleration system of claim 6, characterized in that each partition is provided with a partition queue for buffering tuple data.
8. The FPGA-based parallel hash join acceleration system of claim 7, wherein the hash table building module comprises:
the hash table updating submodule is used for reading element group data from a partition queue in each set period, calculating the hash value of the read element group data by using a hash function, determining the hash entry address of the hash table to be accessed by the read element group data by using the hash value and a partition number, reading an item corresponding to the hash entry address, generating a new item according to the read element group data, writing the new item into the position of the hash entry address, and sending the item corresponding to the hash entry address and a written linked list address to the network building module, wherein the written linked list address is the address pointed by a next element group pointer in the new item, each partition is provided with a hash table updating submodule, and the hash table updating submodule reads the element group data from the corresponding partition queue;
the linked list address calculation submodule is used for generating a next tuple pointer in a new item in a preset address distribution mode;
the network building submodule is used for sending an item corresponding to the hash entry address and the written linked list address to a linked list updating submodule corresponding to the partition according to the partition indicated by the written linked list address, wherein each partition is provided with the linked list updating submodule;
and the linked list updating submodule is used for writing the item corresponding to the hash entry address into the corresponding linked list partition and writing the position pointed by the linked list address.
9. The FPGA-based parallel hash join acceleration system of claim 8, wherein the hash table probing module comprises:
the hash table detection submodule is used for reading one tuple data from the partition queue in each set period, calculating the hash value of the currently read tuple data by using a hash function, determining the hash entry address of the hash table to be accessed by the currently read tuple data by using the hash value and the partition number, reading the item corresponding to the hash entry address, and sending the item corresponding to the hash entry address and the currently read tuple data to the verification submodule, wherein each partition is provided with the hash table detection submodule, and the hash table detection submodule reads the tuple data from the corresponding partition queue;
the verification submodule is used for performing matching verification on the item corresponding to the received hash entry address and the currently read tuple data, or the item corresponding to the next tuple pointer in the received linked list partition and the currently read tuple data, if the next tuple pointer in the item corresponding to the hash entry address or the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is 'END', ending the matching verification of the currently read tuple data, if the next tuple pointer in the item corresponding to the hash entry address or the next tuple pointer in the item corresponding to the next tuple pointer in the linked list partition is not 'END', verifying whether the tuple in the item corresponding to the hash entry address and the currently read tuple data, or the tuple in the item corresponding to the next tuple pointer in the linked list partition is matched with the currently read tuple data, and if the tuple in the item corresponding to the hash entry address and the currently read tuple data, or the tuple in the item corresponding to the next tuple pointer in the linked list partition and the currently read tuple data in the linked list partition are connected to the sub-module; and the number of the first and second groups,
the detection network submodule is used for sending the next tuple pointer in the items corresponding to the currently read tuple data and the hash entry address or the next tuple pointer in the currently read tuple data and the data read from the detection queue to the detection network submodule;
the detection network submodule is used for sending the received tuple data and the next tuple pointer to the detection queues of the corresponding linked list partitions for caching according to the partition number in the received next tuple pointer, wherein each linked list partition is provided with an independent detection queue;
the linked list detection submodule is used for reading a group of data from the corresponding detection queue in each set period, accessing an item corresponding to a next unary group pointer in a linked list partition according to the next unary group pointer in the read data, and sending the item corresponding to the next unary group pointer in the linked list partition to the verification submodule, wherein each partition is provided with the linked list detection submodule;
and the connection submodule is used for performing connection operation on the received tuple and outputting a connection result.
10. The FPGA-based parallel hash join acceleration system of claim 9, wherein the validation submodule selects the data of the linked list probing submodule as input when the hash table probing submodule and the linked list probing submodule send data to the validation submodule at the same time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110792999.8A CN113468181B (en) | 2021-07-14 | 2021-07-14 | Parallel Hash connection acceleration method and system based on FPGA |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110792999.8A CN113468181B (en) | 2021-07-14 | 2021-07-14 | Parallel Hash connection acceleration method and system based on FPGA |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113468181A CN113468181A (en) | 2021-10-01 |
| CN113468181B true CN113468181B (en) | 2022-10-11 |
Family
ID=77880213
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110792999.8A Active CN113468181B (en) | 2021-07-14 | 2021-07-14 | Parallel Hash connection acceleration method and system based on FPGA |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113468181B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011131470A1 (en) * | 2010-04-22 | 2011-10-27 | International Business Machines Corporation | Gpu enabled database systems |
| CN109977116A (en) * | 2019-03-14 | 2019-07-05 | 山东超越数控电子股份有限公司 | Hash connection operator accelerated method and system based on FPGA-DDR |
| CN110109898A (en) * | 2019-04-23 | 2019-08-09 | 山东超越数控电子股份有限公司 | Hash connection accelerated method and system based on BRAM in FPGA piece |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9734284B2 (en) * | 2013-03-15 | 2017-08-15 | Micron Technology, Inc. | Hardware acceleration of short read mapping for genomic and other types of analyses |
-
2021
- 2021-07-14 CN CN202110792999.8A patent/CN113468181B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011131470A1 (en) * | 2010-04-22 | 2011-10-27 | International Business Machines Corporation | Gpu enabled database systems |
| CN109977116A (en) * | 2019-03-14 | 2019-07-05 | 山东超越数控电子股份有限公司 | Hash connection operator accelerated method and system based on FPGA-DDR |
| CN110109898A (en) * | 2019-04-23 | 2019-08-09 | 山东超越数控电子股份有限公司 | Hash connection accelerated method and system based on BRAM in FPGA piece |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113468181A (en) | 2021-10-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2885728B1 (en) | Hardware implementation of the aggregation/group by operation: hash-table method | |
| US7805427B1 (en) | Integrated search engine devices that support multi-way search trees having multi-column nodes | |
| AU598857B2 (en) | Move-out queue buffer | |
| US7844422B2 (en) | Method and system for changing a description for a state transition function of a state machine engine | |
| CN101221538B (en) | System and method for implementing fast data search in caching | |
| US7603346B1 (en) | Integrated search engine devices having pipelined search and b-tree maintenance sub-engines therein | |
| US10152434B2 (en) | Efficient arbitration for memory accesses | |
| CN107273042A (en) | Deduplication DRAM system algorithm framework | |
| JP2002510079A (en) | Method and apparatus for forcing ordered execution of reads and writes between memory interfaces | |
| US8086641B1 (en) | Integrated search engine devices that utilize SPM-linked bit maps to reduce handle memory duplication and methods of operating same | |
| US20160294693A1 (en) | Routing Lookup Method and Device, and Construction Method for B-tree Structure | |
| CN114356223B (en) | Memory access method and device, chip and electronic equipment | |
| CN108139882B (en) | Implement the system and method for stratum's distribution lists of links for network equipment | |
| CN111694770B (en) | Method and device for processing IO (input/output) request | |
| CN114020790A (en) | Data query method and device | |
| US7987205B1 (en) | Integrated search engine devices having pipelined node maintenance sub-engines therein that support database flush operations | |
| CN105359142B (en) | Hash connecting method and device | |
| US7953721B1 (en) | Integrated search engine devices that support database key dumping and methods of operating same | |
| CN113468181B (en) | Parallel Hash connection acceleration method and system based on FPGA | |
| CN105389394A (en) | Data request processing method and device based on a plurality of database clusters | |
| CN112241325A (en) | Ultra-large-scale integrated circuit database based on memory pool and design method | |
| CN117015767A (en) | On-chip interconnect for memory channel controllers | |
| KR20110068578A (en) | Parallel range query process method on r-tree with graphics processing units | |
| CN113779154B (en) | Construction method and application of distributed learning index model | |
| US9583158B2 (en) | Method of managing requests for access to memories and data storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |