WO2023029254A1 - Data query method and apparatus, and device and medium - Google Patents

Data query method and apparatus, and device and medium Download PDF

Info

Publication number
WO2023029254A1
WO2023029254A1 PCT/CN2021/134413 CN2021134413W WO2023029254A1 WO 2023029254 A1 WO2023029254 A1 WO 2023029254A1 CN 2021134413 W CN2021134413 W CN 2021134413W WO 2023029254 A1 WO2023029254 A1 WO 2023029254A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
query
data block
block
ratio
Prior art date
Application number
PCT/CN2021/134413
Other languages
French (fr)
Chinese (zh)
Inventor
任智新
张闯
谢志勇
王敏
孙忠祥
Original Assignee
苏州浪潮智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州浪潮智能科技有限公司 filed Critical 苏州浪潮智能科技有限公司
Publication of WO2023029254A1 publication Critical patent/WO2023029254A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/24569Query processing with adaptation to specific hardware, e.g. adapted for using GPUs or SSDs
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2453Query optimisation

Definitions

  • the present application relates to the technical field of data processing, and in particular to a data query method, device, equipment and medium.
  • FPGA field-programmable gate array
  • the central idea is to offload computationally intensive tasks from the CPU to the FPGA for execution.
  • task allocation most of the tasks related to database query are directly offloaded to FPGA for execution, but for some query tasks, directly offloading tasks to FPGA will consume more time and affect user experience.
  • the purpose of the present application is to provide a data query method, device, device, and medium, which can improve the processing efficiency of data query tasks, thereby improving user experience.
  • the specific plan is as follows:
  • the application discloses a data query method, including:
  • the average value of the data ratio is obtained; wherein, the average value of the data ratio is the average value of the ratio of the amount of data that meets the query conditions in previous data query operations to the total data amount of the corresponding data block, Moreover, the data query operations are all query operations under the same data query task, the data block is a data block of a target database, and the target database is a database corresponding to the data query task;
  • the data query task corresponding to the test data set is processed by FPGA acceleration device and database management software;
  • the data ratio is the ratio of the amount of data meeting the query conditions to the total data amount of the corresponding data block in any data query operation under the data query task, and the first consumption time corresponding to the target ratio is It is equal to the second elapsed time.
  • the FPGA acceleration device after moving the current data block to the FPGA acceleration device for processing, or using the database management software to process the current data block, it also includes:
  • Median filtering is performed on the ratio of the amount of data meeting the query conditions to the total data amount of the corresponding data block in this and previous data query operations, and the filtering result is used as the data proportion threshold corresponding to the next data query operation.
  • the FPGA acceleration device after moving the current data block to the FPGA acceleration device for processing, or using the database management software to process the current data block, it also includes:
  • the type of data in the target database Based on the type of the target database, the type of data in the target database, the type of query data, the ratio of the amount of data meeting the query conditions in this and previous data query operations to the total data amount of the corresponding data block to determine the next time The data proportion threshold corresponding to the data query operation.
  • the FPGA acceleration device after moving the current data block to the FPGA acceleration device for processing, or using the database management software to process the current data block, it also includes:
  • the size of the data block corresponding to each data query operation is determined based on the data storage method and data volume of the target database, and the bandwidth and data processing capability of the FPGA acceleration device.
  • the current data block is directly moved to the FPGA acceleration device for processing.
  • a data query device including:
  • the data ratio mean value acquisition module is used to obtain the data ratio mean value when performing a data query operation for the data block; wherein, the data ratio mean value is the amount of data that meets the query conditions and the corresponding data block in previous data query operations
  • the average value of the ratio of the total data volume, and the data query operations are all query operations under the same data query task, the data block is a data block of the target database, and the target database is the corresponding data query task database;
  • a data proportion threshold judging module configured to determine whether the data proportion average value is less than the data proportion threshold
  • the data block processing module is used to move the current data block to the FPGA acceleration device for processing, otherwise use the database management software to process the current data block.
  • an electronic device comprising:
  • a processor configured to execute the computer program, so as to realize the aforementioned data query method.
  • the present application discloses a computer-readable storage medium for storing a computer program, and when the computer program is executed by a processor, the aforementioned data query method is implemented.
  • the average value of the data ratio is obtained; wherein, the average data ratio is the ratio of the amount of data that meets the query conditions in previous data query operations and the total data amount of the corresponding data block.
  • the average value of the ratio, and the data query operations are all query operations under the same data query task, the data block is a data block of the target database, and the target database is the database corresponding to the data query task, and then it is judged that the Whether the average value of the above data ratio is less than the data ratio threshold, if so, move the current data block to the FPGA acceleration device for processing, otherwise use the database management software to process the current data block.
  • the current data block when performing a data query operation for a data block under a data query task, the ratio of the amount of data that meets the query conditions in the previous data query operations under the data query task to the total data amount of the corresponding data block Compared with the data ratio threshold, the current data block will be moved to the FPGA acceleration device for processing when it is less than the threshold, otherwise, the current data block will be processed by the database management software, so that the data query is performed according to the amount of data that meets the query conditions Task, to determine whether to move the data block to the FPGA for processing, avoiding the problem of large data delay caused by moving the data block to the FPGA when the amount of data that meets the query conditions is large, and can improve the processing efficiency of the data query task, thereby Improve user experience.
  • Fig. 1 is a flow chart of a data query method disclosed in the present application
  • Fig. 2 is a kind of concrete realization block diagram based on FPGA database heterogeneous acceleration system that the application provides;
  • FIG. 3 is a flow chart of a specific data query method disclosed in the present application.
  • FIG. 4 is a flow chart of a specific data query method disclosed in the present application.
  • Fig. 5 is a schematic structural diagram of a data query device disclosed in the present application.
  • FIG. 6 is a structural diagram of an electronic device disclosed in the present application.
  • the central idea is to offload computationally intensive tasks from the CPU to the FPGA for execution.
  • task allocation most of the tasks related to database query are directly offloaded to FPGA for execution, but for some query tasks, directly offloading tasks to FPGA will consume more time and affect user experience.
  • the present application provides a data query solution, which can improve the processing efficiency of data query tasks, thereby improving user experience.
  • a data query method including:
  • Step S11 When performing a data query operation for a data block, obtain the average value of the data proportion; wherein, the average data proportion is the ratio of the amount of data that meets the query conditions in previous data query operations to the total data amount of the corresponding data block , and the data query operations are all query operations under the same data query task, the data block is a data block of a target database, and the target database is a database corresponding to the data query task.
  • the size of the data block corresponding to each data query operation is determined based on the data storage method and data volume of the target database, and the bandwidth and data processing capability of the FPGA acceleration device.
  • the memory of the FPGA acceleration device and the memory of the host are fixed and limited.
  • the data needs to be partitioned.
  • the data transmission bandwidth and memory capacity between devices are limited, and they are also moved to FPGA acceleration devices in batches and multiple times.
  • the size of each data block moved can be set according to the storage method and size of the data table of the database, as well as the bandwidth and data processing capability of the FPGA acceleration device.
  • the data query task includes multiple data query operations for data blocks.
  • Step S12 Judging whether the average data proportion is smaller than a data proportion threshold.
  • Step S13 If yes, move the current data block to the FPGA acceleration device for processing, otherwise use the database management software to process the current data block.
  • the embodiment of the present application calculates the amount of data that meets the query conditions and the corresponding data in this and previous data query operations.
  • the average value of the ratio of the total data volume of the block is used as the average value of the data proportion corresponding to the next data query operation.
  • the current data block is directly moved to the FPGA acceleration device for processing.
  • Figure 2 is a block diagram of a specific FPGA-based database heterogeneous acceleration system provided by the embodiment of the present application, a server or service terminal device running the database system and an FPGA accelerator card that realizes intensive task offloading
  • the devices were previously interconnected through the high-speed interface PCIe (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard).
  • the FPGA accelerator card includes multiple acceleration function units such as acceleration function unit 0 and acceleration function unit 1 .
  • the general processing flow of FPGA-based heterogeneous database acceleration tasks is as follows: (1) On the server side, when a user initiates a database operation request, the CPU reads the relevant original data block from the disk into the memory, parses and unloads the task , notify the FPGA accelerator card to start the task; (2) send the original data and the calculation task to the DDR ( (Double Data Rate, double rate) memory; (3) After receiving the notification, the FPGA fetches the data and caches the source data from the DDR to the internal RAM according to the assigned calculation tasks, then preprocesses the data, checks Filter the data that meets the query conditions; (4) send the qualified data to the acceleration calculation unit module, perform calculation processing on the data, and write the calculation results back to the internal RAM buffer; (5) the FPGA acceleration unit continues to write the output results Go back to the DDR on the accelerator card and notify the host to read the result, and the host reads the result back to the host-side memory through the PCIe channel in DMA mode. After completing the preceding steps,
  • the FPGA accelerator card cannot directly read data from the memory in the host system. It needs to move the original data of the database to the DDR in the accelerator card through DMA first, and then transfer the result from The acceleration card is moved back to the host memory. In this process, the delay of data transmission is a non-negligible factor. The more data that meets the query conditions, the larger the amount of data moved, the more data returned, and the greater the PCIe transmission delay. When the amount of data selected is greater than a certain value, the FPGA acceleration method will cause a greater delay instead of achieving the effect of performance acceleration. At this time, you should choose to use the original software query method to perform tasks, thereby improving the overall performance. performance and improve user experience.
  • this application can dynamically select whether to offload the query task to the FPGA according to the size of the query task.
  • the FPGA acceleration function is started, and the data query operation under the query task is offloaded to the FPGA for execution. Otherwise, the FPGA hardware acceleration function is invalid, and the data query operation under the query task is still in the software. implement.
  • a task selector is designed and implemented on the host side. When the user sends a query task, the CPU makes a judgment during preprocessing, and selects whether to start the FPGA accelerator according to the threshold and the amount of data to be selected. Referring to FIG.
  • the embodiment of the present application discloses a specific data query method flow chart.
  • the task selector is implemented on the host side. Before the original data is moved, threshold judgment is performed. If the accelerator startup condition is met, the query task is offloaded to the FPGA for execution. Otherwise, the original process is continued, that is, the query task is executed by the CPU.
  • the specific process includes: when the system executes the task for the first time, that is, when it executes the first data query operation under the current task, the FPGA accelerator is selected by default, and the task selector is opened; when a data block is processed, the calculation meets the query conditions
  • the data volume of the data and calculate the ratio of the data volume to the total data volume of the data block, denoted as ⁇ e, and use this value as the estimated value of whether to use hardware acceleration for the data query operation of the next data block, when ⁇ e is less than When ⁇ , the query operation of the next data block still uses hardware acceleration, otherwise, the original software method is used for query.
  • the value of ⁇ e uses the average value of the ratio of the data volume that satisfies the query condition in all data blocks that have been calculated to the total data volume of the corresponding data block. The amount of data is more objective and real.
  • the average value of the data proportion is obtained; wherein, the average value of the data proportion is the amount of data that meets the query conditions in previous data query operations and the total data of the corresponding data block
  • the average value of the ratio of the amount, and the data query operations are all query operations under the same data query task
  • the data block is a data block of the target database
  • the target database is the database corresponding to the data query task, and then It is judged whether the average value of the data ratio is less than the data ratio threshold, if so, the current data block is moved to the FPGA acceleration device for processing, otherwise the current data block is processed by using the database management software.
  • the current data block when performing a data query operation for a data block under a data query task, the ratio of the amount of data that meets the query conditions in the previous data query operations under the data query task to the total data amount of the corresponding data block Compared with the average value of the data ratio threshold, the current data block will be moved to the FPGA acceleration device for processing when it is less than the threshold value, otherwise, the current data block will be processed by the database management software.
  • the data query is performed according to the amount of data that meets the query conditions Task, to determine whether to move the data block to the FPGA for processing, avoiding the problem of large data delay caused by moving the data block to the FPGA when the amount of data that meets the query conditions is large, and can improve the processing efficiency of the data query task, thereby Improve user experience.
  • the embodiment of the present application discloses a specific data query method, including:
  • Step S21 The data query task corresponding to the test data set is processed by the FPGA acceleration device and the database management software respectively.
  • Step S22 Determine the first time consumption for processing by the FPGA acceleration device and the second time consumption for processing by the database management software.
  • Step S23 Determine the target proportion from all data proportions based on the first consumption time and the second consumption time, and use the target proportion as a data proportion threshold.
  • the data ratio is the ratio of the amount of data that meets the query conditions to the total data amount of the corresponding data block in any data query operation under the data query task, and the first consumption time corresponding to the target ratio is equal to the second consumption time .
  • the corresponding first consumption time is shorter than the second consumption time, and for a data proportion greater than the target proportion, the corresponding first consumption time is greater than the second consumption time.
  • the embodiment of the present application can determine the size of the threshold through experiments, that is, based on a certain test set, the query operation is performed in two ways of starting the accelerator and closing the accelerator respectively.
  • the time consumption of each query method is used to find the intersection point of performance, and the percentage of the amount of data that satisfies the query condition to the total size of the data block is used as the threshold ⁇ .
  • This value can be used as the threshold for whether the FPGA acceleration selector is started or not in the current verified database system.
  • Step S24 When performing a data query operation for a data block, obtain the average value of the data proportion; wherein, the average data proportion is the average value of the ratio of the amount of data that meets the query conditions in previous data query operations to the total data amount of the corresponding data block , and the data query operations are all query operations under the same data query task, the data block is the data block of the target database, and the target database is the database corresponding to the data query task.
  • Step S25 Determine whether the average data proportion is greater than the data proportion threshold.
  • Step S26 If yes, move the current data block to the FPGA acceleration device for processing, otherwise use the database management software to process the current data block.
  • the data that meets the query conditions in this and previous data query operations can be The ratio of the amount to the total data amount of the corresponding data block is subjected to median filtering, and the filtering result is used as the data proportion threshold corresponding to the next data query operation, or, based on the type of the target database, in the target database
  • the type of data, the type of query data, the ratio of the amount of data meeting the query conditions in this and previous data query operations to the total data amount of the corresponding data block determines the corresponding data proportion threshold for the next data query operation.
  • the data proportion threshold can also be manually determined.
  • a threshold determination method suitable for an actual system may be selected according to an actual application scenario, a data storage structure of a database, and query conditions.
  • the embodiment of the present application discloses a data query device, which is characterized in that it includes:
  • the data ratio mean value acquisition module 11 is used to obtain the data ratio mean value when performing a data query operation for a data block; wherein, the data ratio mean value is the amount of data and the corresponding data that meet the query conditions in previous data query operations.
  • the average value of the ratio of the total data volume of the block, and the data query operations are all query operations under the same data query task, the data block is a data block of the target database, and the target database corresponds to the data query task database.
  • the data proportion threshold judging module 12 is configured to judge whether the average data proportion is smaller than the data proportion threshold.
  • the data block processing module 13 is used to move the current data block to the FPGA acceleration device for processing, otherwise use the database management software to process the current data block.
  • the average value of the data proportion is obtained; wherein, the average value of the data proportion is the amount of data that meets the query conditions in previous data query operations and the total data of the corresponding data block
  • the average value of the ratio of the amount, and the data query operations are all query operations under the same data query task
  • the data block is a data block of the target database
  • the target database is the database corresponding to the data query task, and then It is judged whether the average value of the data ratio is less than the data ratio threshold, if so, the current data block is moved to the FPGA acceleration device for processing, otherwise the current data block is processed by using the database management software.
  • the current data block when performing a data query operation for a data block under a data query task, the ratio of the amount of data that meets the query conditions in the previous data query operations under the data query task to the total data amount of the corresponding data block Compared with the data ratio threshold, the current data block will be moved to the FPGA acceleration device for processing when it is less than the threshold, otherwise, the current data block will be processed by the database management software, so that the data query is performed according to the amount of data that meets the query conditions Task, to determine whether to move the data block to the FPGA for processing, avoiding the problem of large data delay caused by moving the data block to the FPGA when the amount of data that meets the query conditions is large, and can improve the processing efficiency of the data query task, thereby Improve user experience.
  • the device also includes a threshold determination module, specifically for processing the data query tasks corresponding to the test data set using the FPGA acceleration device and the database management software respectively; determining the first consumption time for processing by the FPGA acceleration device, and The second consumption time for processing by using database management software; determining a target proportion from all data proportions based on the first consumption time and the second consumption time, and using the target proportion as the data proportion than the threshold;
  • the data ratio is the ratio of the amount of data meeting the query conditions to the total data amount of the corresponding data block in any data query operation under the data query task, and the first consumption time corresponding to the target ratio It is equal to the second elapsed time.
  • the threshold determination module is specifically used to process the current and previous data blocks after the data block processing module 13 moves the current data block to the FPGA acceleration device for processing, or uses the database management software to process the current data block.
  • the ratio of the amount of data meeting the query conditions to the total data amount of the corresponding data block is subjected to median filtering, and the filtering result is used as the data proportion threshold corresponding to the next data query operation.
  • the threshold determination module is specifically used to move the current data block to the FPGA acceleration device for processing by the data block processing module 13, or after the current data block is processed by the database management software, it also includes:
  • the type of data in the target database Based on the type of the target database, the type of data in the target database, the type of query data, the ratio of the amount of data meeting the query conditions in this and previous data query operations to the total data amount of the corresponding data block to determine the next time The data proportion threshold corresponding to the data query operation.
  • the embodiment of the present application also includes a data ratio mean determination module, which is specifically used to calculate the current data block after the data block processing module 13 moves the current data block to the FPGA acceleration device for processing, or uses the database management software to process the current data block.
  • the average value of the ratio of the amount of data that meets the query conditions to the total data amount of the corresponding data block in this and previous data query operations is used as the average value of the data ratio corresponding to the next data query operation.
  • the device also includes: a data block size determination module, configured to determine the corresponding data block size of each data query operation based on the data storage method and data volume of the target database, and the bandwidth and data processing capability of the FPGA acceleration device. The size of the data block.
  • the data block processing module 13 is further configured to directly move the current data block to the FPGA acceleration device for processing when the data query operation for the data block under the data query task is executed for the first time.
  • the embodiment of the present application discloses an electronic device 20, including a processor 21 and a memory 22; wherein, the memory 22 is used to store computer programs; the processor 21 is used to execute the A computer program, wherein, when the computer program is executed by a processor, the data query method disclosed in the foregoing embodiments is implemented.
  • the memory 22, as a resource storage carrier may be a read-only memory, random access memory, magnetic disk or optical disk, etc., and the storage method may be temporary storage or permanent storage.
  • the electronic device 20 also includes a power supply 23, a communication interface 24, an input and output interface 25, and a communication bus 26; wherein the power supply 23 is used to provide operating voltages for each hardware device on the electronic device 20; the The communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, which is not specifically limited here;
  • the input and output interface 25 is used to obtain external input data or output data to the external, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
  • the embodiment of the present application also discloses a computer-readable storage medium for storing a computer program, wherein, when the computer program is executed by a processor, the data query method disclosed in the foregoing embodiments is implemented.
  • each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same or similar parts of each embodiment can be referred to each other.
  • the description is relatively simple, and for the related information, please refer to the description of the method part.
  • RAM random access memory
  • ROM read-only memory
  • EEPROM electrically programmable ROM
  • EEPROM electrically erasable programmable ROM
  • registers hard disk, removable disk, CD-ROM, or any other Any other known storage medium.

Abstract

Disclosed in the present application are a data query method and apparatus, and a device and a medium. The method comprises: when a data query operation for a data block is executed, acquiring a data proportion mean value, wherein the data proportion mean value is a mean value of ratios of the amounts of data, which meets a query condition, to the total data amounts of corresponding data blocks in all previous data query operations, the data query operations are all query operations under the same data query task, the data blocks are data blocks of a target database, and the target database is a database corresponding to the data query task; determining whether the data proportion mean value is less than a data proportion threshold value; and if so, moving the current data block to an FPGA acceleration device for processing, otherwise, processing the current data block by using database management software. In this way, the efficiency of data query task processing can be improved, thereby improving the user experience.

Description

一种数据查询方法、装置、设备及介质A data query method, device, equipment and medium
本申请要求在2021年9月3日提交中国专利局、申请号为202111029248.7、发明名称为“一种数据查询方法、装置、设备及介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application with the application number 202111029248.7 and the title of the invention "a data query method, device, equipment and medium" submitted to the China Patent Office on September 3, 2021, the entire contents of which are incorporated by reference in this application.
技术领域technical field
本申请涉及数据处理技术领域,特别涉及一种数据查询方法、装置、设备及介质。The present application relates to the technical field of data processing, and in particular to a data query method, device, equipment and medium.
背景技术Background technique
随着智能社会、云时代、物联网万物互连等数据密集型应用的出现,数据库中的数据量和用户访问量急剧上升,亟需提升数据库的处理速度。为了缓解日益增长的数据量给CPU(即Central Processing Unit,中央处理器)带来的计算压力,很多研究都集中在硬件加速上,硬件加速已经被证明是一个有效的解决方法,是提高数据库管理系统中数据密集型应用程序性能的解决方案。与高性能GPU(即Graphics Processing Unit,图形处理器)相比,FPGA更适用于出现计算瓶颈的数据库应用,因为它具有相当大的计算能力和更高的能效比,这对于数据中心来说至关重要。另外,由于FPGA本身的高并行性,对于相同的数据库查询任务,它比CPU和ARM(即Advanced RISC Machine,微处理器)等通用处理器消耗的计算时间更少,且消耗的功耗更低,因此业界出现了一系列基于FPAG的异构加速解决方案。With the emergence of data-intensive applications such as the intelligent society, the cloud era, and the Internet of Things, the amount of data in the database and the number of user visits have risen sharply, and there is an urgent need to increase the processing speed of the database. In order to alleviate the computing pressure brought by the increasing amount of data to the CPU (Central Processing Unit, central processing unit), many studies have focused on hardware acceleration. Hardware acceleration has been proven to be an effective solution, which is to improve database management. The solution for data-intensive application performance in the system. Compared with high-performance GPU (Graphics Processing Unit, graphics processing unit), FPGA is more suitable for database applications with computing bottlenecks, because it has considerable computing power and higher energy efficiency ratio, which is very important for data centers. important. In addition, due to the high parallelism of FPGA itself, for the same database query task, it consumes less computing time and consumes less power than general-purpose processors such as CPU and ARM (Advanced RISC Machine, microprocessor). , so a series of FPAG-based heterogeneous acceleration solutions have appeared in the industry.
基于FPGA的异构加速方案中,中心思想将计算密集的任务由CPU执行卸载到FPGA中执行。而对于任务的分配上,大都是直接将数据库查询相关的任务全部卸载到FPGA中执行,但有些查询任务,将任务直接卸载到FPGA反而会消耗更多的时间,影响用户体验。In the FPGA-based heterogeneous acceleration solution, the central idea is to offload computationally intensive tasks from the CPU to the FPGA for execution. As for task allocation, most of the tasks related to database query are directly offloaded to FPGA for execution, but for some query tasks, directly offloading tasks to FPGA will consume more time and affect user experience.
发明内容Contents of the invention
有鉴于此,本申请的目的在于提供一种数据查询方法、装置、设备及介质,能够提升数据查询任务的处理效率,从而提升用户体验。其具体方案如下:In view of this, the purpose of the present application is to provide a data query method, device, device, and medium, which can improve the processing efficiency of data query tasks, thereby improving user experience. The specific plan is as follows:
第一方面,本申请公开了一种数据查询方法,包括:In the first aspect, the application discloses a data query method, including:
当执行针对数据块的数据查询操作时,获取数据占比均值;其中,所述数据占比均值为历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,并且,所述数据查询操作均为同一数据查询任务下的查询操作,所述数据块为目标数据库的数据块,所述目标数据库为所述数据查询任务对应的数据库;When performing a data query operation for a data block, the average value of the data ratio is obtained; wherein, the average value of the data ratio is the average value of the ratio of the amount of data that meets the query conditions in previous data query operations to the total data amount of the corresponding data block, Moreover, the data query operations are all query operations under the same data query task, the data block is a data block of a target database, and the target database is a database corresponding to the data query task;
判断所述数据占比均值是否小于数据占比阈值;Judging whether the average value of the data proportion is less than the data proportion threshold;
若是,则将当前数据块搬移至FPGA加速设备进行处理,否则利用数据库管理软件对当前数据块进行处理。If yes, move the current data block to the FPGA acceleration device for processing, otherwise use the database management software to process the current data block.
可选的,还包括:Optionally, also include:
将测试数据集对应的数据查询任务分别利用FPGA加速设备和数据库管理软件进行处理;The data query task corresponding to the test data set is processed by FPGA acceleration device and database management software;
确定利用FPGA加速设备进行处理的第一消耗时间,以及利用数据库管理软件进行处理的第二消耗时间;Determining the first consuming time for processing by the FPGA acceleration device, and the second consuming time for processing by using the database management software;
基于所述第一消耗时间和所述第二消耗时间从全部数据占比中确定出目标占比,并将所述目标占比作为所述数据占比阈值;determining a target proportion from all data proportions based on the first consumption time and the second consumption time, and using the target proportion as the data proportion threshold;
其中,所述数据占比为数据查询任务下任一次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值,并且,所述目标占比对应的所述第一消耗时间与所述第二消耗时间相等。Wherein, the data ratio is the ratio of the amount of data meeting the query conditions to the total data amount of the corresponding data block in any data query operation under the data query task, and the first consumption time corresponding to the target ratio is It is equal to the second elapsed time.
可选的,将当前数据块搬移至FPGA加速设备进行处理,或利用数据库管理软件对当前数据块进行处理之后,还包括:Optionally, after moving the current data block to the FPGA acceleration device for processing, or using the database management software to process the current data block, it also includes:
对本次以及历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值进行中值滤波,并将滤波结果作为下一次数据查询操作对应的所述数据占比阈值。Median filtering is performed on the ratio of the amount of data meeting the query conditions to the total data amount of the corresponding data block in this and previous data query operations, and the filtering result is used as the data proportion threshold corresponding to the next data query operation.
可选的,将当前数据块搬移至FPGA加速设备进行处理,或利用数据库管理软件对当前数据块进行处理之后,还包括:Optionally, after moving the current data block to the FPGA acceleration device for processing, or using the database management software to process the current data block, it also includes:
基于所述目标数据库的类型、所述目标数据库中数据的类型、查询数据的类型、本次以及历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值确定出下一次数据查询操作对应的所述数据占比阈值。Based on the type of the target database, the type of data in the target database, the type of query data, the ratio of the amount of data meeting the query conditions in this and previous data query operations to the total data amount of the corresponding data block to determine the next time The data proportion threshold corresponding to the data query operation.
可选的,将当前数据块搬移至FPGA加速设备进行处理,或利用数据库管理软件对当前数据块进行处理之后,还包括:Optionally, after moving the current data block to the FPGA acceleration device for processing, or using the database management software to process the current data block, it also includes:
计算本次以及历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,将该均值作为下一次数据查询操作对应的数据占比均值。Calculate the average value of the ratio of the amount of data that meets the query conditions to the total data amount of the corresponding data block in this and previous data query operations, and use the average value as the average value of the data proportion corresponding to the next data query operation.
可选的,还包括:Optionally, also include:
基于所述目标数据库的数据存储方式和数据量、以及所述FPGA加速设备的带宽和数据处理能力确定每次数据查询操作对应的所述数据块的大小。The size of the data block corresponding to each data query operation is determined based on the data storage method and data volume of the target database, and the bandwidth and data processing capability of the FPGA acceleration device.
可选的,还包括:Optionally, also include:
当首次执行所述数据查询任务下的针对数据块的数据查询操作时,则直接将当前数据块搬移至FPGA加速设备进行处理。When performing the data query operation for the data block under the data query task for the first time, the current data block is directly moved to the FPGA acceleration device for processing.
第二方面,本申请公开了一种数据查询装置,包括:In a second aspect, the present application discloses a data query device, including:
数据占比均值获取模块,用于当执行针对数据块的数据查询操作时,获取数据占比均值;其中,所述数据占比均值为历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,并且,所述数据查询操作均为同一数据查询任务下的查询操作,所述数据块为目标数据库的数据块,所述目标数据库为所述数据查询任务对应的数据库;The data ratio mean value acquisition module is used to obtain the data ratio mean value when performing a data query operation for the data block; wherein, the data ratio mean value is the amount of data that meets the query conditions and the corresponding data block in previous data query operations The average value of the ratio of the total data volume, and the data query operations are all query operations under the same data query task, the data block is a data block of the target database, and the target database is the corresponding data query task database;
数据占比阈值判断模块,用于判断所述数据占比均值是否小于数据占比阈值;A data proportion threshold judging module, configured to determine whether the data proportion average value is less than the data proportion threshold;
数据块处理模块,用于将当前数据块搬移至FPGA加速设备进行处理,否则利用数据库管理软件对当前数据块进行处理。The data block processing module is used to move the current data block to the FPGA acceleration device for processing, otherwise use the database management software to process the current data block.
第三方面,本申请公开了一种电子设备,包括:In a third aspect, the present application discloses an electronic device, comprising:
存储器,用于保存计算机程序;memory for storing computer programs;
处理器,用于执行所述计算机程序,以实现前述的数据查询方法。A processor, configured to execute the computer program, so as to realize the aforementioned data query method.
第四方面,本申请公开了一种计算机可读存储介质,用于保存计算机程序,所述计算机程序被处理器执行时实现前述的数据查询方法。In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, and when the computer program is executed by a processor, the aforementioned data query method is implemented.
可见,本申请当执行针对数据块的数据查询操作时,获取数据占比均值;其中,所述数据占比均值为历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,并且,所述数据查询操作均为同一数据查询任务下的查询操作,所述数据块为目标数据库的数据块,所述目标数据库为所述数据查询任务对应的数据库,然后判断所述数据占比均值是否小于数据占比阈值,若是,则将当前数据块搬移至FPGA加速设备进行处理,否则利用数据库管理软件对当前数据块进行处理。也即,本申请中,当执行数据查询任务下针对数据块的数据查询操作时,将该数据查询任务下的历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值与数据占比阈值比较,小于阈值时才会将当前数据块搬移至FPGA加速设备进行处理,否则利用数据库管理软件对当前数据块进行处理,这样,根据符合查询条件的数据量执行数据查询任务,判断是否将数据块搬移至FPGA中处理,避免了符合查询条件的数据量较大时仍搬移至FPGA中处理,造成数据延时较大的问题,能够提升数据查询任务的处理效率,从而提升用户体验。It can be seen that when the application performs a data query operation for a data block, the average value of the data ratio is obtained; wherein, the average data ratio is the ratio of the amount of data that meets the query conditions in previous data query operations and the total data amount of the corresponding data block. The average value of the ratio, and the data query operations are all query operations under the same data query task, the data block is a data block of the target database, and the target database is the database corresponding to the data query task, and then it is judged that the Whether the average value of the above data ratio is less than the data ratio threshold, if so, move the current data block to the FPGA acceleration device for processing, otherwise use the database management software to process the current data block. That is, in this application, when performing a data query operation for a data block under a data query task, the ratio of the amount of data that meets the query conditions in the previous data query operations under the data query task to the total data amount of the corresponding data block Compared with the data ratio threshold, the current data block will be moved to the FPGA acceleration device for processing when it is less than the threshold, otherwise, the current data block will be processed by the database management software, so that the data query is performed according to the amount of data that meets the query conditions Task, to determine whether to move the data block to the FPGA for processing, avoiding the problem of large data delay caused by moving the data block to the FPGA when the amount of data that meets the query conditions is large, and can improve the processing efficiency of the data query task, thereby Improve user experience.
附图说明Description of drawings
为了更清楚地说明本申请实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据提供的附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present application, and those skilled in the art can also obtain other drawings according to the provided drawings without creative work.
图1为本申请公开的一种数据查询方法流程图;Fig. 1 is a flow chart of a data query method disclosed in the present application;
图2为本申请提供的一种具体的基于FPGA数据库异构加速系统实现框图;Fig. 2 is a kind of concrete realization block diagram based on FPGA database heterogeneous acceleration system that the application provides;
图3为本申请公开的一种具体的数据查询方法流程图;FIG. 3 is a flow chart of a specific data query method disclosed in the present application;
图4为本申请公开的一种具体的数据查询方法流程图;FIG. 4 is a flow chart of a specific data query method disclosed in the present application;
图5为本申请公开的一种数据查询装置结构示意图;Fig. 5 is a schematic structural diagram of a data query device disclosed in the present application;
图6为本申请公开的一种电子设备结构图。FIG. 6 is a structural diagram of an electronic device disclosed in the present application.
具体实施方式Detailed ways
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.
基于FPGA的异构加速方案中,中心思想将计算密集的任务由CPU执行卸载到FPGA中执行。而对于任务的分配上,大都是直接将数据库查询相关的任务全部卸载到FPGA中执行,但有些查询任务,将任务直接卸载到FPGA反而会消耗更多的时间,影响用户体验。为此,本申请提供了一种数据查询方案,能够提升数据查询任务的处理效率,从而提升用户体验。In the FPGA-based heterogeneous acceleration solution, the central idea is to offload computationally intensive tasks from the CPU to the FPGA for execution. As for task allocation, most of the tasks related to database query are directly offloaded to FPGA for execution, but for some query tasks, directly offloading tasks to FPGA will consume more time and affect user experience. For this reason, the present application provides a data query solution, which can improve the processing efficiency of data query tasks, thereby improving user experience.
参见图1所示,本申请实施例公开了一种数据查询方法,包括:Referring to Fig. 1, the embodiment of the present application discloses a data query method, including:
步骤S11:当执行针对数据块的数据查询操作时,获取数据占比均值;其中,所述数据占比均值为历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,并且,所述数据查询操作均为同一数据查询任务下的查询操作,所述数据块为目标数据库的数据块,所述目标数据库为所述数据查询任务对应的数据库。Step S11: When performing a data query operation for a data block, obtain the average value of the data proportion; wherein, the average data proportion is the ratio of the amount of data that meets the query conditions in previous data query operations to the total data amount of the corresponding data block , and the data query operations are all query operations under the same data query task, the data block is a data block of a target database, and the target database is a database corresponding to the data query task.
在具体的实施方式中,基于所述目标数据库的数据存储方式和数据量、以及所述FPGA加速设备的带宽和数据处理能力确定每次数据查询操作对应的所述数据块的大小。In a specific implementation manner, the size of the data block corresponding to each data query operation is determined based on the data storage method and data volume of the target database, and the bandwidth and data processing capability of the FPGA acceleration device.
需要指出的是,在系统中,FPGA加速设备的内存以及主机内存的容量是固定有限的,当进行大数据处理时,需要对数据进行分区,即便是卸载到FPGA中执行的任务,受两个设备之间数据传输带宽以及内存容量的限制,也是分批多次搬移到FPGA加速设备中。在实际应用场景下,可以根据数据库的数据表的存储方式和表的大小,以及FPGA加速设备的带宽和数据处理能力,设置每次搬移数据块的大小。It should be pointed out that in the system, the memory of the FPGA acceleration device and the memory of the host are fixed and limited. When processing large data, the data needs to be partitioned. The data transmission bandwidth and memory capacity between devices are limited, and they are also moved to FPGA acceleration devices in batches and multiple times. In the actual application scenario, the size of each data block moved can be set according to the storage method and size of the data table of the database, as well as the bandwidth and data processing capability of the FPGA acceleration device.
具体的,当获取到用户发送的数据查询请求,生成相应的数据查询任务,该数据查询任务下包括多次针对数据块的数据查询操作。Specifically, when the data query request sent by the user is obtained, a corresponding data query task is generated, and the data query task includes multiple data query operations for data blocks.
步骤S12:判断所述数据占比均值是否小于数据占比阈值。Step S12: Judging whether the average data proportion is smaller than a data proportion threshold.
步骤S13:若是,则将当前数据块搬移至FPGA加速设备进行处理,否则利用数据库管理软件对当前数据块进行处理。Step S13: If yes, move the current data block to the FPGA acceleration device for processing, otherwise use the database management software to process the current data block.
并且,在将当前数据块搬移至FPGA加速设备进行处理,或利用数据库管 理软件对当前数据块进行处理之后,本申请实施例计算本次以及历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,将该均值作为下一次数据查询操作对应的数据占比均值。And, after moving the current data block to the FPGA acceleration device for processing, or using the database management software to process the current data block, the embodiment of the present application calculates the amount of data that meets the query conditions and the corresponding data in this and previous data query operations. The average value of the ratio of the total data volume of the block is used as the average value of the data proportion corresponding to the next data query operation.
并且,本申请实施例中,当首次执行所述数据查询任务下的针对数据块的数据查询操作时,则直接将当前数据块搬移至FPGA加速设备进行处理。Moreover, in the embodiment of the present application, when the data query operation for the data block under the data query task is executed for the first time, the current data block is directly moved to the FPGA acceleration device for processing.
例如,参见图2所示,图2为本申请实施例提供的一种具体的基于FPGA数据库异构加速系统实现框图,运行数据库系统的服务器或者服务终端设备与实现密集型任务卸载的FPGA加速卡设备之前通过高速接口PCIe(即Peripheral Component Interconnect Express,一种高速串行计算机扩展总线标准)互联。FPGA加速卡包括加速功能单元0、加速功能单元1等多个加速功能单元。当前,基于FPGA的数据库异构加速任务一般处理流程为:(1)在服务器端,当用户发起数据库操作请求时,CPU将相关原始数据块从磁盘读取到内存中,将任务进行解析并卸载,通知FPGA加速卡启动任务;(2)将原始数据以及计算任务以二者之间协商的协议,以DMA(即Direct Memory Access,直接存储器访问)方式通过PCIe发给FPGA加速卡中的DDR(即Double Data Rate,双倍速率)内存中;(3)FPGA接收到通知后,取数并根据分配的计算任务,将源数据从DDR中缓存到内部RAM中,接着对数据进行预处理,检查筛选符合查询条件的数据;(4)符合条件的数据送入加速计算单元模块,对数据进行计算处理,将计算结果写回到内部RAM缓冲区;(5)FPGA加速单元将输出的结果继续写回到加速卡上的DDR并通知主机读取结果,主机以DMA方式通过PCIe通道读回结果到主机端内存中。完成前述步骤后,CPU将计算结果数据传递给查询请求操作。For example, as shown in Figure 2, Figure 2 is a block diagram of a specific FPGA-based database heterogeneous acceleration system provided by the embodiment of the present application, a server or service terminal device running the database system and an FPGA accelerator card that realizes intensive task offloading The devices were previously interconnected through the high-speed interface PCIe (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard). The FPGA accelerator card includes multiple acceleration function units such as acceleration function unit 0 and acceleration function unit 1 . At present, the general processing flow of FPGA-based heterogeneous database acceleration tasks is as follows: (1) On the server side, when a user initiates a database operation request, the CPU reads the relevant original data block from the disk into the memory, parses and unloads the task , notify the FPGA accelerator card to start the task; (2) send the original data and the calculation task to the DDR ( (Double Data Rate, double rate) memory; (3) After receiving the notification, the FPGA fetches the data and caches the source data from the DDR to the internal RAM according to the assigned calculation tasks, then preprocesses the data, checks Filter the data that meets the query conditions; (4) send the qualified data to the acceleration calculation unit module, perform calculation processing on the data, and write the calculation results back to the internal RAM buffer; (5) the FPGA acceleration unit continues to write the output results Go back to the DDR on the accelerator card and notify the host to read the result, and the host reads the result back to the host-side memory through the PCIe channel in DMA mode. After completing the preceding steps, the CPU passes the calculation result data to the query request operation.
需要指出的是,现有技术中,FPGA加速卡无法直接从主机系统中的内存中读取数据,需要先经过DMA将数据库原始数据搬移到加速卡中的DDR中,计算完成后将结果再从加速卡中搬回到主机内存中,这个过程中,数据传输的延时是不可忽略的因素。满足查询条件的数据越多,搬移的数据量越大,返回的数据越多,PCIe传输延时就越大。当选择的数据量大于某一值时,通过FPGA进行加速的方式反而造成更大的延时,没有达到性能加速的效果,此时应该选择使用原有的软件查询方式执行任务,从而提高整体的性能,提升用户体验。It should be pointed out that in the prior art, the FPGA accelerator card cannot directly read data from the memory in the host system. It needs to move the original data of the database to the DDR in the accelerator card through DMA first, and then transfer the result from The acceleration card is moved back to the host memory. In this process, the delay of data transmission is a non-negligible factor. The more data that meets the query conditions, the larger the amount of data moved, the more data returned, and the greater the PCIe transmission delay. When the amount of data selected is greater than a certain value, the FPGA acceleration method will cause a greater delay instead of achieving the effect of performance acceleration. At this time, you should choose to use the original software query method to perform tasks, thereby improving the overall performance. performance and improve user experience.
因此,本申请可以在基于FPGA异构加速的数据库系统中,可自适应根据查询任务量的大小,来动态选择是否将查询任务卸载到FPGA中执行的方法。当符合查询条件的数据小于设置的阈值θ时,启动FPGA加速功能,查询任务下的数据查询操作卸载到FPGA中执行,反之,FPGA硬件加速功能无效,查询任务下的数据查询操作依然在软件中执行。本申请中,设计了一个任务选择器,在主机端中实现,在用户发来查询任务时,CPU在做预处理时进行判断,根据阈值和待选择数据量的大小进行选择是否启动FPGA加速器。参见图3所示,本申请实施例公开了一种具体的数据查询方法流程图。任务选择器在主机端实现,在原始数据进行搬移之前先进行阈值判断,满足加速器启动条件的,将查询任务卸载到FPGA中执行,否则继续执行原流程,即由CPU执行查询任务。具体流程包括:系统在第一次执行任务时也即执行当前任务下的首次数据查询操作时,默认选择启动FPGA加速器,任务选择器是打开状态;当一个数据块处理完成后,计算符合查询条件的数据的数据量,并计算该数据量与该数据块的总数据量的比值,记为θe,将这个值作为下一个数据块的数据查询操作是否使用硬件加速的预估值,当θe小于θ时,下一个数据块的查询操作依然使用硬件加速,否则使用原有的软件方式进行查询。后续数据查询操作,θe的值使用已经完成计算的所有数据块中满足查询条件数据量与相应数据块的总数据量的比值的平均值,这样能比较均衡地评估某一查询条件下满足条件的数据量,更为客观真实。Therefore, in the database system based on FPGA heterogeneous acceleration, this application can dynamically select whether to offload the query task to the FPGA according to the size of the query task. When the data that meets the query conditions is less than the set threshold θ, the FPGA acceleration function is started, and the data query operation under the query task is offloaded to the FPGA for execution. Otherwise, the FPGA hardware acceleration function is invalid, and the data query operation under the query task is still in the software. implement. In this application, a task selector is designed and implemented on the host side. When the user sends a query task, the CPU makes a judgment during preprocessing, and selects whether to start the FPGA accelerator according to the threshold and the amount of data to be selected. Referring to FIG. 3 , the embodiment of the present application discloses a specific data query method flow chart. The task selector is implemented on the host side. Before the original data is moved, threshold judgment is performed. If the accelerator startup condition is met, the query task is offloaded to the FPGA for execution. Otherwise, the original process is continued, that is, the query task is executed by the CPU. The specific process includes: when the system executes the task for the first time, that is, when it executes the first data query operation under the current task, the FPGA accelerator is selected by default, and the task selector is opened; when a data block is processed, the calculation meets the query conditions The data volume of the data, and calculate the ratio of the data volume to the total data volume of the data block, denoted as θe, and use this value as the estimated value of whether to use hardware acceleration for the data query operation of the next data block, when θe is less than When θ, the query operation of the next data block still uses hardware acceleration, otherwise, the original software method is used for query. For subsequent data query operations, the value of θe uses the average value of the ratio of the data volume that satisfies the query condition in all data blocks that have been calculated to the total data volume of the corresponding data block. The amount of data is more objective and real.
可见,本申请实施例当执行针对数据块的数据查询操作时,获取数据占比均值;其中,所述数据占比均值为历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,并且,所述数据查询操作均为同一数据查询任务下的查询操作,所述数据块为目标数据库的数据块,所述目标数据库为所述数据查询任务对应的数据库,然后判断所述数据占比均值是否小于数据占比阈值,若是,则将当前数据块搬移至FPGA加速设备进行处理,否则利用数据库管理软件对当前数据块进行处理。也即,本申请中,当执行数据查询任务下针对数据块的数据查询操作时,将该数据查询任务下的历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值与数据占比阈值比较,小于阈值时才会将当前数据块搬移至FPGA加速设备进行处理,否则利用数据库管理软件对当前数据块进行处理,这样, 根据符合查询条件的数据量执行数据查询任务,判断是否将数据块搬移至FPGA中处理,避免了符合查询条件的数据量较大时仍搬移至FPGA中处理,造成数据延时较大的问题,能够提升数据查询任务的处理效率,从而提升用户体验。It can be seen that in the embodiment of the present application, when the data query operation for the data block is executed, the average value of the data proportion is obtained; wherein, the average value of the data proportion is the amount of data that meets the query conditions in previous data query operations and the total data of the corresponding data block The average value of the ratio of the amount, and the data query operations are all query operations under the same data query task, the data block is a data block of the target database, and the target database is the database corresponding to the data query task, and then It is judged whether the average value of the data ratio is less than the data ratio threshold, if so, the current data block is moved to the FPGA acceleration device for processing, otherwise the current data block is processed by using the database management software. That is, in this application, when performing a data query operation for a data block under a data query task, the ratio of the amount of data that meets the query conditions in the previous data query operations under the data query task to the total data amount of the corresponding data block Compared with the average value of the data ratio threshold, the current data block will be moved to the FPGA acceleration device for processing when it is less than the threshold value, otherwise, the current data block will be processed by the database management software. In this way, the data query is performed according to the amount of data that meets the query conditions Task, to determine whether to move the data block to the FPGA for processing, avoiding the problem of large data delay caused by moving the data block to the FPGA when the amount of data that meets the query conditions is large, and can improve the processing efficiency of the data query task, thereby Improve user experience.
参见图4所示,本申请实施例公开了一种具体的数据查询方法,包括:Referring to Figure 4, the embodiment of the present application discloses a specific data query method, including:
步骤S21:将测试数据集对应的数据查询任务分别利用FPGA加速设备和数据库管理软件进行处理。Step S21: The data query task corresponding to the test data set is processed by the FPGA acceleration device and the database management software respectively.
步骤S22:确定利用FPGA加速设备进行处理的第一消耗时间,以及利用数据库管理软件进行处理的第二消耗时间。Step S22: Determine the first time consumption for processing by the FPGA acceleration device and the second time consumption for processing by the database management software.
步骤S23:基于第一消耗时间和第二消耗时间从全部数据占比中确定出目标占比,并将目标占比作为数据占比阈值。Step S23: Determine the target proportion from all data proportions based on the first consumption time and the second consumption time, and use the target proportion as a data proportion threshold.
其中,数据占比为数据查询任务下任一次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值,并且,目标占比对应的第一消耗时间与第二消耗时间相等。Among them, the data ratio is the ratio of the amount of data that meets the query conditions to the total data amount of the corresponding data block in any data query operation under the data query task, and the first consumption time corresponding to the target ratio is equal to the second consumption time .
需要指出的是,对于小于目标占比的数据占比,相应的第一消耗时间小于第二消耗时间,对于大于目标占比的数据占比,相应的第一消耗时间大于第二消耗时间。It should be pointed out that, for a data proportion smaller than the target proportion, the corresponding first consumption time is shorter than the second consumption time, and for a data proportion greater than the target proportion, the corresponding first consumption time is greater than the second consumption time.
也即,本申请实施例可以通过实验的方式确定阈值的大小,即基于某一个测试集分别进行启动加速器和关闭加速器两种方式执行查询操作,通过修改满足查询条件的数据量的大小,测试两种查询方式的时间消耗情况,找到性能的交叉点,将满足查询条件的数据量占数据块总大小的百分比,当做阈值θ。该值即可作为当次验证的数据库系统中FPGA加速选择器启动与否的阈值。That is to say, the embodiment of the present application can determine the size of the threshold through experiments, that is, based on a certain test set, the query operation is performed in two ways of starting the accelerator and closing the accelerator respectively. The time consumption of each query method is used to find the intersection point of performance, and the percentage of the amount of data that satisfies the query condition to the total size of the data block is used as the threshold θ. This value can be used as the threshold for whether the FPGA acceleration selector is started or not in the current verified database system.
步骤S24:当执行针对数据块的数据查询操作时,获取数据占比均值;其中,数据占比均值为历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,并且,数据查询操作均为同一数据查询任务下的查询操作,数据块为目标数据库的数据块,目标数据库为数据查询任务对应的数据库。Step S24: When performing a data query operation for a data block, obtain the average value of the data proportion; wherein, the average data proportion is the average value of the ratio of the amount of data that meets the query conditions in previous data query operations to the total data amount of the corresponding data block , and the data query operations are all query operations under the same data query task, the data block is the data block of the target database, and the target database is the database corresponding to the data query task.
步骤S25:判断数据占比均值是否大于数据占比阈值。Step S25: Determine whether the average data proportion is greater than the data proportion threshold.
步骤S26:若是,则将当前数据块搬移至FPGA加速设备进行处理,否则利用数据库管理软件对当前数据块进行处理。Step S26: If yes, move the current data block to the FPGA acceleration device for processing, otherwise use the database management software to process the current data block.
当然,在另外一些实施例中,在将当前数据块搬移至FPGA加速设备进行处理,或利用数据库管理软件对当前数据块进行处理之后,可以对本次以及历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值进行中值滤波,并将滤波结果作为下一次数据查询操作对应的所述数据占比阈值,或,基于所述目标数据库的类型、所述目标数据库中数据的类型、查询数据的类型、本次以及历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值确定出下一次数据查询操作对应的所述数据占比阈值。Of course, in some other embodiments, after the current data block is moved to the FPGA acceleration device for processing, or the current data block is processed by the database management software, the data that meets the query conditions in this and previous data query operations can be The ratio of the amount to the total data amount of the corresponding data block is subjected to median filtering, and the filtering result is used as the data proportion threshold corresponding to the next data query operation, or, based on the type of the target database, in the target database The type of data, the type of query data, the ratio of the amount of data meeting the query conditions in this and previous data query operations to the total data amount of the corresponding data block determines the corresponding data proportion threshold for the next data query operation.
当然,也可以人工经验确定数据占比阈值。具体的,可根据实际的应用场景、数据库的数据存储结构以及查询条件等,选择适合实际系统的阈值确定方法。Of course, the data proportion threshold can also be manually determined. Specifically, a threshold determination method suitable for an actual system may be selected according to an actual application scenario, a data storage structure of a database, and query conditions.
参见图5所示,本申请实施例公开了一种数据查询装置,其特征在于,包括:Referring to Figure 5, the embodiment of the present application discloses a data query device, which is characterized in that it includes:
数据占比均值获取模块11,用于当执行针对数据块的数据查询操作时,获取数据占比均值;其中,所述数据占比均值为历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,并且,所述数据查询操作均为同一数据查询任务下的查询操作,所述数据块为目标数据库的数据块,所述目标数据库为所述数据查询任务对应的数据库。The data ratio mean value acquisition module 11 is used to obtain the data ratio mean value when performing a data query operation for a data block; wherein, the data ratio mean value is the amount of data and the corresponding data that meet the query conditions in previous data query operations The average value of the ratio of the total data volume of the block, and the data query operations are all query operations under the same data query task, the data block is a data block of the target database, and the target database corresponds to the data query task database.
数据占比阈值判断模块12,用于判断所述数据占比均值是否小于数据占比阈值。The data proportion threshold judging module 12 is configured to judge whether the average data proportion is smaller than the data proportion threshold.
数据块处理模块13,用于将当前数据块搬移至FPGA加速设备进行处理,否则利用数据库管理软件对当前数据块进行处理。The data block processing module 13 is used to move the current data block to the FPGA acceleration device for processing, otherwise use the database management software to process the current data block.
可见,本申请实施例当执行针对数据块的数据查询操作时,获取数据占比均值;其中,所述数据占比均值为历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,并且,所述数据查询操作均为同一数据查询任务下的查询操作,所述数据块为目标数据库的数据块,所述目标数据库为所述数据查询任务对应的数据库,然后判断所述数据占比均值 是否小于数据占比阈值,若是,则将当前数据块搬移至FPGA加速设备进行处理,否则利用数据库管理软件对当前数据块进行处理。也即,本申请中,当执行数据查询任务下针对数据块的数据查询操作时,将该数据查询任务下的历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值与数据占比阈值比较,小于阈值时才会将当前数据块搬移至FPGA加速设备进行处理,否则利用数据库管理软件对当前数据块进行处理,这样,根据符合查询条件的数据量执行数据查询任务,判断是否将数据块搬移至FPGA中处理,避免了符合查询条件的数据量较大时仍搬移至FPGA中处理,造成数据延时较大的问题,能够提升数据查询任务的处理效率,从而提升用户体验。It can be seen that in the embodiment of the present application, when the data query operation for the data block is executed, the average value of the data proportion is obtained; wherein, the average value of the data proportion is the amount of data that meets the query conditions in previous data query operations and the total data of the corresponding data block The average value of the ratio of the amount, and the data query operations are all query operations under the same data query task, the data block is a data block of the target database, and the target database is the database corresponding to the data query task, and then It is judged whether the average value of the data ratio is less than the data ratio threshold, if so, the current data block is moved to the FPGA acceleration device for processing, otherwise the current data block is processed by using the database management software. That is, in this application, when performing a data query operation for a data block under a data query task, the ratio of the amount of data that meets the query conditions in the previous data query operations under the data query task to the total data amount of the corresponding data block Compared with the data ratio threshold, the current data block will be moved to the FPGA acceleration device for processing when it is less than the threshold, otherwise, the current data block will be processed by the database management software, so that the data query is performed according to the amount of data that meets the query conditions Task, to determine whether to move the data block to the FPGA for processing, avoiding the problem of large data delay caused by moving the data block to the FPGA when the amount of data that meets the query conditions is large, and can improve the processing efficiency of the data query task, thereby Improve user experience.
进一步的,所述装置还包括阈值确定模块,具体用于将测试数据集对应的数据查询任务分别利用FPGA加速设备和数据库管理软件进行处理;确定利用FPGA加速设备进行处理的第一消耗时间,以及利用数据库管理软件进行处理的第二消耗时间;基于所述第一消耗时间和所述第二消耗时间从全部数据占比中确定出目标占比,并将所述目标占比作为所述数据占比阈值;Further, the device also includes a threshold determination module, specifically for processing the data query tasks corresponding to the test data set using the FPGA acceleration device and the database management software respectively; determining the first consumption time for processing by the FPGA acceleration device, and The second consumption time for processing by using database management software; determining a target proportion from all data proportions based on the first consumption time and the second consumption time, and using the target proportion as the data proportion than the threshold;
其中,所述数据占比为数据查询任务下任一次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值,并且,所述目标占比对应的所述第一消耗时间与所述第二消耗时间相等。Wherein, the data ratio is the ratio of the amount of data meeting the query conditions to the total data amount of the corresponding data block in any data query operation under the data query task, and the first consumption time corresponding to the target ratio It is equal to the second elapsed time.
在一些实施例中,阈值确定模块,具体用于在数据块处理模块13将当前数据块搬移至FPGA加速设备进行处理,或利用数据库管理软件对当前数据块进行处理之后,对本次以及历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值进行中值滤波,并将滤波结果作为下一次数据查询操作对应的所述数据占比阈值。In some embodiments, the threshold determination module is specifically used to process the current and previous data blocks after the data block processing module 13 moves the current data block to the FPGA acceleration device for processing, or uses the database management software to process the current data block. In the query operation, the ratio of the amount of data meeting the query conditions to the total data amount of the corresponding data block is subjected to median filtering, and the filtering result is used as the data proportion threshold corresponding to the next data query operation.
在一些实施例中,阈值确定模块,具体用于在数据块处理模块13将当前数据块搬移至FPGA加速设备进行处理,或利用数据库管理软件对当前数据块进行处理之后,还包括:In some embodiments, the threshold determination module is specifically used to move the current data block to the FPGA acceleration device for processing by the data block processing module 13, or after the current data block is processed by the database management software, it also includes:
基于所述目标数据库的类型、所述目标数据库中数据的类型、查询数据的类型、本次以及历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值确定出下一次数据查询操作对应的所述数据占比阈值。Based on the type of the target database, the type of data in the target database, the type of query data, the ratio of the amount of data meeting the query conditions in this and previous data query operations to the total data amount of the corresponding data block to determine the next time The data proportion threshold corresponding to the data query operation.
并且,本申请实施例还包括数据占比均值确定模块,具体用于在数据块处理模块13将当前数据块搬移至FPGA加速设备进行处理,或利用数据库管理软件对当前数据块进行处理之后,计算本次以及历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,将该均值作为下一次数据查询操作对应的数据占比均值。Moreover, the embodiment of the present application also includes a data ratio mean determination module, which is specifically used to calculate the current data block after the data block processing module 13 moves the current data block to the FPGA acceleration device for processing, or uses the database management software to process the current data block. The average value of the ratio of the amount of data that meets the query conditions to the total data amount of the corresponding data block in this and previous data query operations is used as the average value of the data ratio corresponding to the next data query operation.
进一步的,所述装置还包括:数据块大小确定模块,用于基于所述目标数据库的数据存储方式和数据量、以及所述FPGA加速设备的带宽和数据处理能力确定每次数据查询操作对应的所述数据块的大小。Further, the device also includes: a data block size determination module, configured to determine the corresponding data block size of each data query operation based on the data storage method and data volume of the target database, and the bandwidth and data processing capability of the FPGA acceleration device. The size of the data block.
数据块处理模块13,还用于当首次执行所述数据查询任务下的针对数据块的数据查询操作时,则直接将当前数据块搬移至FPGA加速设备进行处理。The data block processing module 13 is further configured to directly move the current data block to the FPGA acceleration device for processing when the data query operation for the data block under the data query task is executed for the first time.
参见图6所示,本申请实施例公开了一种电子设备20,包括处理器21和存储器22;其中,所述存储器22,用于保存计算机程序;所述处理器21,用于执行所述计算机程序,其中,所述计算机程序被处理器执行时实现前述实施例公开的数据查询方法。As shown in FIG. 6, the embodiment of the present application discloses an electronic device 20, including a processor 21 and a memory 22; wherein, the memory 22 is used to store computer programs; the processor 21 is used to execute the A computer program, wherein, when the computer program is executed by a processor, the data query method disclosed in the foregoing embodiments is implemented.
关于上述数据查询方法的具体过程可以参考前述实施例中公开的相应内容,在此不再进行赘述。Regarding the specific process of the above data query method, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details are not repeated here.
并且,所述存储器22作为资源存储的载体,可以是只读存储器、随机存储器、磁盘或者光盘等,存储方式可以是短暂存储或者永久存储。Moreover, the memory 22, as a resource storage carrier, may be a read-only memory, random access memory, magnetic disk or optical disk, etc., and the storage method may be temporary storage or permanent storage.
另外,所述电子设备20还包括电源23、通信接口24、输入输出接口25和通信总线26;其中,所述电源23用于为所述电子设备20上的各硬件设备提供工作电压;所述通信接口24能够为所述电子设备20创建与外界设备之间的数据传输通道,其所遵循的通信协议是能够适用于本申请技术方案的任意通信协议,在此不对其进行具体限定;所述输入输出接口25,用于获取外界输入数据或向外界输出数据,其具体的接口类型可以根据具体应用需要进行选取,在此不进行具体限定。In addition, the electronic device 20 also includes a power supply 23, a communication interface 24, an input and output interface 25, and a communication bus 26; wherein the power supply 23 is used to provide operating voltages for each hardware device on the electronic device 20; the The communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, which is not specifically limited here; The input and output interface 25 is used to obtain external input data or output data to the external, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
进一步的,本申请实施例还公开了一种计算机可读存储介质,用于保存计算机程序,其中,所述计算机程序被处理器执行时实现前述实施例公开的数据查询方法。Further, the embodiment of the present application also discloses a computer-readable storage medium for storing a computer program, wherein, when the computer program is executed by a processor, the data query method disclosed in the foregoing embodiments is implemented.
关于上述数据查询方法的具体过程可以参考前述实施例中公开的相应内容,在此不再进行赘述。Regarding the specific process of the above data query method, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details are not repeated here.
本说明书中各个实施例采用递进的方式描述,每个实施例重点说明的都是与其它实施例的不同之处,各个实施例之间相同或相似部分互相参见即可。对于实施例公开的装置而言,由于其与实施例公开的方法相对应,所以描述的比较简单,相关之处参见方法部分说明即可。Each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same or similar parts of each embodiment can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and for the related information, please refer to the description of the method part.
结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块,或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be directly implemented by hardware, software modules executed by a processor, or a combination of both. Software modules can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other Any other known storage medium.
以上对本申请所提供的一种数据查询方法、装置、设备及介质进行了详细介绍,本文中应用了具体个例对本申请的原理及实施方式进行了阐述,以上实施例的说明只是用于帮助理解本申请的方法及其核心思想;同时,对于本领域的一般技术人员,依据本申请的思想,在具体实施方式及应用范围上均会有改变之处,综上所述,本说明书内容不应理解为对本申请的限制。The above is a detailed introduction to a data query method, device, equipment and medium provided by this application. In this paper, specific examples are used to illustrate the principle and implementation of this application. The description of the above embodiments is only for helping understanding The method of this application and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation and scope of application. In summary, the content of this specification should not understood as a limitation of the application.

Claims (10)

  1. 一种数据查询方法,其特征在于,包括:A data query method, characterized in that, comprising:
    当执行针对数据块的数据查询操作时,获取数据占比均值;其中,所述数据占比均值为历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,并且,所述数据查询操作均为同一数据查询任务下的查询操作,所述数据块为目标数据库的数据块,所述目标数据库为所述数据查询任务对应的数据库;When performing a data query operation for a data block, the average value of the data ratio is obtained; wherein, the average value of the data ratio is the average value of the ratio of the amount of data that meets the query conditions in previous data query operations to the total data amount of the corresponding data block, Moreover, the data query operations are all query operations under the same data query task, the data block is a data block of a target database, and the target database is a database corresponding to the data query task;
    判断所述数据占比均值是否小于数据占比阈值;Judging whether the average value of the data proportion is less than the data proportion threshold;
    若是,则将当前数据块搬移至FPGA加速设备进行处理,否则利用数据库管理软件对当前数据块进行处理。If yes, move the current data block to the FPGA acceleration device for processing, otherwise use the database management software to process the current data block.
  2. 根据权利要求1所述的数据查询方法,其特征在于,还包括:The data query method according to claim 1, further comprising:
    将测试数据集对应的数据查询任务分别利用FPGA加速设备和数据库管理软件进行处理;The data query task corresponding to the test data set is processed by FPGA acceleration device and database management software;
    确定利用FPGA加速设备进行处理的第一消耗时间,以及利用数据库管理软件进行处理的第二消耗时间;Determining the first consuming time for processing by the FPGA acceleration device, and the second consuming time for processing by using the database management software;
    基于所述第一消耗时间和所述第二消耗时间从全部数据占比中确定出目标占比,并将所述目标占比作为所述数据占比阈值;determining a target proportion from all data proportions based on the first consumption time and the second consumption time, and using the target proportion as the data proportion threshold;
    其中,所述数据占比为数据查询任务下任一次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值,并且,所述目标占比对应的所述第一消耗时间与所述第二消耗时间相等。Wherein, the data ratio is the ratio of the amount of data meeting the query conditions to the total data amount of the corresponding data block in any data query operation under the data query task, and the first consumption time corresponding to the target ratio It is equal to the second elapsed time.
  3. 根据权利要求1所述的数据查询方法,其特征在于,将当前数据块搬移至FPGA加速设备进行处理,或利用数据库管理软件对当前数据块进行处理之后,还包括:The data query method according to claim 1, wherein, after the current data block is moved to the FPGA acceleration device for processing, or after the current data block is processed by database management software, it also includes:
    对本次以及历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值进行中值滤波,并将滤波结果作为下一次数据查询操作对应的所述数据占比阈值。Median filtering is performed on the ratio of the amount of data meeting the query conditions to the total data amount of the corresponding data block in this and previous data query operations, and the filtering result is used as the data proportion threshold corresponding to the next data query operation.
  4. 根据权利要求1所述的数据查询方法,其特征在于,将当前数据块搬移至FPGA加速设备进行处理,或利用数据库管理软件对当前数据块进行处理之后,还包括:The data query method according to claim 1, wherein, after the current data block is moved to the FPGA acceleration device for processing, or after the current data block is processed by database management software, it also includes:
    基于所述目标数据库的类型、所述目标数据库中数据的类型、查询数据的类型、本次以及历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值确定出下一次数据查询操作对应的所述数据占比阈值。Based on the type of the target database, the type of data in the target database, the type of query data, the ratio of the amount of data meeting the query conditions in this and previous data query operations to the total data amount of the corresponding data block to determine the next time The data proportion threshold corresponding to the data query operation.
  5. 根据权利要求1所述的数据查询方法,其特征在于,将当前数据块搬移至FPGA加速设备进行处理,或利用数据库管理软件对当前数据块进行处理之后,还包括:The data query method according to claim 1, wherein, after the current data block is moved to the FPGA acceleration device for processing, or after the current data block is processed by database management software, it also includes:
    计算本次以及历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,将该均值作为下一次数据查询操作对应的数据占比均值。Calculate the average value of the ratio of the amount of data that meets the query conditions to the total data amount of the corresponding data block in this and previous data query operations, and use the average value as the average value of the data proportion corresponding to the next data query operation.
  6. 根据权利要求1所述的数据查询方法,其特征在于,还包括:The data query method according to claim 1, further comprising:
    基于所述目标数据库的数据存储方式和数据量、以及所述FPGA加速设备的带宽和数据处理能力确定每次数据查询操作对应的所述数据块的大小。The size of the data block corresponding to each data query operation is determined based on the data storage method and data volume of the target database, and the bandwidth and data processing capability of the FPGA acceleration device.
  7. 根据权利要求1所述的数据查询方法,其特征在于,还包括:The data query method according to claim 1, further comprising:
    当首次执行所述数据查询任务下的针对数据块的数据查询操作时,则直接将当前数据块搬移至FPGA加速设备进行处理。When performing the data query operation for the data block under the data query task for the first time, the current data block is directly moved to the FPGA acceleration device for processing.
  8. 一种数据查询装置,其特征在于,包括:A data query device, characterized in that it comprises:
    数据占比均值获取模块,用于当执行针对数据块的数据查询操作时,获取数据占比均值;其中,所述数据占比均值为历次数据查询操作中符合查询条件的数据量与相应数据块的总数据量的比值的均值,并且,所述数据查询操作均为同一数据查询任务下的查询操作,所述数据块为目标数据库的数据块,所述目标数据库为所述数据查询任务对应的数据库;The data ratio mean value acquisition module is used to obtain the data ratio mean value when performing a data query operation for the data block; wherein, the data ratio mean value is the amount of data that meets the query conditions and the corresponding data block in previous data query operations The average value of the ratio of the total data volume, and the data query operations are all query operations under the same data query task, the data block is a data block of the target database, and the target database is the corresponding data query task database;
    数据占比阈值判断模块,用于判断所述数据占比均值是否小于数据占比阈值;A data proportion threshold judging module, configured to determine whether the data proportion average value is less than the data proportion threshold;
    数据块处理模块,用于将当前数据块搬移至FPGA加速设备进行处理,否则利用数据库管理软件对当前数据块进行处理。The data block processing module is used to move the current data block to the FPGA acceleration device for processing, or use the database management software to process the current data block.
  9. 一种电子设备,其特征在于,包括:An electronic device, characterized in that it comprises:
    存储器,用于保存计算机程序;memory for storing computer programs;
    处理器,用于执行所述计算机程序,以实现如权利要求1至7任一项所述的数据查询方法。A processor, configured to execute the computer program, so as to realize the data query method according to any one of claims 1-7.
  10. 一种计算机可读存储介质,其特征在于,用于保存计算机程序,所述计算机程序被处理器执行时实现如权利要求1至7任一项所述的数据查询方法。A computer-readable storage medium, characterized in that it is used to store a computer program, and when the computer program is executed by a processor, the data query method according to any one of claims 1 to 7 is implemented.
PCT/CN2021/134413 2021-09-03 2021-11-30 Data query method and apparatus, and device and medium WO2023029254A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111029248.7 2021-09-03
CN202111029248.7A CN113468220A (en) 2021-09-03 2021-09-03 Data query method, device, equipment and medium

Publications (1)

Publication Number Publication Date
WO2023029254A1 true WO2023029254A1 (en) 2023-03-09

Family

ID=77868032

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/134413 WO2023029254A1 (en) 2021-09-03 2021-11-30 Data query method and apparatus, and device and medium

Country Status (2)

Country Link
CN (1) CN113468220A (en)
WO (1) WO2023029254A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113468220A (en) * 2021-09-03 2021-10-01 苏州浪潮智能科技有限公司 Data query method, device, equipment and medium
CN114443616B (en) * 2021-12-30 2024-01-16 苏州浪潮智能科技有限公司 FPGA-based parallel heterogeneous database acceleration method and device
CN115544069B (en) * 2022-09-26 2023-06-20 山东浪潮科学研究院有限公司 Reconfigurable database query acceleration processor and system
CN117407179B (en) * 2023-12-15 2024-04-02 成都凯迪飞研科技有限责任公司 Acceleration sub-card processing method and system based on cooperative processing

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105677812A (en) * 2015-12-31 2016-06-15 华为技术有限公司 Method and device for querying data
US20180373760A1 (en) * 2017-06-23 2018-12-27 Xilinx, Inc. Parallel compute offload to database accelerator
CN110704489A (en) * 2019-09-29 2020-01-17 苏州浪潮智能科技有限公司 Database query method, device, equipment and computer storage medium
CN112699140A (en) * 2019-10-23 2021-04-23 阿里巴巴集团控股有限公司 Data processing method, device, equipment and storage medium
CN113468220A (en) * 2021-09-03 2021-10-01 苏州浪潮智能科技有限公司 Data query method, device, equipment and medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8995729B2 (en) * 2011-08-30 2015-03-31 The Mitre Corporation Accelerated comparison using scores from coarse and fine matching processes
CN109325034B (en) * 2018-10-12 2023-10-20 平安科技(深圳)有限公司 Data processing method, device, computer equipment and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105677812A (en) * 2015-12-31 2016-06-15 华为技术有限公司 Method and device for querying data
US20180373760A1 (en) * 2017-06-23 2018-12-27 Xilinx, Inc. Parallel compute offload to database accelerator
CN110704489A (en) * 2019-09-29 2020-01-17 苏州浪潮智能科技有限公司 Database query method, device, equipment and computer storage medium
CN112699140A (en) * 2019-10-23 2021-04-23 阿里巴巴集团控股有限公司 Data processing method, device, equipment and storage medium
CN113468220A (en) * 2021-09-03 2021-10-01 苏州浪潮智能科技有限公司 Data query method, device, equipment and medium

Also Published As

Publication number Publication date
CN113468220A (en) 2021-10-01

Similar Documents

Publication Publication Date Title
WO2023029254A1 (en) Data query method and apparatus, and device and medium
CN102906726B (en) Association process accelerated method, Apparatus and system
US9300578B2 (en) Large receive offload functionality for a system on chip
CN106293944B (en) non-consistency-based I/O access system and optimization method under virtualized multi-core environment
CN108268328B (en) Data processing device and computer
CN110351336B (en) Edge service migration method based on docker container
CN111625558A (en) Server architecture, database query method thereof and storage medium
WO2022199027A1 (en) Random write method, electronic device and storage medium
WO2021164163A1 (en) Request processing method and apparatus, device and storage medium
JP2019185764A (en) Data-centric computing architecture based on storage server in ndp server data center
CN103049215B (en) The method of IO command process and solid state hard disc thereof
WO2021207923A1 (en) Cluster expansion method and apparatus, storage medium, and electronic device
CN111880611B (en) Server for quick transaction and quick transaction data processing method
Lin et al. Towards Accelerating Data Intensive Application's Shuffle Process Using SmartNICs
US9588570B2 (en) Apparatus and method for adjusting bandwidth
CN111209263A (en) Data storage method, device, equipment and storage medium
CN116126916A (en) Data query method, device and equipment based on intelligent network card
Li et al. Improving spark performance with zero-copy buffer management and RDMA
WO2021159608A1 (en) Protocol buffers-based mirror cache method
US20210328945A1 (en) Configurable receive buffer size
CN116431356B (en) Cloud network cache acceleration method and system based on intelligent network card
CN115695432B (en) Load balancing method and device, electronic equipment and storage medium
CN113918527B (en) Scheduling method and device based on file cache and computing equipment
WO2024027395A1 (en) Data processing method and apparatus
CN112799783B (en) Virtual machine placement method and system for cloud platform resource fragment optimization

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21955782

Country of ref document: EP

Kind code of ref document: A1