WO2021004266A1 - Procédé et appareil d'insertion de données, dispositif et support de stockage - Google Patents

Procédé et appareil d'insertion de données, dispositif et support de stockage Download PDF

Info

Publication number
WO2021004266A1
WO2021004266A1 PCT/CN2020/097563 CN2020097563W WO2021004266A1 WO 2021004266 A1 WO2021004266 A1 WO 2021004266A1 CN 2020097563 W CN2020097563 W CN 2020097563W WO 2021004266 A1 WO2021004266 A1 WO 2021004266A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
group
partition
target table
current
Prior art date
Application number
PCT/CN2020/097563
Other languages
English (en)
Chinese (zh)
Inventor
张泓毅
陈振强
Original Assignee
星环信息科技（上海）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 星环信息科技（上海）有限公司 filed Critical 星环信息科技（上海）有限公司
Publication of WO2021004266A1 publication Critical patent/WO2021004266A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G06F16/2255—Hash tables
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2282—Tablespace storage structures; Management thereof

Definitions

  • This application relates to data storage technology, such as a data insertion method, device, equipment, and storage medium.
  • Partitioning is a commonly used data organization method in databases. Most of the related technologies use a single partition to insert one by one, and the performance cannot meet the requirements when processing large quantities of data. If you want to insert different partitions at the same time, it will cause system instability due to operating too many files in different partitions at the same time for a long time, such as excessive memory usage and too many open file handles.
  • This application provides a data insertion method, device, equipment, and storage medium to improve the performance of dynamic partition insertion while ensuring system stability.
  • a data insertion method including:
  • the insertion action type is dynamic partition insertion, determine the partition and bucket information of the target table according to the meta information of the target table;
  • each group of data to be inserted is dynamically inserted into the corresponding target table file in turn.
  • a data insertion device including:
  • An insert action type determining module configured to determine the insert action type of the data insert command according to the acquired data insert command
  • the partitioning bucket information determining module is configured to determine the partitioning and bucketing information of the target table according to the meta-information of the target table when the insertion action type is dynamic partition insertion;
  • a data grouping module configured to divide the data to be inserted into at least one group according to the partition and bucket information
  • a data sorting module configured to sort the data to be inserted in each group according to the partition and bucket information
  • the data insertion module is configured to dynamically insert each group of data to be inserted into the corresponding target table file in sequence according to the sequence of the data to be inserted in each group.
  • a device is also provided, and the device includes:
  • One or more processors are One or more processors;
  • Memory set to store one or more programs
  • the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement the data insertion method provided in any embodiment of the present application.
  • a storage medium containing computer-executable instructions is also provided, and the computer-executable instructions are used to execute the data insertion method provided in any embodiment of the present application when executed by a computer processor.
  • FIG. 1 is a flowchart of a data insertion method in Embodiment 1 of the present application
  • FIG. 2 is a flowchart of a data insertion method in Embodiment 2 of the present application.
  • FIG. 3 is a flowchart of a data insertion method in Embodiment 3 of the present application.
  • FIG. 5 is a flowchart of a data insertion method in Embodiment 5 of the present application.
  • FIG. 6 is a schematic structural diagram of a data insertion device in the sixth embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a device in Embodiment 7 of the present application.
  • Fig. 1 is a flowchart of a data insertion method provided in the first embodiment of this application. This embodiment is applicable to the case of importing data into a database.
  • the method can be executed by a data insertion device, which can be hardware and/or Software implementation, including the following steps:
  • Step 110 Determine the insertion action type of the data insertion command according to the acquired data insertion command.
  • a corresponding insertion command when a data insertion task needs to be executed, a corresponding insertion command will be executed.
  • the data insertion command of the database engine it can be determined whether it is a dynamic partition insertion or a static partition insertion.
  • Static partition insertion refers to specifying the target partition when inserting data, and it can only be inserted into one partition at a time;
  • dynamic partition insertion refers to not specifying the target partition when inserting data, but dynamically judging the target partition of the data according to the partition column. Insert into multiple partitions.
  • Step 120 When the insert action type is dynamic partition insertion, determine the partition and bucket information of the target table according to the meta information of the target table.
  • the meta information of the target table is obtained, and the partition and bucket information of the target table is determined. For example, according to the insert command of the sql statement, determine whether it is a dynamic partition insert, and then obtain the meta-information of the target table, determine whether the target table is a partitioned non-bucketed table, or a partitioned bucketed table, and partition columns , Bucket column and other information.
  • Step 130 According to the partition and bucket information, divide the data to be inserted into at least one group.
  • the target table is a bucketed table
  • the data to be inserted when the data to be inserted is grouped, ensure that the data of the same bucketed file is in the same group. If the target table is not a bucketed table, theoretically any grouping of the inserted data will not be inserted into the dynamic partition. Make an impact.
  • the data to be inserted can be grouped to avoid too much data to be inserted in a single group. Generally, the data to be inserted can be Group evenly.
  • the data to be inserted is divided into at least one group according to the first grouping rule.
  • dividing the data to be inserted into at least one group according to the first grouping rule includes: obtaining a preset number of groups; determining the group identification number of each piece of data to be inserted, wherein the group identification number includes the preset field of the data to be inserted The hash value or the random number corresponding to the data to be inserted; the result of modulo the preset group number and the group number according to the group identification number, and the data to be inserted is allocated to the result of the modulus and the group number corresponding to the Array.
  • the group identification number of the data to be inserted is modulo the preset grouping number according to the group identification number, and the result of the modulus is matched with the group number, so that the data to be inserted is allocated to the array corresponding to each group of data to be inserted.
  • the preset group number is 7 (group number is 0-6)
  • the hash value of a selected field of the data to be inserted is 123878437872
  • the modulo 7 is 0, so the data to be inserted is assigned to the group number 0 group. It is also possible to divide the data to be inserted into at least one group according to the amount of data to be inserted and the amount of the preset single group of data; wherein the amount of data to be inserted in each group is less than or equal to the amount of the preset single group of data.
  • the number of data to be inserted is less than or equal to the preset single group of data, then only one group is enough; if the number of data to be inserted is greater than the preset single group of data, then it is divided into multiple groups, each group to be inserted
  • the amount of data is less than or equal to the preset single set of data amount.
  • the data can be grouped in equal proportions or in unequal proportions, which is not limited here.
  • Step 140 Sort the data to be inserted in each group according to the partition and bucket information.
  • each group of data to be inserted is sorted within the group according to the value of the partition column, and the sorting is to ensure that the data of the same partition is arranged continuously.
  • the data to be inserted corresponding to the same target table partition is adjacent and arranged consecutively.
  • the data to be inserted corresponding to the same target table partition is inserted into the partition file of the target table. Insert the data to be inserted into the next target table partition.
  • Step 150 According to the order of the data to be inserted in each group, dynamically insert each group of data to be inserted into the corresponding target table file in turn.
  • the technical solution of this embodiment solves the problem of system instability caused by operating a large number of files in different partitions at the same time for a long time by sorting the data to be inserted into the target table one by one, inserting into the target table one by one, and ensuring the stability of the system. Next, the effect of improving the performance of dynamic partition insertion.
  • Fig. 2 is a flowchart of a data insertion method provided in the second embodiment of the application.
  • the technical solution of this embodiment is described on the basis of the above technical solution, and includes the following steps:
  • Step 210 Determine the insertion action type of the data insertion command according to the acquired data insertion command.
  • Step 220 When the insert action type is dynamic partition insertion, determine the partition and bucket information of the target table according to the meta information of the target table.
  • Step 230 When the target table is a partitioned and bucketed table, group the data to be inserted according to the hash value of the bucketed column.
  • the target table is a partition and bucket table
  • the data must be in the same group, which is semantically equivalent to distribute by buckets.
  • Step 240 Sort the data to be inserted in the group according to the partition and bucket information.
  • Step 250 According to the sequence of the data to be inserted in the group, dynamically insert each group of data to be inserted into the corresponding target table file in turn.
  • the technical solution of this embodiment by grouping the data in the same bucket, ensures that the data to be inserted that needs to be inserted into the same bucket file are in the same group, so as to avoid repeated scanning of partitions during dynamic partition insertion, and improve dynamic partition insertion. performance.
  • Fig. 3 is a flowchart of a data insertion method provided in the third embodiment of the application.
  • the technical solution of this embodiment is described on the basis of the above technical solution, and includes the following steps:
  • Step 310 Determine the insertion action type of the data insertion command according to the acquired data insertion command.
  • Step 320 When the insert action type is dynamic partition insertion, determine the partition and bucket information of the target table according to the meta information of the target table.
  • Step 330 According to the partition and bucket information, divide the data to be inserted into at least one group.
  • Step 340 When the target table is a multi-level partition table, sort the data to be inserted in the group according to the order of the partitions of the target table.
  • the target table is a multi-level partition table determined by the partition and bucket information
  • the data to be inserted in the group is sorted in order according to the partition order of the target table. Semantically equivalent to sort by partition column 1, partition column 2... This ensures the continuous arrangement of data in the same partition.
  • Step 350 When the target table is a multi-level range partition table, sort the data to be inserted in the group according to the sequence after the target table partition field is mapped to the range partition of the target table.
  • the target table is determined by the partition and bucket information to be a multi-level range partition table
  • the multi-level range partition table is not a single-value partition corresponding to a value of a partition key, but a range of a partition corresponding to a partition key value. That is, the range of the value of a partition key. Therefore, multi-level range partition sorting should sort the data to be inserted in the group after the partition field is mapped to the partition.
  • the range partition r1, r2 with 2 fields.
  • r1 is the date (date) type
  • r2 is the integer (int) type
  • the range partition is first partitioned by the month of r1 ([1970-01-01,1970-01-31] a partition, [1970-02-01,1970 -02-28] a partition), and then partition according to the hundreds of r2 (ie [0,99] is in a partition, [100,199] is in a partition, etc.). Then, to_month(r1), r2/10 should be sorted.
  • the equivalent syntax is distribute by to_month(r1), r2/10.
  • the data to be inserted in the group is sorted directly by the value of the partition key, the following order may appear: (1970-01-01,1),(1970-01-01,11),(1970-01-02,2 ), and they belong to partition 1, partition 2, and partition 1, respectively.
  • the data in the same partition is not arranged together. If it is for a single range partition field, it can be simplified to directly sort the data to be inserted in the group by the value of the partition field.
  • Step 360 According to the sequence of the data to be inserted in the group, dynamically insert each group of data to be inserted into the corresponding target table file in turn.
  • step 340 and step 350 are executed selectively according to the partition type of the target table, and the order of execution is not limited as shown in FIG. 3.
  • the data to be inserted in the group is sorted through the case that the target table is a range partition to ensure that the data to be inserted in the same partition is arranged continuously, so that when the data in the group is dynamically partitioned and inserted sequentially, There is no need to repeatedly open different partition files, which improves the performance of dynamic partition insertion.
  • FIG. 4 is a flowchart of a data insertion method provided in Embodiment 4 of this application.
  • the technical solution of this embodiment is described on the basis of the above technical solution, and includes the following steps:
  • Step 410 Determine the insertion action type of the data insertion command according to the acquired data insertion command.
  • Step 420 When the insert action type is dynamic partition insertion, determine the partition and bucket information of the target table according to the meta information of the target table.
  • Step 430 According to the partition and bucket information, divide the data to be inserted into at least one group.
  • Step 440 Sort the data to be inserted in the group according to the partition and bucket information.
  • Step 450 Read the first row in the current group, open the file handle corresponding to the first row as the current file handle, and write the data to be inserted in the first row into the current file handle.
  • Step 460 Read the next row as the current row in sequence, and if the current row belongs to the target table file corresponding to the current file handle, write the data to be inserted in the current row into the current file handle.
  • Step 470 If the current row does not belong to the target table file corresponding to the current file handle, close the current file handle, open the file handle corresponding to the current row, and write the data to be inserted in the current row into the file handle corresponding to the current row.
  • Step 480 When the next row cannot be read in the current group, the insertion operation of the data to be inserted in the group is ended; the insertion operation of all groups is completed in a preset order.
  • the target table is a partitioned and bucketed table
  • the data to be inserted in the group is sorted according to the partition attribution, thus ensuring The data belonging to the same partition must be arranged continuously during the insertion process. Therefore, in the process of inserting files, you only need to open one file at a time for continuous insertion. When a row is found that does not belong to the currently opened file, then all the data to be inserted belonging to the currently opened file in this group must have been inserted previously. Therefore, you can directly close the currently opened file, and open the corresponding file to which the current line belongs, and you can continue to insert.
  • the SQL insert instruction is executed to query the meta information of the target table to obtain the partition and bucket information of the target table. Determine whether it is a dynamic partition insertion. If it is, determine whether the target table is bucketed according to the meta-information of the target table. If the target table is bucketed, group the source table data according to the hash value of the bucket column. If the target table is not bucketed according to The preset rules divide the data to be inserted into at least one group. Perform sorting within the group according to the partition column field for each grouping.
  • the target table is a partitioned and bucketed table, as shown in Table 1.
  • FIG. 6 is a schematic structural diagram of a data insertion device provided by Embodiment 6 of the application.
  • the device can be configured in a database.
  • the device includes: an insertion action type determination module 610, a partition and bucket information determination module 620, and a data grouping module 630 , The data sorting module 640 and the data inserting module 650.
  • the insertion action type determining module 610 is configured to determine the insertion action type of the data insertion command according to the acquired data insertion command;
  • the partition bucket information determining module 620 is configured to determine the partition bucket information of the target table according to the meta information of the target table when the insert action type is dynamic insert;
  • the data grouping module 630 is configured to divide the data to be inserted into at least one group according to the partition and bucket information;
  • the data sorting module 640 is configured to sort the data to be inserted in each group according to the partition and bucket information
  • the data insertion module 650 is configured to dynamically insert each group of data to be inserted into the corresponding target table file in sequence according to the sequence of the data to be inserted in each group.
  • the technical solution of this embodiment solves the problem of system instability caused by operating a large number of files in different partitions at the same time for a long time by sorting the data to be inserted into the target table one by one, inserting into the target table one by one, and ensuring the stability of the system. Next, the effect of improving the performance of dynamic partition insertion.
  • the data grouping module 630 includes:
  • the first grouping unit is configured to divide the data to be inserted into at least one group according to the first grouping rule when the target table is a partitioned non-bucketed table.
  • the first grouping unit is set to:
  • the data to be inserted is divided into at least one group according to the amount of data to be inserted and the amount of the preset single group of data; wherein the amount of data to be inserted in each group is less than or equal to the amount of preset single group of data.
  • the first grouping unit is set to:
  • the group identification number includes a hash value of a preset field of the data to be inserted or a random number corresponding to the data to be inserted;
  • the data to be inserted is allocated to the corresponding array.
  • the data grouping module 630 includes:
  • the second grouping unit is set to group the data to be inserted according to the hash value of the bucketing column when the target table is a partitioned bucketing table.
  • the data sorting module 640 includes:
  • the first sorting unit is set to sort the data to be inserted in each group according to the order of the partitions of the target table when the target table is a multi-level partition table.
  • the data sorting module 640 includes:
  • the second sorting unit is set to sort the data to be inserted in each group according to the sequence after the target table partition field is mapped to the range partition of the target table when the target table is a multi-level range partition table.
  • the data insertion module 650 includes:
  • the first writing unit is set to read the first row in the current group, open the file handle corresponding to the first row as the current file handle, and write the data to be inserted in the first row into the current file handle;
  • the second writing unit is set to read the next row as the current row in sequence. If the current row belongs to the target table file corresponding to the current file handle, write the data to be inserted in the current row into the current file handle;
  • the third writing unit is set to close the current file handle if the current row does not belong to the target table file corresponding to the current file handle, open the file handle corresponding to the current row, and write the data to be inserted in the current row into the file handle corresponding to the current row ;
  • the insertion end unit of the group is set to end the insertion operation of the data to be inserted in the group when the next line cannot be read in the current group;
  • Full group insertion unit set to complete the insertion operation of all groups in a preset order.
  • the data insertion device provided in the embodiment of the present application can execute the data insertion method provided in any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method.
  • FIG. 7 is a schematic structural diagram of a device provided in Embodiment 7 of the application.
  • the device includes a processor 710, a memory 720, an input device 730, and an output device 740; the number of processors 710 in the device may be One or more, one processor 710 is taken as an example in FIG. 7; the processor 710, the memory 720, the input device 730, and the output device 740 in the device can be connected by a bus or other means. In FIG. 7, the connection by a bus is taken as an example .
  • the memory 720 can be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the data insertion method in the embodiment of the present application (for example, the insertion action in the data insertion device).
  • the processor 710 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 720, that is, realizes the aforementioned data insertion method.
  • the memory 720 may include a program storage area and a data storage area.
  • the program storage area may store an operating system and an application program required for at least one function; the data storage area may store data created according to the use of the terminal, and the like.
  • the memory 720 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
  • the memory 720 may include a memory remotely provided with respect to the processor 710, and these remote memories may be connected to the device through a network. Examples of the aforementioned networks include the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
  • the input device 730 may be used to receive inputted numeric or character information, and generate key signal input related to user settings and function control of the device.
  • the output device 740 may include a display device such as a display screen.
  • the eighth embodiment of the present application also provides a storage medium containing computer-executable instructions, which are used to execute a data insertion method when the computer-executable instructions are executed by a computer processor, and the method includes:
  • the insertion action type is dynamic partition insertion, determine the partition and bucket information of the target table according to the meta information of the target table;
  • each group of data to be inserted is dynamically inserted into the corresponding target table file in turn.
  • An embodiment of the present application provides a storage medium containing computer-executable instructions.
  • the computer-executable instructions include the method operations described above, and can also perform related operations in the data insertion method provided in any embodiment of the present application.
  • This application can be implemented with the help of software and necessary general-purpose hardware, or can be implemented with hardware.
  • the technical solution of this application can be embodied in the form of a software product.
  • the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), and random access memory ( Random Access Memory, flash memory (FLASH), hard disk or optical disk, etc., including at least one instruction to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the multiple embodiments of this application Methods.
  • the multiple units and modules included are only divided according to the functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the multiple functional units
  • the names are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé et un appareil d'insertion de données, ainsi qu'un dispositif et un support de stockage. Le procédé d'insertion de données consiste à : déterminer, en fonction d'une instruction d'insertion de données acquise, un type d'action d'insertion de l'instruction d'insertion de données ; dans la mesure où le type d'action d'insertion est une insertion de partition dynamique, déterminer, en fonction des méta-informations d'une table cible, des informations de partition et de division de compartiment de la table cible ; diviser, en fonction des informations de partition et de division de compartiment, des données qui doivent être insérées dans au moins un groupe ; trier lesdites données dans chaque groupe en fonction des informations de partition et de division de compartiment ; et en fonction de la séquence desdites données dans chaque groupe, insérer dynamiquement lesdites données dans chaque groupe dans un fichier de table cible correspondant en séquence.
PCT/CN2020/097563 2019-07-10 2020-06-23 Procédé et appareil d'insertion de données, dispositif et support de stockage WO2021004266A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910619873.3A CN110362577B (zh) 2019-07-10 2019-07-10 一种数据插入方法、装置、设备和储存介质
CN201910619873.3 2019-07-10

Publications (1)

Publication Number Publication Date
WO2021004266A1 true WO2021004266A1 (fr) 2021-01-14

Family

ID=68218660

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/097563 WO2021004266A1 (fr) 2019-07-10 2020-06-23 Procédé et appareil d'insertion de données, dispositif et support de stockage

Country Status (2)

Country Link
CN (1) CN110362577B (fr)
WO (1) WO2021004266A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114528053A (zh) * 2022-02-28 2022-05-24 上海识装信息科技有限公司 一种业务功能执行方法、装置、电子设备及存储介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110362577B (zh) * 2019-07-10 2020-06-09 星环信息科技（上海）有限公司 一种数据插入方法、装置、设备和储存介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105022763A (zh) * 2014-04-30 2015-11-04 博雅网络游戏开发（深圳）有限公司 实现数据查询的方法和系统
EP3098730A1 (fr) * 2015-05-29 2016-11-30 Sap Se Agrégation par hachage d'entrées de base de données
CN110362577A (zh) * 2019-07-10 2019-10-22 星环信息科技（上海）有限公司 一种数据插入方法、装置、设备和储存介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109241121A (zh) * 2017-06-29 2019-01-18 阿里巴巴集团控股有限公司 时间序列数据的存储和查询方法、装置、系统及电子设备
CN108446399B (zh) * 2018-03-29 2021-07-30 重庆大学 一种结构化海量实时数据的动态存储优化方法
CN109299190B (zh) * 2018-09-10 2020-11-17 华为技术有限公司 分布式存储系统中处理对象的元数据的方法及装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105022763A (zh) * 2014-04-30 2015-11-04 博雅网络游戏开发（深圳）有限公司 实现数据查询的方法和系统
EP3098730A1 (fr) * 2015-05-29 2016-11-30 Sap Se Agrégation par hachage d'entrées de base de données
CN110362577A (zh) * 2019-07-10 2019-10-22 星环信息科技（上海）有限公司 一种数据插入方法、装置、设备和储存介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
WANG, YUCONG ET AL.: "NTCI-Flow:A Scalable High-speed Network Traffic Processing Framework", ADVANCED ENGINEERING SCIENCES, no. A1, 30 April 2017 (2017-04-30), ISSN: 2096-3246, DOI: 20200921232540 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114528053A (zh) * 2022-02-28 2022-05-24 上海识装信息科技有限公司 一种业务功能执行方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN110362577B (zh) 2020-06-09
CN110362577A (zh) 2019-10-22

Similar Documents

Publication Publication Date Title
US20140351239A1 (en) Hardware acceleration for query operators
US10769147B2 (en) Batch data query method and apparatus
Parker et al. Comparing nosql mongodb to an sql db
CN104135437B (zh) 用于在网络交换机中利用集中式存储器池的表搜索的装置和方法
US9195701B2 (en) System and method for flexible distributed massively parallel processing (MPP) database
AU2014201593C1 (en) Shared cache used to provide zero copy memory mapped database
WO2015184762A1 (fr) Procédé et dispositif d'interrogation d'une base de données
US8380737B2 (en) Computing intersection of sets of numbers
US20160103858A1 (en) Data management system comprising a trie data structure, integrated circuits and methods therefor
WO2021004266A1 (fr) Procédé et appareil d'insertion de données, dispositif et support de stockage
Cossu et al. Prost: Distributed execution of sparql queries using mixed partitioning strategies
US10645090B2 (en) Access control for objects having attributes defined against hierarchically organized domains containing fixed number of values
US20120096054A1 (en) Reading rows from memory prior to reading rows from secondary storage
CN112015741A (zh) 一种海量数据的分库分表存储方法与装置
CN112579595A (zh) 数据处理方法、装置、电子设备及可读存储介质
CN105677687A (zh) 一种数据处理方法及装置
TW201926081A (zh) 資料轉移系統及方法
US10678789B2 (en) Batch data query method and apparatus
WO2016175880A1 (fr) Fusion de données d'entrée dans une base de données
CN111475511A (zh) 基于树状结构的数据存储方法、访问方法、装置及设备
CN108710640B (zh) 一种提高Spark SQL的查询效率的方法
CN113297266B (zh) 数据处理方法、装置、设备及计算机存储介质
CN106991116A (zh) 数据库执行计划的优化方法和装置
Lou et al. Research on data query optimization based on SparkSQL and MongoDB
CN111143456B (zh) 基于Spark的Cassandra数据导入方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20836655

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20836655

Country of ref document: EP

Kind code of ref document: A1