CN116383192A

CN116383192A - Data query method, device, equipment and storage medium

Info

Publication number: CN116383192A
Application number: CN202111607541.7A
Authority: CN
Inventors: 苏军
Original assignee: 360 Digital Security Technology Group Co Ltd
Current assignee: 360 Digital Security Technology Group Co Ltd
Priority date: 2021-12-24
Filing date: 2021-12-24
Publication date: 2023-07-04

Abstract

The invention belongs to the technical field of computers, and discloses a data query method, a device, equipment and a storage medium. The invention analyzes the received association inquiry statement to determine a plurality of association data tables, table association fields and data inquiry conditions; acquiring cloth Long Suoyin data corresponding to table association fields in each association data table; filtering the data in the associated data table according to the data of the cloth Long Suoyin to obtain the data to be retrieved; screening the data to be retrieved according to the data query conditions to obtain target data corresponding to the associated query statement. Because the data in the associated data table is filtered according to the cloth Long Suoyin data corresponding to the associated field in the associated data table in the data query process, the data which does not meet the table association relationship is filtered in advance, the data quantity which needs to be processed in the subsequent data query is reduced, and the execution efficiency of the data query is improved.

Description

Data query method, device, equipment and storage medium

Technical Field

The present invention relates to the field of computer technologies, and in particular, to a data query method, apparatus, device, and storage medium.

Background

Today, high compression ratio and high read efficiency columnar files are used by many large data items, with the relatively common columnar file formats being orc and parquet. Since the column files generally include index information (such as the record number, the maximum value, the minimum value, whether null values and summation) of each column in the files, in the data query stage, a search engine generally accelerates the query efficiency by using the index information of the column files, but there is time and performance overhead for temporarily reading the index information of the column files during the execution of a search task, and as the data amount and the number of files increase, the overhead also increases, and as the data amount and the number of files increase, a time delay also exists in the current distributed file system, and the related query (the query is performed after the data in a plurality of data tables with related relationships) causes the data amount to be exponentially increased due to the specificity of the data association, and if the index files of all the column files are still read during the related query, the query efficiency is lower during the query.

The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.

Disclosure of Invention

The invention mainly aims to provide a data query method, a device, equipment and a storage medium, and aims to solve the technical problem of low query efficiency when related query is carried out on a column file in the prior art.

In order to achieve the above object, the present invention provides a data query method, which includes the steps of:

analyzing the received association inquiry statement, and determining a plurality of association data tables, table association fields and data inquiry conditions;

acquiring cloth Long Suoyin data corresponding to table association fields in each association data table;

filtering the data in the associated data table according to the cloth Long Suoyin data to obtain data to be retrieved;

and screening the data to be retrieved according to the data query conditions to obtain target data corresponding to the associated query statement.

Optionally, the step of obtaining the cloth Long Suoyin data corresponding to the table association field in each association data table includes:

determining a table index field according to the table association field and a preset index suffix;

and searching cloth Long Suoyin data corresponding to the table association fields in each association data table in a preset index database according to the association data table and the table index fields.

Optionally, the step of filtering the data in the association data table according to the data of the fabric Long Suoyin to obtain the data to be retrieved includes:

carrying out data association on the bloom index data corresponding to the table association fields in each association data table according to the table association rule to obtain a plurality of association index groups;

taking the associated index group which does not meet the preset data filtering condition as a target index group;

and filtering the data in the associated data table according to the target index group to obtain the data to be retrieved.

Optionally, before the step of filtering the data in the associated data table according to the target index group to obtain the data to be retrieved, the method further includes:

detecting whether the associated query condition corresponding to the table associated field exists in the data query conditions;

if yes, acquiring a query type corresponding to the associated query condition;

if the query type is a complete matching type, selecting an index group to be filtered from the associated index groups according to the associated query condition;

correspondingly, the step of filtering the data in the associated data table according to the target index group to obtain the data to be retrieved includes:

And filtering the data in the associated data table according to the target index group and the index group to be filtered to obtain the data to be retrieved.

Optionally, the step of selecting the index group to be filtered from the associated index groups according to the associated query condition includes:

acquiring a condition bloom value corresponding to the associated query condition;

comparing the cloth Long Suoyin data in the association index set to the conditional bloom value;

an associated index group that does not contain cloth Long Suoyin data that is consistent with the conditional bloom value is taken as the index group to be filtered.

Optionally, the step of filtering the data in the associated data table according to the target index group and the index group to be filtered to obtain the data to be retrieved includes:

filtering the target index group according to the index group to be filtered to obtain an index group to be retrieved;

and filtering the data in the associated data table according to the index group to be searched to obtain the data to be searched.

Optionally, before the step of taking the associated index group that does not meet the preset data filtering condition as the target index group, the method further includes:

performing AND operation on cloth Long Suoyin data in each associated index group to obtain merging index data corresponding to each associated index group;

Detecting whether the combined index data has a first type value;

if the corresponding merging index data does not have the first type value, judging that the associated index group corresponding to the merging index data meets the preset data filtering condition.

Optionally, before the step of analyzing the received association query statement to determine the plurality of association data tables, the table association fields and the data query conditions, the method further includes:

when a data storage request is received, determining data to be stored and a target data table according to the data storage request;

analyzing the data to be stored to obtain a data storage field in the data to be stored;

acquiring a preset common field set corresponding to the target data table and a data table identifier;

searching table index data corresponding to the target data table in a preset index database according to the data table identification;

selecting a target data field from the data storage fields according to the preset common field set;

acquiring a field bloom value corresponding to the target data field;

and updating the table index data according to the field bloom value, and storing the data to be stored in the target data table.

Optionally, the step of screening the data to be retrieved according to the data query condition to obtain target data corresponding to the associated query statement includes:

determining data screening conditions and data sorting conditions according to the data query conditions;

screening the data to be retrieved according to the data screening conditions to obtain temporary data;

and ordering the temporary data according to the data ordering condition to obtain target data corresponding to the associated query statement.

Optionally, before the step of obtaining the cloth Long Suoyin data corresponding to the table association field in each association data table, the method further includes:

detecting whether the table association field is a cloth Long Suoyin field;

if yes, executing the step of acquiring the cloth Long Suoyin data corresponding to the table association field in each association data table.

detecting a data table associated keyword in a data query statement when the data query statement is received;

and if the data query statement exists, taking the data query statement as an associated query statement.

In addition, in order to achieve the above object, the present invention also provides a data query device, which includes the following modules:

the data receiving module is used for analyzing the received association inquiry statement and determining a plurality of association data tables, table association fields and data inquiry conditions;

the data searching module is used for acquiring cloth Long Suoyin data corresponding to the table association fields in each association data table;

the data filtering module is used for filtering the data in the associated data table according to the cloth Long Suoyin data to obtain data to be retrieved;

and the data screening module is used for screening the data to be retrieved according to the data query conditions to obtain target data corresponding to the associated query statement.

Optionally, the data searching module is further configured to determine a table index field according to the table association field and a preset index suffix; and searching cloth Long Suoyin data corresponding to the table association fields in each association data table in a preset index database according to the association data table and the table index fields.

Optionally, the data filtering module is further configured to perform data association on bloom index data corresponding to the table association field in each association data table according to a table association rule, so as to obtain a plurality of association index groups; taking the associated index group which does not meet the preset data filtering condition as a target index group; and filtering the data in the associated data table according to the target index group to obtain the data to be retrieved.

Optionally, the data filtering module is further configured to detect whether an association query condition corresponding to the table association field exists in the data query conditions; if yes, acquiring a query type corresponding to the associated query condition; if the query type is a complete matching type, selecting an index group to be filtered from the associated index groups according to the associated query condition;

the data filtering module is further configured to filter data in the associated data table according to the target index group and the index group to be filtered, and obtain a filtered associated data table.

Optionally, the data filtering module is further configured to obtain a conditional bloom value corresponding to the associated query condition; comparing the cloth Long Suoyin data in the association index set to the conditional bloom value; an associated index group that does not contain cloth Long Suoyin data that is consistent with the conditional bloom value is taken as the index group to be filtered.

Optionally, the data filtering module is further configured to filter the target index set according to the index set to be filtered to obtain an index set to be retrieved; and filtering the data in the associated data table according to the index group to be searched to obtain the data to be searched.

Optionally, the data filtering module is further configured to perform an and operation on the fabric Long Suoyin data in each associated index group, so as to obtain combined index data corresponding to each associated index group; detecting whether the combined index data has a first type value; if the corresponding merging index data does not have the first type value, judging that the associated index group corresponding to the merging index data meets the preset data filtering condition.

In addition, to achieve the above object, the present invention also proposes a data query device including: a processor, a memory and a data query program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the data query method as described above.

In addition, in order to achieve the above object, the present invention also proposes a computer-readable storage medium having stored thereon a data query program which, when executed, implements the steps of the data query method as described above.

The invention analyzes the received association inquiry statement to determine a plurality of association data tables, table association fields and data inquiry conditions; acquiring cloth Long Suoyin data corresponding to table association fields in each association data table; filtering the data in the associated data table according to the data of the cloth Long Suoyin to obtain the data to be retrieved; screening the data to be retrieved according to the data query conditions to obtain target data corresponding to the associated query statement. Because the data in the associated data table is filtered according to the cloth Long Suoyin data corresponding to the associated field in the associated data table in the data query process, the data which does not meet the table association relationship is filtered in advance, the data quantity which needs to be processed in the subsequent data query is reduced, and the execution efficiency of the data query is improved.

Drawings

FIG. 1 is a schematic diagram of an electronic device of a hardware operating environment according to an embodiment of the present invention;

FIG. 2 is a flowchart of a first embodiment of a data query method according to the present invention;

FIG. 3 is a flowchart of a second embodiment of the data query method of the present invention;

FIG. 4 is a flowchart of a third embodiment of a data query method according to the present invention;

fig. 5 is a block diagram of a first embodiment of a data query device according to the present invention.

The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.

Detailed Description

It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

Referring to fig. 1, fig. 1 is a schematic diagram of a data query device structure of a hardware running environment according to an embodiment of the present invention.

As shown in fig. 1, the electronic device may include: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 1005 may be a high-speed random access Memory (Random Access Memory, RAM) or a stable nonvolatile Memory (NVM), such as a disk Memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.

Those skilled in the art will appreciate that the structure shown in fig. 1 is not limiting of the electronic device and may include more or fewer components than shown, or may combine certain components, or may be arranged in different components.

As shown in fig. 1, an operating system, a network communication module, a user interface module, and a data query program may be included in the memory 1005 as one type of storage medium.

In the electronic device shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present invention may be disposed in a data query device, where the electronic device invokes a data query program stored in the memory 1005 through the processor 1001, and executes a data query method provided by an embodiment of the present invention.

An embodiment of the present invention provides a data query method, referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the data query method of the present invention.

In this embodiment, the data query method includes the following steps:

step S10: and analyzing the received association query statement to determine a plurality of association data tables, table association fields and data query conditions.

It should be noted that, the execution body of the embodiment may be the data query device, and the data query device may be an electronic device such as a personal computer, a server, or other devices capable of implementing the same or similar functions.

The related query statement may be a query statement used when querying a plurality of related data tables, and the related query statement may be sent to the data query device by another device.

In practical use, the received association query statement is parsed to determine a plurality of key data tables, table association fields and data query conditions, which may be determined by splitting statement keywords in the association query statement.

For example: assuming that the related query statement is "select a.name, b.score from table 1A. Join table 2B. Name=b.name where a.age >18 and B.score>60", the statement keywords are "select", "from", "join", "on", "where" and the like, and the related data tables obtained after parsing the related query statement are table1 and table2, which are respectively represented by aliases a and B, the table related field is a name field, the table related condition is "a.name=b.name", and the data query condition is "a.age >18" and "b.score >60".

Further, since the data query device is a device for providing data query, which not only provides related query, but also may provide other query modes, the received query statement is not only one related query statement, and in order to distinguish related query statements, before step S10 in this embodiment, the method may further include:

It will be appreciated that if a data query statement is an associative query statement, then in order to specify an associative query table, there must be a data table associative key "join" in the data query statement, so it may be determined whether the data query statement is an associative query statement by detecting whether there is a data table associative key in the data query statement.

Step S20: cloth Long Suoyin data corresponding to the table association fields in each association data table is acquired.

It should be noted that, the acquiring of the bloom index data corresponding to the table association field in each association data table may be the searching of the bloom Long Suoyin data corresponding to the table association field in the preset index library. The preset index library may be a preset database for storing index information of the columnar file. The bloom index data may be index data obtained by calculating through the bloom Long Suanfa based on data in the columnar file.

Further, since the same field may exist in different data tables, but the table names of the data tables are unique, the index data may be distinguished by combining the table names of the data tables, and then the query is also required by combining the table names and the field names when querying the data of the fabric Long Suoyin, so step S20 in this embodiment may include:

It should be noted that, the preset index suffix may be preset by a manager of the data query device according to actual needs. Determining the table index field according to the table association field and the preset index suffix may be to splice the field name of the table association field with the preset index suffix to obtain the index field name, thereby determining the table index field. For example: assuming that the field name of the table association field is name and the preset index suffix is "_item_bf", the table index field name is "name_item_bf" at this time.

In practical use, the searching the bloom index data corresponding to the table association field in the association data table according to the association data table and the table index field in the preset index database may use the table name of the association data table and the field name of the table index field together as a query condition, and searching the cloth Long Suoyin data corresponding to the table association field in the association data table in the preset index database according to the query condition.

Further, in order to ensure that the data query method of the present embodiment can operate normally, before step S20, the method may further include:

detecting whether the table association field is a cloth Long Suoyin field;

It can be understood that, in the preset index database, only the cloth Long Suoyin data corresponding to the field of the cloth Long Suoyin is stored, the other fields all store common index data, the subsequent scheme is operated to perform file filtering through the cloth Long Suoyin, and the subsequent step performed on the common index data may cause filtering errors, so that the data query is abnormal, in order to avoid such a situation, whether the associated field of the table is the field of the cloth Long Suoyin may be detected in advance, if yes, the cloth Long Suoyin data corresponding to the associated field of the table in each associated data table is obtained, and if not, the query is directly performed in a common mode.

In actual use, one fabric Long Zi section table may be used to record the fabric Long Suoyin fields corresponding to each data table, and then it is determined whether the table association field is the fabric Long Suoyin field or not, and it may be determined whether the table association field belongs to the fabric Long Zi section table, if so, it is determined that the table association field is the fabric Long Suoyin field; if not, the decision table association field is not the fabric Long Suoyin field. Of course, it is also possible to distinguish whether one field is the cloth Long Suoyin field by the data type corresponding to the field, for example: if the data corresponding to one field is a character string type, the field is determined to be a cloth Long Suoyin field.

Further, in order to increase the execution efficiency of the data query as much as possible, before step S10 in this embodiment, the method may further include:

acquiring a field bloom value corresponding to the target data field;

It should be noted that, the data to be stored may be a column file that needs to be stored. The target data table may be a data table that the user specifies to store the columnar file. The data storage request may be a request sent by a user to a data querying device when the columnar file needs to be stored. The determining of the data to be stored and the target data table according to the data storage request may be reading the data to be stored and the target data table identifier from the data storage request, and determining the target data table according to the target data table identifier, where the target data table identifier may be a table name of the target data table. The obtaining the preset common field set and the data table identifier corresponding to the target data table may be according to the data table identifier of the target data table, searching the corresponding preset common field table according to the data table identifier, and constructing the preset common field set according to the preset common field in the preset common field table.

In practical use, selecting the target data field from the data storage fields according to the preset common field set may be selecting the target data field from the data storage fields belonging to the preset common field set. The field bloom value corresponding to the acquisition target data field may be the acquisition target data corresponding to the acquisition target data field, and performing Long Yunsuan on the target data field through a preset cloth Long Suanfa to obtain a field bloom value. Updating the table index data according to the field bloom value may be to obtain a data storage field corresponding to the field bloom value, read the cloth Long Suoyin data corresponding to the data storage field in the table index data, and update the cloth Long Suoyin data according to the field cloth Long Zhi.

It can be understood that when the column files are stored, the field bloom value corresponding to the target data field is calculated, and then the table index data is updated through the field bloom value, so that the preset index database can be ensured to store the cloth Long Suoyin data corresponding to all the column files stored in each data table, the index information in the column files is not required to be read when the data in the data tables are filtered, the cloth Long Suoyin data is directly read from the preset index database, and the performance cost when the column files are analyzed can be reduced, thereby improving the execution efficiency of the data query.

Step S30: and filtering the data in the associated data table according to the data of the cloth Long Suoyin to obtain the data to be retrieved.

It should be noted that, filtering the data in the associated data table according to the data of the fabric Long Suoyin to obtain the data to be retrieved may be to clear the data which is determined to obviously not conform to the table association condition according to the data of the fabric Long Suoyin, and then associate the remaining data with the table data according to the table association relationship, so as to obtain the data to be retrieved. For example: assuming that the association data table is a and B, where a includes three columnar files, i.e. file1, file2 and file3, and B includes two columnar files, i.e. file4 and file5, and it is determined according to the data of the fabric Long Suoyin that file2 and file5 obviously do not conform to the table association condition, at this time, file1 and file3 may be associated with file4 respectively, so as to obtain two pieces of data to be retrieved, i.e. "file1-file4", "file3-file4", respectively.

Step S40: and screening the data to be retrieved according to the data query conditions to obtain target data corresponding to the associated query statement.

It should be noted that, filtering the data to be searched according to the data query condition, obtaining the target data corresponding to the associated query statement may be reading index information corresponding to the data to be searched, filtering the data to be searched according to the index information and the data query condition, removing the data which does not meet the data query condition in the data to be searched, and finally obtaining the target data corresponding to the associated query statement.

In a specific implementation, in order to make the finally queried data display orderly, so as to improve the use experience of the user, step S40 in this embodiment may include:

It should be noted that, determining the data sorting condition according to the data query condition may be to detect whether the sorting field and the sorting keyword exist in the data query condition, if so, constructing the data sorting condition according to the sorting field and the sorting keyword, and if not, acquiring the default sorting field and the default sorting keyword, and constructing the data sorting condition according to the default sorting field and the default sorting keyword. The determining of the data filtering condition according to the data query condition may be reading index information of the data to be searched, determining a partial condition of the data query condition, which can utilize the index information, converting the partial condition into the index searching condition, and then constructing the data filtering condition according to the partial condition of the data query condition, which cannot utilize the index information, and the index searching condition.

It can be understood that the data to be retrieved is screened according to the data screening condition, so that the data required by the user can be obtained, and then the temporary data obtained by screening is ordered according to the data ordering condition, so that the finally obtained data is ordered, the user can check conveniently, and the user experience is improved.

In the embodiment, a plurality of associated data tables, table associated fields and data query conditions are determined by analyzing the received associated query statement; acquiring cloth Long Suoyin data corresponding to table association fields in each association data table; filtering the data in the associated data table according to the data of the cloth Long Suoyin to obtain the data to be retrieved; screening the data to be retrieved according to the data query conditions to obtain target data corresponding to the associated query statement. Because the data in the associated data table is filtered according to the cloth Long Suoyin data corresponding to the associated field in the associated data table in the data query process, the data which does not meet the table association relationship is filtered in advance, the data quantity which needs to be processed in the subsequent data query is reduced, and the execution efficiency of the data query is improved.

Referring to fig. 3, fig. 3 is a flowchart of a second embodiment of a data query method according to the present invention.

Based on the above-mentioned first embodiment, the step S30 of the data query method of this embodiment includes:

step S301: and carrying out data association on the bloom index data corresponding to the table association fields in each association data table according to the table association rule to obtain a plurality of association index groups.

The table association rule may be a rule to be observed when table data having an association relationship is associated, and is generally associated by a cartesian product method. The data association is carried out on the bloom index data corresponding to the table association field in each association data table according to the table association rule, so that a plurality of association index groups are obtained, namely, the data identifications which are required to be associated are calculated according to the Cartesian product mode, and then the data of the cloth Long Suoyin corresponding to the data identifications which are required to be associated are combined into a group.

For example: assuming that the association data table includes a and B, where a includes three column file data with data identifiers of file1, file2 and file3, corresponding fabric Long Suoyin data is "010000", "100010" and "01100000", and B includes two column file data with data identifiers of file4 and file5, respectively, "000001" and "010100", corresponding fabric Long Suoyin data is associated in a cartesian product manner to obtain 6 groups of data, and the data identifiers may be "file1-file4", "file2-file4", "file3-file4", "file1-file5", "file2-file5", "file3-file5", and the obtained association index groups are also 6 groups, respectively, "010000-000001", "100010-000001", "011000-000001", "010000-010100", "100010-010100", and "01110-010100".

Step S302: and taking the associated index group which does not meet the preset data filtering condition as a target index group.

It should be noted that, the preset data filtering condition may be preset by a manager of the data query device, if an association index group satisfies the preset data filtering condition, it indicates that there is no association part of the plurality of bloom index data in the association index group, then it may be determined that data in the association data table corresponding to the association index group does not satisfy the table association condition, and data corresponding to the association index group may be cleared, so that the association index group satisfying the preset data filtering condition may be removed, and the association index group not satisfying the preset data filtering condition may be taken as the target index group.

Further, in order to accurately determine whether the association index set meets the preset data filtering condition, before step S302 in this embodiment, the method may further include:

detecting whether the combined index data has a first type value;

It should be noted that the and (≡) operation may be whether the values on the same bit are all 1, if so, the bit value is set to 1, and if not, the bit value is set to 0, i.e., 0&1 =0, 1& 1=1, 0& 0=0, 1& 0=0. The first type has a value of 1. And performing an and operation on the fabric Long Suoyin data in the association index group, to obtain the merging index data corresponding to the association index group may be performing an and operation on each bit number of the fabric Long Suoyin data in the association index group, so as to obtain the merging index data corresponding to the association index group. For example: assuming that the cloth Long Suoyin data included in the association index group is "010000" and "010100", respectively, the merge index data obtained at this time is "010000".

It should be noted that, if the first type value does not exist in the merging index data generated after the two pieces of cloth Long Suoyin data are subjected to the and operation, it is indicated that the data of the two bloom units have no association relationship, so that it can be determined that the association index group corresponding to the merging index data meets the preset data filtering condition.

Step S303: and filtering the data in the associated data table according to the target index group to obtain the data to be retrieved.

It should be noted that, filtering the data in the associated data table according to the target index group, obtaining the data to be retrieved may be obtaining the data identifier corresponding to the data of the fabric Long Suoyin in the target index group, filtering the data in the associated data table unrelated to the data identifier, and then associating the remaining data in the associated data table according to the grouping manner of the target index group, so as to obtain the data to be retrieved. For example: assuming that the association data table includes a and B, wherein a includes three column-type file data with data identifiers of file1, file2 and file3 respectively, corresponding cloth Long Suoyin data is "010000", "100010", "01100000", and B includes two column-type file data with data identifiers of file4 and file5 respectively, corresponding cloth Long Suoyin data is "000001", "010100", and an association index group is 6 groups, which are "010000-000001", "100010-000001", "01110-000001", "010000-010100", "100010-010100", "01110-010100", and a target association group is "010000-010100", then column-type file data in the data association table unrelated to the data identifiers of the target association group is "file 2 and file 4", and then the remaining file1, file3 and file5 can be associated in a grouping manner of the target association group, so as to obtain the to-retrieve data "file 1-5" and "file 5-3".

In the embodiment, data association is carried out on bloom index data corresponding to table association fields in each association data table according to table association rules, so that a plurality of association index groups are obtained; taking the associated index group which does not meet the preset data filtering condition as a target index group; and filtering the data in the associated data table according to the target index group to obtain the data to be retrieved. Because the data of the cloth Long Suoyin are subjected to data association in advance according to the table association rule to obtain a plurality of association index groups, whether the data corresponding to the association index groups are filtered is determined by judging whether the association index groups meet preset data filtering conditions, and therefore data which do not meet the table association conditions in an association data table are accurately filtered, and the data quantity required to be processed by subsequent inquiry is reduced.

Referring to fig. 4, fig. 4 is a flowchart of a third embodiment of a data query method according to the present invention.

Based on the foregoing fifth embodiment, before the step S303 of the data query method of the present embodiment, the method further includes:

step S3021: and detecting whether the associated query condition corresponding to the table associated field exists in the data query conditions.

It should be noted that the association query condition may be a query condition related to a table association field. For example: assuming that the data query conditions are "a.age >18", "a.name= ' Zhang Sanza" and "b.score >60", and the table association field is "name", then the data query condition "a.name= ' Zhang Sanza '" is the association query condition corresponding to the table association field.

Step S3022: if so, acquiring the query type corresponding to the associated query condition.

It should be noted that, query types may be classified into a complete match type, a partial match type, and a range match type, for example: if the data query condition is "a.name=' Zhang Sanzhen", the data query condition is matched in the exactly same way, so that the corresponding query type is the exactly matched type; if the data query condition is "A.name like' Zhongzhen", the data with the query name of Zhangbeginning is indicated, and the corresponding query type is a partial matching type; if the data query condition is "a.age >18", the data within a range with the query age greater than 18 is queried, and the corresponding query type is a range matching type.

Step S3023: and if the query type is the complete matching type, selecting an index group to be filtered from the associated index groups according to the associated query condition.

It should be noted that, if the query type is a perfect match type, the cloth Long Suoyin corresponding to the field in the data meeting the association query condition should be equal to the cloth Long Zhi corresponding to the query value used in the association query condition, and the data may be further filtered by the association query condition, so that the index group to be filtered may be selected from the association index groups according to the association query condition.

In a specific implementation, the step of selecting the index group to be filtered from the associated index groups according to the associated query condition may include:

It should be noted that, the acquiring the conditional bloom value corresponding to the associated query condition may be acquiring the conditional value corresponding to the associated query condition, and then calculating the conditional bloom value corresponding to the conditional value through the preset fabric Long Suanfa.

It can be understood that if the set of associated index groups does not include the data of the fabric Long Suoyin consistent with the bloom value, the data corresponding to the associated index groups may not satisfy the table association condition, but may not necessarily satisfy the association query condition, and the data is necessarily not actually required by the user, so that the data may be used as the index groups to be filtered, so that the data corresponding to the index groups to be filtered can be filtered, the number of the column file data to be read in the subsequent query process is reduced, and thus the query efficiency is improved.

Accordingly, the step S303 includes:

step S303': and filtering the data in the associated data table according to the target index group and the index group to be filtered to obtain the data to be retrieved.

It should be noted that, the filtering the data in the associated data table according to the target index group and the index group to be filtered may be performed after the filtering the data in the associated data table by the index group to be over-rate, and the secondary filtering the filtered data by the index group to be filtered, so as to obtain the data to be retrieved.

Further, if the data is filtered after being associated and combined, the data query efficiency may be improved to a certain extent, but unnecessary data is wasted in combination with unnecessary performance, and in order to reduce unnecessary performance consumption, the step S303' in this embodiment may include:

It should be noted that, filtering the target index group according to the index group to be filtered, and obtaining the index group to be retrieved may be to clear the associated index group in the target index group that is the same as the index group to be filtered, so as to obtain the index group to be retrieved.

It can be understood that the target index group is filtered according to the index group to be filtered to obtain the power search index group, and then the data in the associated data table is filtered according to the index group to be searched to obtain the data to be searched, so that the filtering after the data association combination can be avoided, the performance loss of the data association can be reduced, and the efficiency of data query is further improved.

In the embodiment, whether the associated query condition corresponding to the table associated field exists in the data query condition is detected; if yes, acquiring a query type corresponding to the associated query condition; if the query type is a complete matching type, selecting an index group to be filtered from the associated index groups according to the associated query condition; and filtering the data in the associated data table according to the target index group and the index group to be filtered to obtain the data to be retrieved. Because the data in the associated data table is further filtered according to the associated query condition that the query type is the complete matching type before the data in the associated data table is filtered, the data volume required to be processed by the subsequent query is further reduced, and therefore the execution efficiency of the data query is further improved.

In addition, the embodiment of the invention also provides a storage medium, wherein the storage medium stores a data query program, and the data query program realizes the steps of the data query method when being executed by a processor.

Referring to fig. 5, fig. 5 is a block diagram illustrating a first embodiment of a data query device according to the present invention.

As shown in fig. 5, a data query device according to an embodiment of the present invention includes:

the data receiving module 10 is configured to parse the received association query statement to determine a plurality of association data tables, table association fields and data query conditions;

the data searching module 20 is configured to obtain cloth Long Suoyin data corresponding to the table association fields in each association data table;

the data filtering module 30 is configured to perform filtering processing on the data in the association data table according to the data of the fabric Long Suoyin, so as to obtain data to be retrieved;

and the data screening module 40 is configured to screen the data to be retrieved according to the data query condition, so as to obtain target data corresponding to the associated query statement.

Further, the data searching module 20 is further configured to determine a table index field according to the table association field and a preset index suffix; and searching cloth Long Suoyin data corresponding to the table association fields in each association data table in a preset index database according to the association data table and the table index fields.

Further, the data filtering module 30 is further configured to perform data association on the bloom index data corresponding to the table association field in each association data table according to the table association rule, so as to obtain a plurality of association index groups; taking the associated index group which does not meet the preset data filtering condition as a target index group; and filtering the data in the associated data table according to the target index group to obtain the data to be retrieved.

Further, the data filtering module 30 is further configured to detect whether an association query condition corresponding to the table association field exists in the data query conditions; if yes, acquiring a query type corresponding to the associated query condition; if the query type is a complete matching type, selecting an index group to be filtered from the associated index groups according to the associated query condition;

the data filtering module 30 is further configured to filter data in the associated data table according to the target index set and the index set to be filtered, so as to obtain a filtered associated data table.

Further, the data filtering module 30 is further configured to obtain a conditional bloom value corresponding to the associated query condition; comparing the cloth Long Suoyin data in the association index set to the conditional bloom value; an associated index group that does not contain cloth Long Suoyin data that is consistent with the conditional bloom value is taken as the index group to be filtered.

Further, the data filtering module 30 is further configured to filter the target index set according to the index set to be filtered to obtain an index set to be retrieved; and filtering the data in the associated data table according to the index group to be searched to obtain the data to be searched.

Further, the data filtering module 30 is further configured to perform an and operation on the fabric Long Suoyin data in each associated index group, so as to obtain combined index data corresponding to each associated index group; detecting whether the combined index data has a first type value; if the corresponding merging index data does not have the first type value, judging that the associated index group corresponding to the merging index data meets the preset data filtering condition.

Further, the data receiving module 10 is further configured to determine, when receiving a data storage request, data to be stored and a target data table according to the data storage request; analyzing the data to be stored to obtain a data storage field in the data to be stored; acquiring a preset common field set corresponding to the target data table and a data table identifier; searching table index data corresponding to the target data table in a preset index database according to the data table identification; selecting a target data field from the data storage fields according to the preset common field set; acquiring a field bloom value corresponding to the target data field; and updating the table index data according to the field bloom value, and storing the data to be stored in the target data table.

Further, the data filtering module 40 is further configured to determine a data filtering condition and a data sorting condition according to the data query condition; screening the data to be retrieved according to the data screening conditions to obtain temporary data; and ordering the temporary data according to the data ordering condition to obtain target data corresponding to the associated query statement.

Further, the data lookup module 20 is further configured to detect whether the table association field is a fabric Long Suoyin field; if so, cloth Long Suoyin data corresponding to the table association field in each association data table is acquired.

Further, the data receiving module 10 is further configured to detect a data table associated keyword in a data query statement when the data query statement is received; and if the data query statement exists, taking the data query statement as an associated query statement.

It should be understood that the foregoing is illustrative only and is not limiting, and that in specific applications, those skilled in the art may set the invention as desired, and the invention is not limited thereto.

It should be noted that the above-described working procedure is merely illustrative, and does not limit the scope of the present invention, and in practical application, a person skilled in the art may select part or all of them according to actual needs to achieve the purpose of the embodiment, which is not limited herein.

In addition, technical details not described in detail in this embodiment may refer to the data query method provided in any embodiment of the present invention, which is not described herein again.

Furthermore, it should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.

The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. Read Only Memory)/RAM, magnetic disk, optical disk) and including several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.

The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.

The invention discloses A1, a data query method, which comprises the following steps:

A2, the data query method as described in A1, wherein the step of obtaining the cloth Long Suoyin data corresponding to the table association fields in each association data table comprises:

A3, the data query method of A1, the step of filtering the data in the associated data table according to the data of the cloth Long Suoyin to obtain the data to be retrieved, includes:

A4, the data query method according to A3, wherein before the step of filtering the data in the associated data table according to the target index group to obtain the data to be retrieved, further includes:

if yes, acquiring a query type corresponding to the associated query condition;

A5, the data query method as described in A4, wherein the step of selecting the index group to be filtered from the associated index groups according to the associated query condition includes:

A6, the data query method according to A4, wherein the step of filtering the data in the associated data table according to the target index group and the index group to be filtered to obtain the data to be retrieved includes:

A7, the data query method as described in A3, wherein before the step of using the associated index group that does not meet the preset data filtering condition as the target index group, further comprises:

detecting whether the combined index data has a first type value;

A8, the data query method according to any one of A1-A7, wherein before the step of analyzing the received association query statement to determine a plurality of association data tables, table association fields and data query conditions, the method further comprises:

acquiring a field bloom value corresponding to the target data field;

A9, the data query method according to any one of A1-A7, the step of screening the data to be retrieved according to the data query condition to obtain target data corresponding to the associated query statement, includes:

A10, the data query method according to any one of A1-A7, before the step of obtaining the fabric Long Suoyin data corresponding to the table association field in each association data table, further comprises:

detecting whether the table association field is a cloth Long Suoyin field;

A11, the data query method according to any one of A1-A7, wherein before the step of analyzing the received association query statement to determine a plurality of association data tables, table association fields and data query conditions, the method further comprises:

The invention also discloses a B12 and a data query device, wherein the data query device comprises the following modules:

B13, the data query device as described in B12, wherein the data search module is further configured to determine a table index field according to the table association field and a preset index suffix; and searching cloth Long Suoyin data corresponding to the table association fields in each association data table in a preset index database according to the association data table and the table index fields.

B14, the data query device as described in B12, the data filtering module is further configured to perform data association on bloom index data corresponding to table association fields in each association data table according to a table association rule, so as to obtain a plurality of association index groups; taking the associated index group which does not meet the preset data filtering condition as a target index group; and filtering the data in the associated data table according to the target index group to obtain the data to be retrieved.

B15, the data query device as described in B14, wherein the data filtering module is further configured to detect whether an association query condition corresponding to the table association field exists in the data query conditions; if yes, acquiring a query type corresponding to the associated query condition; if the query type is a complete matching type, selecting an index group to be filtered from the associated index groups according to the associated query condition;

The data query device as described in B16, wherein the data filtering module is further configured to obtain a conditional bloom value corresponding to the associated query condition; comparing the cloth Long Suoyin data in the association index set to the conditional bloom value; an associated index group that does not contain cloth Long Suoyin data that is consistent with the conditional bloom value is taken as the index group to be filtered.

B17, the data query device of B16, the said data filter module, is used for also filtering the said goal index group according to the index group to be filtered, obtain the index group to be retrieved; and filtering the data in the associated data table according to the index group to be searched to obtain the data to be searched.

B18, the data query device as described in B14, wherein the data filtering module is further configured to perform an AND operation on the fabric Long Suoyin data in each associated index group to obtain merged index data corresponding to each associated index group; detecting whether the combined index data has a first type value; if the corresponding merging index data does not have the first type value, judging that the associated index group corresponding to the merging index data meets the preset data filtering condition.

The invention also discloses C19, a data query device, the data query device includes: a processor, a memory and a data query program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the data query method as described above.

The invention also discloses D20 and a computer readable storage medium, wherein the computer readable storage medium stores a data query program, and the data query program realizes the steps of the data query method when being executed.

Claims

1. A data query method, characterized in that the data query method comprises the steps of:

2. The data query method as claimed in claim 1, wherein the step of obtaining the cloth Long Suoyin data corresponding to the table association field in each association data table comprises:

3. The data query method of claim 1, wherein the step of filtering the data in the associated data table according to the data of the fabric Long Suoyin to obtain the data to be retrieved comprises:

4. The data query method as claimed in claim 3, wherein before the step of filtering the data in the associated data table according to the target index group to obtain the data to be retrieved, the method further comprises:

if yes, acquiring a query type corresponding to the associated query condition;

5. The data query method of claim 4, wherein the selecting the index group to be filtered from the associated index groups according to the associated query condition comprises:

6. The method of claim 4, wherein the step of filtering the data in the associated data table according to the target index group and the index group to be filtered to obtain the data to be retrieved comprises:

7. The data query method as claimed in claim 3, wherein before the step of taking the associated index group that does not satisfy the preset data filtering condition as the target index group, further comprising:

detecting whether the combined index data has a first type value;

8. A data query device, comprising the following modules:

9. A data query device, the data query device comprising: a processor, a memory and a data query program stored on the memory and executable on the processor, the data query program when executed by the processor implementing the steps of the data query method of any of claims 1 to 7.

10. A computer-readable storage medium, wherein a data query program is stored on the computer-readable storage medium, which when executed implements the steps of the data query method of any of claims 1-7.