CN107273504A - Data query method and device based on Kudu - Google Patents
Data query method and device based on Kudu Download PDFInfo
- Publication number
- CN107273504A CN107273504A CN201710465025.2A CN201710465025A CN107273504A CN 107273504 A CN107273504 A CN 107273504A CN 201710465025 A CN201710465025 A CN 201710465025A CN 107273504 A CN107273504 A CN 107273504A
- Authority
- CN
- China
- Prior art keywords
- kudu
- data
- keyword
- current keyword
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012545 processing Methods 0.000 claims description 48
- 238000004891 communication Methods 0.000 claims description 22
- 238000006243 chemical reaction Methods 0.000 claims description 17
- 238000004458 analytical method Methods 0.000 claims description 8
- 230000001960 triggered effect Effects 0.000 claims description 3
- 239000002184 metal Substances 0.000 description 8
- 230000008569 process Effects 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000004883 computer application Methods 0.000 description 1
- 238000013501 data transformation Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a data query method and a data query device based on Kudu, wherein the method comprises the following steps: receiving SQL sentences corresponding to data to be queried; analyzing the SQL statement to obtain at least one current keyword and a variable value corresponding to each current keyword; constructing a syntax parsing tree according to the at least one obtained current keyword and the variable value corresponding to each current keyword; and querying the data to be queried stored in the Kudu according to the syntax parsing tree. By the technical scheme, the data stored in the Kudu can be queried through the SQL statement.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a data query method and device based on Kudu.
Background
With the continuous development of computer application technology, the amount of data required to be stored in a data center is continuously increased, and a Kudu distributed storage system is generally required to be deployed in the data center to store massive data. Kudu is a distributed storage system that includes a master server node and multiple slave server nodes, which relies on Hadoop (distributed system infrastructure).
Currently, Hadoop includes multiple components, with each component having a different function. When data to be queried stored in Kudu needs to be queried to achieve deletion, modification and other processing, multiple components such as HDFS and request need to be deployed at the same time to query the data stored in Kudu, and the data stored in Kudu cannot be queried through SQL statements.
Disclosure of Invention
The embodiment of the invention provides a data query method and device based on Kudu, which can realize the query of data stored in Kudu through SQL statements.
In a first aspect, the present invention provides a data query method based on Kudu, including:
receiving SQL sentences corresponding to data to be queried;
analyzing the SQL statement to obtain at least one current keyword and a variable value corresponding to each current keyword;
constructing a syntax parsing tree according to the at least one obtained current keyword and the variable value corresponding to each current keyword;
and querying the data to be queried stored in the Kudu according to the syntax parsing tree.
Preferably, the first and second electrodes are formed of a metal,
before the parsing the SQL statement, the method comprises the following steps: presetting a keyword comparison list, wherein at least one sample keyword is stored in the keyword comparison list;
then, the analyzing the SQL statement to obtain at least one current keyword and a variable value corresponding to each current keyword includes:
and identifying the SQL statement according to at least one sample keyword stored in the keyword comparison list so as to obtain at least one current keyword carried by the SQL statement and a variable value corresponding to each current keyword.
Preferably, the first and second electrodes are formed of a metal,
before the constructing a syntax parse tree according to the obtained at least one current keyword and the variable value corresponding to each current keyword, the method further includes:
presetting a grammar checking rule;
according to the grammar checking rule, checking the obtained at least one current keyword and the variable value corresponding to each current keyword respectively to determine whether the grammar of the SQL statement is qualified;
if yes, executing the variable values respectively corresponding to the at least one obtained current keyword and each current keyword to construct a syntax parse tree;
otherwise, generating error prompt information corresponding to the SQL statement.
Preferably, the first and second electrodes are formed of a metal,
before the querying the data to be queried stored in the Kudu according to the syntax parse tree, the method further includes:
acquiring a communication address of a main server node of the Kudu, and connecting the main server node of the Kudu according to the communication address;
then, the querying the data to be queried stored in the Kudu according to the syntax parse tree includes: and sending a data processing instruction to the master server node of the Kudu according to the syntax parsing tree, so that the master server node of the Kudu performs corresponding service processing on the data to be queried stored in at least one slave server node of the Kudu according to the data processing instruction.
Preferably, the first and second electrodes are formed of a metal,
before the querying the data to be queried stored in the Kudu according to the syntax parse tree, the method further includes: setting a data conversion rule;
the querying the data to be queried stored in the Kudu according to the syntax parse tree includes:
converting at least one object parameter in the syntax parsing tree according to the data conversion rule;
and querying the data to be queried stored in the Kudu according to the converted syntax parse tree.
In a second aspect, an embodiment of the present invention provides a data querying device based on Kudu, including:
the input module is used for receiving SQL sentences corresponding to the data to be queried;
the lexical analysis module is used for analyzing the SQL sentences to obtain at least one current keyword and a variable value corresponding to each current keyword;
the grammar processing module is used for constructing a grammar parsing tree according to the acquired at least one current keyword and the variable value corresponding to each current keyword;
and the query processing module is used for querying the data to be queried stored in the Kudu according to the syntax parse tree.
Preferably, the first and second electrodes are formed of a metal,
further comprising: a keyword comparison list; wherein,
the keyword comparison list is used for storing at least one sample keyword;
the lexical analysis module is configured to identify the SQL statement according to at least one sample keyword stored in the keyword comparison list, so as to obtain at least one current keyword carried by the SQL statement and a variable value corresponding to each current keyword.
Preferably, the first and second electrodes are formed of a metal,
further comprising: the system comprises a first setting module, a grammar checking module and a warning module; wherein,
the first setting module is used for presetting grammar checking rules;
the grammar checking module is used for checking the obtained at least one current keyword and the variable value corresponding to each current keyword according to the grammar checking rule so as to determine whether the grammar of the SQL statement is qualified or not, and if so, the grammar processing module is triggered; otherwise, triggering the warning module;
the grammar processing module is used for constructing a grammar parsing tree according to the acquired at least one current keyword and the variable value corresponding to each current keyword under the trigger of the grammar checking module;
and the warning module is used for generating error prompt information corresponding to the SQL statement under the triggering of the grammar checking module.
Preferably, the first and second electrodes are formed of a metal,
further comprising: a communication processing module; wherein,
the communication processing module is configured to acquire a communication address of the main server node of the Kudu, and connect the main server node of the Kudu according to the communication address;
the query processing module is configured to send a data processing instruction to the master server node of the Kudu according to the syntax parse tree, so that the master server node of the Kudu performs corresponding service processing on the data to be queried stored in at least one slave server node of the Kudu according to the data processing instruction.
Preferably, the first and second electrodes are formed of a metal,
further comprising: a second setting module; wherein,
the second setting module is used for setting a data conversion rule;
then, the query processing module is configured to perform conversion processing on at least one object parameter in the syntax parsing tree according to the data conversion rule; and querying the data to be queried stored in the Kudu according to the converted syntax parse tree.
The embodiment of the invention provides a data query method and a data query device based on Kudu, in the method, only SQL sentences corresponding to data to be queried need to be received, the received SQL sentences can be analyzed to obtain at least one current keyword, variable values corresponding to the current keywords are obtained at the same time, a syntax analysis tree capable of being identified by Kudu can be constructed subsequently according to the current keywords and the variable values corresponding to the current keywords, and the data to be queried stored in Kudu is queried according to the constructed syntax analysis tree. Therefore, the data stored in Kudu can be queried through the SQL statement.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a data query method based on Kudu according to an embodiment of the present invention;
fig. 2 is a flowchart of another Kudu-based data query method according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a Kudu-based data query apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of another Kudu-based data query apparatus according to an embodiment of the present invention
Fig. 5 is a schematic structural diagram of another Kudu-based data query apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention, and based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts belong to the scope of the present invention.
As shown in fig. 1, an embodiment of the present invention provides a data query method based on Kudu, including:
step 101, receiving an SQL statement corresponding to data to be queried;
102, analyzing the SQL statement to obtain at least one current keyword and a variable value corresponding to each current keyword;
103, constructing a syntax parse tree according to the acquired at least one current keyword and the variable value respectively corresponding to each current keyword;
and 104, inquiring the data to be inquired stored in the Kudu according to the syntax parse tree.
In the above embodiment of the present invention, only the SQL statements corresponding to the data to be queried need to be received, the received SQL statements may be parsed to obtain at least one current keyword, and at the same time, the variable values corresponding to each current keyword are obtained, and then, a syntax parse tree that can be identified by Kudu may be constructed according to each current keyword and the variable values corresponding to each current keyword, and the data to be queried stored in Kudu may be queried according to the constructed syntax parse tree. Therefore, the data stored in Kudu can be queried through the SQL statement.
In order to ensure that a specified keyword carried in an SQL statement can be identified, in an embodiment of the present invention, before parsing the SQL statement, the method includes: presetting a keyword comparison list, wherein at least one sample keyword is stored in the keyword comparison list;
then, the analyzing the SQL statement to obtain at least one current keyword and a variable value corresponding to each current keyword includes:
and identifying the SQL statement according to at least one sample keyword stored in the keyword comparison list so as to obtain at least one current keyword carried by the SQL statement and a variable value corresponding to each current keyword.
In the above embodiment of the present invention, by storing each specified keyword as a sample keyword in the keyword comparison list, the specified keywords such as query type, table name, field name, and the like can be stored as sample keywords in the keyword comparison list, and when analyzing the received SQL statement, each current keyword carried in the SQL statement and a variable value corresponding to each current keyword can be accurately read according to the keyword comparison list.
In order to prevent the SQL statements input by the user from forming an incorrect syntax parse tree due to syntax errors, and finally causing that the corresponding data to be queried cannot be accurately queried from Kudu, in an embodiment of the present invention, before constructing the syntax parse tree according to the at least one obtained current keyword and a variable value corresponding to each of the current keywords, the method further includes:
presetting a grammar checking rule;
according to the grammar checking rule, checking the obtained at least one current keyword and the variable value corresponding to each current keyword respectively to determine whether the grammar of the SQL statement is qualified;
if yes, executing the variable values respectively corresponding to the at least one obtained current keyword and each current keyword to construct a syntax parse tree;
otherwise, generating error prompt information corresponding to the SQL statement.
Specifically, in the above embodiment of the present invention, whether the syntax of the received SQL statement is qualified or not may be determined by checking whether at least one current keyword corresponds to each sample keyword stored in the keyword comparison list one to one, and whether a variable value corresponding to each current keyword is null, and if and only if the syntax of the received SQL statement is determined to be qualified, a syntax parse tree that can be identified by ku is constructed according to each current keyword and a variable value corresponding to each current keyword; correspondingly, when the received SQL statement syntax is judged to be unqualified, corresponding error prompt information can be generated, so that a user can input a more correct SQL statement according to the error prompt information.
Kudu usually comprises a master server node and a plurality of slave server nodes, data can be stored in each slave server node of Kudu, and each slave server node is connected with the master server of Kudu, so that the master server node can perform information interaction with the outside to realize management of the data stored in each slave server node through the master server node of Kudu.
Therefore, in an embodiment of the present invention, before querying the data to be queried stored in the Kudu according to the syntax parse tree, the method further includes:
acquiring a communication address of a main server node of the Kudu, and connecting the main server node of the Kudu according to the communication address;
then, the querying the data to be queried stored in the Kudu according to the syntax parse tree includes: and sending a data processing instruction to the master server node of the Kudu according to the syntax parsing tree, so that the master server node of the Kudu performs corresponding service processing on the data to be queried stored in at least one slave server node of the Kudu according to the data processing instruction.
In order to ensure that the data structure of the parse tree and each data object (including each current keyword and the variable value corresponding to each current keyword) of the parse tree can be identified by Kudu, in an embodiment of the present invention, before querying, according to the parse tree, data to be queried stored in Kudu, the method further includes: setting a data conversion rule;
the querying the data to be queried stored in the Kudu according to the syntax parse tree includes:
converting at least one object parameter in the syntax parsing tree according to the data conversion rule;
and querying the data to be queried stored in the Kudu according to the converted syntax parse tree.
In order to more clearly illustrate the technical solution of the present invention, taking Kudu as an example including a master server node a and slave server nodes B1 and B2, where data to be queried includes column 5 of a data table c1, and the data table c1 is stored in a slave server node B1, as shown in fig. 2, the data to be queried stored in Kudu can be specifically implemented through the following steps:
step 201, obtaining the communication address of the main server node a of Kudu, and connecting the main server node a of Kudu according to the obtained communication address.
Here, the communication address may be an IP (Internet Protocol) address of the master server node a, and after the IP address of the master server node a is acquired, communication may be performed with the master server node through the acquired IP address, for example, data in slave server nodes B1 and B2, which are bold to Kudu, is accessed through the master server node a.
Step 202, setting a keyword comparison list, grammar checking rules and data conversion rules.
Here, the user may store one or more specified keywords as sample keywords in the keyword collation list in association with actual business needs. The specified keywords include, but are not limited to: query type, table name, field name or column number, etc.
Step 203, receiving the SQL statement corresponding to the data to be queried, which is input by the user in combination with the actual business requirement.
And 204, identifying the SQL statement according to at least one sample keyword stored in the keyword comparison list so as to obtain at least one current keyword carried by the SQL statement and a variable value corresponding to each current keyword.
It should be understood that the obtained keywords include, but are not limited to, any one or more of query type, table name, field name, or column number, and the embodiment of the present invention only includes, with the obtained at least one current keyword: query type, table name, and column number are examples.
In the embodiment of the invention, when the SQL statement is analyzed, the variable values respectively corresponding to the query type, the table name and the column number can be simultaneously obtained according to the grammar rule of the SQL statement. Here, it is exemplified that the variable value corresponding to the query type is drop, the variable value corresponding to the table name is c1, and the variable values corresponding to the column numbers are 5 in this order.
Step 205, according to the set grammar checking rule, checking the obtained at least one current keyword and the variable value corresponding to each current keyword respectively to determine whether the grammar of the SQL statement is qualified, if so, executing step 206; otherwise, step 209 is performed.
In the embodiment of the invention, the grammar check rule can be reasonably set by combining with actual business requirements, including but not limited to checking whether each obtained current keyword corresponds to each sample keyword in the keyword comparison list one by one, and whether a variable value corresponding to each keyword is null, and if and only if each obtained current keyword corresponds to each sample keyword in the keyword comparison list one by one and the variable value corresponding to each keyword is not null, the grammar of the SQL statement is judged to be qualified.
And step 206, constructing a syntax parse tree according to the at least one current keyword and the variable value corresponding to each current keyword.
In the embodiment of the invention, when the SQL statement is analyzed, the obtained current keywords and the current keywords can exist in a text form, and the data in the text form cannot be identified by Kudu.
Specifically, the syntax parse tree may include a plurality of nodes, each node may be a current keyword, and a variable value corresponding to the current keyword is stored under the node where the current keyword is located.
And step 207, converting each data object under the syntax parsing tree according to a preset data conversion rule.
It should be understood that the data object may be a current keyword or a variable value.
For example, the syntax parsing tree includes a node as a query type, and a variable value stored under the node is drop, in the distributed storage system Kudu, the query type is characterized by a parameter X, and "delete" is characterized by the variable value drop in the SQL statement, and "delete" is characterized by a parameter value n in Kudu, which means that the two parameters cannot be identified, and therefore, a corresponding data transformation rule may be preset to replace the node "query type" in the syntax parsing tree with the node "X", and the parameter value n is stored under the node X, thereby ensuring that the data structure of the syntax parsing tree and each parameter and parameter value in the syntax parsing tree can be identified by Kudu.
And step 208, sending a data processing instruction to the main server node A of Kudu according to the converted syntax parse tree.
Here, the master server node a may identify the converted parse tree, and perform access to the data table c1 stored in the slave server node B1 according to the converted parse tree, and perform deletion processing on the data column with the corresponding column number of 5 in the data table c1, thereby implementing query of the data to be queried stored in Kudu by SQL statements.
Step 209 generates an error prompt message corresponding to the SQL statement.
Here, the user can know that the inputted SQL statement has a syntax error according to the generated error prompt information, and can modify or edit the more accurate SQL statement. The data to be queried stored in the Kudu can be conveniently and accurately queried according to the more accurate SQL sentences in the subsequent process.
As shown in fig. 3, an embodiment of the present invention provides a Kudu-based data query apparatus, including:
the input module 301 is configured to receive an SQL statement corresponding to data to be queried;
a lexical analysis module 302, configured to analyze the SQL statement to obtain at least one current keyword and a variable value corresponding to each current keyword;
a syntax processing module 303, configured to construct a syntax parse tree according to the obtained at least one current keyword and a variable value corresponding to each current keyword;
and the query processing module 304 is configured to query the data to be queried stored in the Kudu according to the syntax parse tree.
As shown in fig. 4, in an embodiment of the present invention, the apparatus further includes: a keyword comparison list 401; wherein,
the keyword comparison list 401 is used for storing at least one sample keyword;
then, the lexical analysis module 302 is configured to identify the SQL statement according to at least one sample keyword stored in the keyword comparison list, so as to obtain at least one current keyword carried by the SQL statement and a variable value corresponding to each current keyword.
As shown in fig. 4, in an embodiment of the present invention, the apparatus further includes: a first setting module 402, a grammar checking module 403 and an alert module 404; wherein,
the first setting module 402 is configured to preset a syntax checking rule;
the syntax checking module 403 is configured to check the obtained at least one current keyword and the variable value corresponding to each current keyword according to the syntax checking rule, to determine whether the syntax of the SQL statement is qualified, and if so, trigger the syntax processing module 303; otherwise, the alert module 404 is triggered;
the grammar processing module 303 is configured to construct a grammar parsing tree according to the at least one obtained current keyword and a variable value corresponding to each of the current keywords under the trigger of the grammar checking module;
the warning module 404 is configured to generate error prompt information corresponding to the SQL statement under the trigger of the syntax checking module.
As shown in fig. 5, in an embodiment of the present invention, the apparatus further includes: a communication processing module 501; wherein,
the communication processing module 501 is configured to obtain a communication address of the main server node of Kudu, and connect the main server node of Kudu according to the communication address;
the query processing module 304 is configured to send a data processing instruction to the master server node of Kudu according to the syntax parse tree, so that the master server node of Kudu performs corresponding service processing on the data to be queried stored in at least one slave server node of Kudu according to the data processing instruction.
In one embodiment of the present invention, the apparatus further comprises: a second setup module (not shown in the drawings); wherein,
the second setting module is used for setting a data conversion rule;
then, the query processing module is configured to perform conversion processing on at least one object parameter in the syntax parsing tree according to the data conversion rule; and querying the data to be queried stored in the Kudu according to the converted syntax parse tree.
Because the information interaction, execution process, and other contents between the units in the device are based on the same concept as the method embodiment of the present invention, specific contents may refer to the description in the method embodiment of the present invention, and are not described herein again.
An embodiment of the present invention provides a readable medium, which includes an execution instruction, and when a processor of a storage controller executes the execution instruction, the storage controller executes the method according to any one of claims 1 to 3.
An embodiment of the present invention provides a storage controller, including: a processor, a memory, and a bus;
the processor and the memory are connected through the bus;
the memory, the processor executing the execution instructions stored by the memory to cause the storage controller to perform the method of any of claims 1-3 when the storage controller is running.
In summary, the embodiments of the present invention have at least the following advantages:
1. in an embodiment of the present invention, only the SQL statements corresponding to the data to be queried need to be received, the received SQL statements may be parsed to obtain at least one current keyword, and at the same time, variable values corresponding to each current keyword are obtained, and then, a syntax parse tree that can be identified by Kudu may be constructed according to each current keyword and the variable values corresponding to each current keyword, and the data to be queried stored in Kudu may be queried according to the constructed syntax parse tree. Therefore, the data stored in Kudu can be queried through the SQL statement.
2. In an embodiment of the present invention, each specified keyword is stored in the keyword comparison list as a sample keyword, and the specified keywords such as a query type, a table name, a field name, and the like can be stored in the keyword comparison list as sample keywords, so that when analyzing a received SQL statement, each current keyword carried in the SQL statement and a variable value corresponding to each current keyword can be accurately read according to the keyword comparison list.
3. In one embodiment of the invention, whether the grammar of the received SQL statement is qualified or not is judged by checking whether at least one current keyword corresponds to each sample keyword stored in the keyword comparison list one by one and whether the variable value corresponding to each current keyword is empty or not, and if and only if the grammar of the received SQL statement is judged to be qualified, a grammar parsing tree which can be identified by Kudu is constructed according to each current keyword and the variable value corresponding to each current keyword, so that the SQL statement input by a user is prevented from forming an incorrect grammar parsing tree due to grammar errors, and finally corresponding data to be queried cannot be accurately queried from Kudu. Correspondingly, when the received SQL statement syntax is judged to be unqualified, corresponding error prompt information can be generated, so that a user can input a more correct SQL statement according to the error prompt information.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a" does not exclude the presence of other similar elements in a process, method, article, or apparatus that comprises the element.
Finally, it is to be noted that: the above description is only a preferred embodiment of the present invention, and is only used to illustrate the technical solutions of the present invention, and not to limit the protection scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (10)
1. A data query method based on Kudu is characterized by comprising the following steps:
receiving SQL sentences corresponding to data to be queried;
analyzing the SQL statement to obtain at least one current keyword and a variable value corresponding to each current keyword;
constructing a syntax parsing tree according to the at least one obtained current keyword and the variable value corresponding to each current keyword;
and querying the data to be queried stored in the Kudu according to the syntax parsing tree.
2. The data query method of claim 1,
before the parsing the SQL statement, the method comprises the following steps: presetting a keyword comparison list, wherein at least one sample keyword is stored in the keyword comparison list;
then, the analyzing the SQL statement to obtain at least one current keyword and a variable value corresponding to each current keyword includes:
and identifying the SQL statement according to at least one sample keyword stored in the keyword comparison list so as to obtain at least one current keyword carried by the SQL statement and a variable value corresponding to each current keyword.
3. The data query method of claim 2,
before the constructing a syntax parse tree according to the obtained at least one current keyword and the variable value corresponding to each current keyword, the method further includes:
presetting a grammar checking rule;
according to the grammar checking rule, checking the obtained at least one current keyword and the variable value corresponding to each current keyword respectively to determine whether the grammar of the SQL statement is qualified;
if yes, executing the variable values respectively corresponding to the at least one obtained current keyword and each current keyword to construct a syntax parse tree;
otherwise, generating error prompt information corresponding to the SQL statement.
4. The method of claim 1,
before the querying the data to be queried stored in the Kudu according to the syntax parse tree, the method further includes:
acquiring a communication address of a main server node of the Kudu, and connecting the main server node of the Kudu according to the communication address;
then, the querying the data to be queried stored in the Kudu according to the syntax parse tree includes: and sending a data processing instruction to the master server node of the Kudu according to the syntax parsing tree, so that the master server node of the Kudu performs corresponding service processing on the data to be queried stored in at least one slave server node of the Kudu according to the data processing instruction.
5. The data query method according to any one of claims 1 to 4,
before the querying the data to be queried stored in the Kudu according to the syntax parse tree, the method further includes: setting a data conversion rule;
the querying the data to be queried stored in the Kudu according to the syntax parse tree includes:
converting at least one object parameter in the syntax parsing tree according to the data conversion rule;
and querying the data to be queried stored in the Kudu according to the converted syntax parse tree.
6. A data inquiry device based on Kudu, characterized by comprising:
the input module is used for receiving SQL sentences corresponding to the data to be queried;
the lexical analysis module is used for analyzing the SQL sentences to obtain at least one current keyword and a variable value corresponding to each current keyword;
the grammar processing module is used for constructing a grammar parsing tree according to the acquired at least one current keyword and the variable value corresponding to each current keyword;
and the query processing module is used for querying the data to be queried stored in the Kudu according to the syntax parse tree.
7. The apparatus of claim 6,
further comprising: a keyword comparison list; wherein,
the keyword comparison list is used for storing at least one sample keyword;
the lexical analysis module is configured to identify the SQL statement according to at least one sample keyword stored in the keyword comparison list, so as to obtain at least one current keyword carried by the SQL statement and a variable value corresponding to each current keyword.
8. The apparatus of claim 7,
further comprising: the system comprises a first setting module, a grammar checking module and a warning module; wherein,
the first setting module is used for presetting grammar checking rules;
the grammar checking module is used for checking the obtained at least one current keyword and the variable value corresponding to each current keyword according to the grammar checking rule so as to determine whether the grammar of the SQL statement is qualified or not, and if so, the grammar processing module is triggered; otherwise, triggering the warning module;
the grammar processing module is used for constructing a grammar parsing tree according to the acquired at least one current keyword and the variable value corresponding to each current keyword under the trigger of the grammar checking module;
and the warning module is used for generating error prompt information corresponding to the SQL statement under the triggering of the grammar checking module.
9. The apparatus of claim 6,
further comprising: a communication processing module; wherein,
the communication processing module is configured to acquire a communication address of the main server node of the Kudu, and connect the main server node of the Kudu according to the communication address;
the query processing module is configured to send a data processing instruction to the master server node of the Kudu according to the syntax parse tree, so that the master server node of the Kudu performs corresponding service processing on the data to be queried stored in at least one slave server node of the Kudu according to the data processing instruction.
10. The apparatus according to any one of claims 6 to 9,
further comprising: a second setting module; wherein,
the second setting module is used for setting a data conversion rule;
then, the query processing module is configured to perform conversion processing on at least one object parameter in the syntax parsing tree according to the data conversion rule; and querying the data to be queried stored in the Kudu according to the converted syntax parse tree.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710465025.2A CN107273504A (en) | 2017-06-19 | 2017-06-19 | Data query method and device based on Kudu |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710465025.2A CN107273504A (en) | 2017-06-19 | 2017-06-19 | Data query method and device based on Kudu |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107273504A true CN107273504A (en) | 2017-10-20 |
Family
ID=60067877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710465025.2A Pending CN107273504A (en) | 2017-06-19 | 2017-06-19 | Data query method and device based on Kudu |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107273504A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109684331A (en) * | 2018-12-20 | 2019-04-26 | 浪潮软件股份有限公司 | A kind of object storage meta data management device and method based on Kudu |
| CN110795455A (en) * | 2019-09-06 | 2020-02-14 | 中国平安财产保险股份有限公司 | Dependency relationship analysis method, electronic device, computer device and readable storage medium |
| CN110795456A (en) * | 2019-10-28 | 2020-02-14 | 北京百度网讯科技有限公司 | Map query method and device, computer equipment and storage medium |
| CN110851678A (en) * | 2018-07-24 | 2020-02-28 | 北京京东金融科技控股有限公司 | Method and device for crawling data |
| CN111123888A (en) * | 2019-12-19 | 2020-05-08 | 江苏中天科技软件技术有限公司 | Industrial control protocol testing method and system, electronic equipment and storage medium |
| CN111562885A (en) * | 2020-04-30 | 2020-08-21 | 苏州亿歌网络科技有限公司 | Data processing method and device, computer equipment and storage medium |
| JP2021522570A (en) * | 2018-04-19 | 2021-08-30 | リスク マネジメント ソリューションズ， インコーポレイテッド | Data storage system to provide search query response with low latency |
| CN113986707A (en) * | 2021-11-03 | 2022-01-28 | 北京科杰科技有限公司 | Method for monitoring and controlling slow SQL based on big data kudu partition |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103761080A (en) * | 2013-12-25 | 2014-04-30 | 中国农业大学 | Structured query language (SQL) based MapReduce operation generating method and system |
| CN105302842A (en) * | 2014-07-31 | 2016-02-03 | 北大方正集团有限公司 | Data processing method and device |
| CN105335366A (en) * | 2014-05-30 | 2016-02-17 | 北大方正信息产业集团有限公司 | SQL statement processing method and apparatus and server |
| CN105989150A (en) * | 2015-03-02 | 2016-10-05 | 中国移动通信集团四川有限公司 | Data query method and device based on big data environment |
| CN106682147A (en) * | 2016-12-22 | 2017-05-17 | 北京锐安科技有限公司 | Mass data based query method and device |
-
2017
- 2017-06-19 CN CN201710465025.2A patent/CN107273504A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103761080A (en) * | 2013-12-25 | 2014-04-30 | 中国农业大学 | Structured query language (SQL) based MapReduce operation generating method and system |
| CN105335366A (en) * | 2014-05-30 | 2016-02-17 | 北大方正信息产业集团有限公司 | SQL statement processing method and apparatus and server |
| CN105302842A (en) * | 2014-07-31 | 2016-02-03 | 北大方正集团有限公司 | Data processing method and device |
| CN105989150A (en) * | 2015-03-02 | 2016-10-05 | 中国移动通信集团四川有限公司 | Data query method and device based on big data environment |
| CN106682147A (en) * | 2016-12-22 | 2017-05-17 | 北京锐安科技有限公司 | Mass data based query method and device |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2022088603A (en) * | 2018-04-19 | 2022-06-14 | リスク マネジメント ソリューションズ， インコーポレイテッド | Data storage system for providing low latency search query responses |
| JP7053885B2 (en) | 2018-04-19 | 2022-04-12 | リスク マネジメント ソリューションズ， インコーポレイテッド | Data storage system to provide search query response with low latency |
| JP2021522570A (en) * | 2018-04-19 | 2021-08-30 | リスク マネジメント ソリューションズ， インコーポレイテッド | Data storage system to provide search query response with low latency |
| CN110851678A (en) * | 2018-07-24 | 2020-02-28 | 北京京东金融科技控股有限公司 | Method and device for crawling data |
| CN110851678B (en) * | 2018-07-24 | 2024-02-02 | 京东科技控股股份有限公司 | Method and device for crawling data |
| CN109684331A (en) * | 2018-12-20 | 2019-04-26 | 浪潮软件股份有限公司 | A kind of object storage meta data management device and method based on Kudu |
| CN110795455A (en) * | 2019-09-06 | 2020-02-14 | 中国平安财产保险股份有限公司 | Dependency relationship analysis method, electronic device, computer device and readable storage medium |
| CN110795455B (en) * | 2019-09-06 | 2023-11-21 | 中国平安财产保险股份有限公司 | Dependency analysis method, electronic device, computer apparatus, and readable storage medium |
| CN110795456A (en) * | 2019-10-28 | 2020-02-14 | 北京百度网讯科技有限公司 | Map query method and device, computer equipment and storage medium |
| CN110795456B (en) * | 2019-10-28 | 2022-06-28 | 北京百度网讯科技有限公司 | Map query method and device, computer equipment and storage medium |
| CN111123888A (en) * | 2019-12-19 | 2020-05-08 | 江苏中天科技软件技术有限公司 | Industrial control protocol testing method and system, electronic equipment and storage medium |
| CN111562885A (en) * | 2020-04-30 | 2020-08-21 | 苏州亿歌网络科技有限公司 | Data processing method and device, computer equipment and storage medium |
| CN113986707B (en) * | 2021-11-03 | 2022-06-14 | 北京科杰科技有限公司 | Method for monitoring and controlling slow SQL based on big data kudu partition |
| CN113986707A (en) * | 2021-11-03 | 2022-01-28 | 北京科杰科技有限公司 | Method for monitoring and controlling slow SQL based on big data kudu partition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107273504A (en) | Data query method and device based on Kudu | |
| CN108090064B (en) | Data query method and device, data storage server and system | |
| CN110609844B (en) | Data updating method, device and system | |
| CN110109656B (en) | Interface simulation method, device, computer equipment and storage medium | |
| CN108958959B (en) | Method and device for detecting hive data table | |
| CN107480252A (en) | A kind of data query method, client, service end and system | |
| CN110351325B (en) | Data processing method and related equipment | |
| EP3217632B1 (en) | Oid configuration, parsing method, client, node, database and storage medium | |
| CN109189395B (en) | Data analysis method and device | |
| CN114553784A (en) | Current limiting processing method and device | |
| CN116483850A (en) | Data processing method, device, equipment and medium | |
| CN110716912B (en) | SQL performance detection method and server | |
| CN112597105B (en) | File association object processing method, server device and storage medium | |
| CN111078773B (en) | Data processing method and device | |
| CN117194389A (en) | SQL-based database compatibility method and device, electronic equipment and storage medium | |
| CN112491943B (en) | Data request method, device, storage medium and electronic equipment | |
| CN115357286B (en) | Program file comparison method and device, electronic equipment and storage medium | |
| CN112541001A (en) | Data query method, device, storage medium and equipment | |
| CN110955460A (en) | Service process starting method and device, electronic equipment and storage medium | |
| CN116382640A (en) | Method, device, equipment and storage medium for managing micro-service | |
| CN114764406B (en) | Database query method and related device | |
| CN113672776B (en) | Fault analysis method and device | |
| CN114490651A (en) | Data storage method and device | |
| CN115118583A (en) | Log query method and device, electronic equipment and computer readable storage equipment | |
| CN114840574A (en) | Data query method, server, node and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171020 |
|
| RJ01 | Rejection of invention patent application after publication |