CN113760949A - Data query method and device - Google Patents

Data query method and device Download PDF

Info

Publication number
CN113760949A
CN113760949A CN202110251343.5A CN202110251343A CN113760949A CN 113760949 A CN113760949 A CN 113760949A CN 202110251343 A CN202110251343 A CN 202110251343A CN 113760949 A CN113760949 A CN 113760949A
Authority
CN
China
Prior art keywords
instruction
hit
sql statement
hit instruction
query
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110251343.5A
Other languages
Chinese (zh)
Other versions
CN113760949B (en
Inventor
郭庆海
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Zhenshi Information Technology Co Ltd
Original Assignee
Beijing Jingdong Zhenshi Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Zhenshi Information Technology Co Ltd filed Critical Beijing Jingdong Zhenshi Information Technology Co Ltd
Priority to CN202110251343.5A priority Critical patent/CN113760949B/en
Publication of CN113760949A publication Critical patent/CN113760949A/en
Application granted granted Critical
Publication of CN113760949B publication Critical patent/CN113760949B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/242Query formulation
    • G06F16/2433Query languages
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

The invention discloses a data query method and device, and relates to the technical field of computers. One embodiment of the method comprises: generating an SQL statement with a hit instruction according to a predefined hit instruction rule; the hit instruction rule includes: the instruction identification and the corresponding template file thereof; the hit instruction comprises an instruction identification; extracting a hit instruction from the SQL statement, acquiring a template file corresponding to the hit instruction according to an instruction identifier of the hit instruction, and rewriting the SQL statement according to the template file; and inquiring data according to the rewritten sentences. According to the implementation mode, different hit instructions are compiled according to predefined hit instruction rules, and declarative encoding is used for replacing instruction encoding in the program encoding process, codes are decoupled, the query performance is improved, syntax supported by an SQL query mode is expanded, and the expandability of a data query mode is improved.

Description

Data query method and device
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for querying data.
Background
The elastic search (hereinafter, abbreviated as ES) is an efficient distributed storage engine, supports near real-time query capability for mass data, and is widely used in a big data real-time OLAP (Online Analytical Processing) scene. The ES authority supports Query modes based on DSL (Domain Specific Language) and SQL (Structured Query Language) syntax.
The existing ES-SQL component has the following defects: the dimension index query is not supported, dimension switching needs to be realized in program coding, and the code expandability is poor; control instructions not supporting the query phase: such as conditional routing queries, cache control, whether the query results need to be cached, etc.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for data query, where different hit instructions are written according to predefined hit instruction rules, so that declarative coding can be used to replace instruction coding in a program coding process, codes are decoupled, query performance is improved, syntax supported by an SQL query mode is expanded, and extensibility of the data query mode is improved.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a data query method including:
generating an SQL statement with a hit instruction according to a predefined hit instruction rule; the hit instruction rule includes: the instruction identification and the corresponding template file thereof; the hit instruction comprises an instruction identification;
extracting a hit instruction from the SQL statement, acquiring a template file corresponding to the hit instruction according to an instruction identifier of the hit instruction, and rewriting the SQL statement according to the template file;
and inquiring data according to the rewritten sentences.
Optionally, after the hit instruction is extracted from the SQL statement, the method further includes: determining an instruction type of the hit instruction; the instruction type is a routing instruction or a control instruction;
rewriting the SQL statement according to the template file, including: if the hit instruction is a routing instruction, rewriting the SQL statement according to the template file; if the hit instruction is a control instruction of a query class, generating a DSL statement according to the template file; if the hit instruction is a control instruction of a result type, acquiring a control rule of the control instruction according to the template file, and then binding the control rule with the SQL statement.
Optionally, the hit instruction includes an instruction value; rewriting the SQL statement according to the template file, including:
positioning a target index in the SQL statement according to the template file, and replacing the target index with the command value to obtain the rewritten SQL statement; and/or the presence of a gas in the gas,
and determining the query conditions in the SQL sentences, and combining the query conditions in the SQL sentences or adjusting the sequence of the query conditions according to the instruction values and the template files to obtain the rewritten SQL sentences.
Optionally, the hit instruction rule includes: an instruction format rule;
after the hit instruction is extracted from the SQL statement, the method further includes: checking the SQL statement to determine that the syntax of the SQL statement is correct; and/or checking the hit instruction according to the instruction format rule to determine that the hit instruction format is correct.
Optionally, the hit instruction rule includes: instruction parameter rules;
before acquiring the template file corresponding to the hit instruction from the template configuration file according to the instruction identifier of the hit instruction, the method further includes: and checking the hit instruction according to the instruction parameter rule to determine that the hit instruction parameter is correct.
Optionally, before generating the SQL statement with the hit instruction according to the predefined hit instruction rule, the method further includes: and writing the hit instruction rule into an etcd or zookeeper cluster.
Optionally, after writing the hit instruction rule into the etcd or zookeeper cluster, the method further includes: updating the hit instruction rule based on a registration mechanism.
According to still another aspect of the embodiments of the present invention, there is provided a data query apparatus including:
the query statement generation module generates an SQL statement with a hit instruction according to a predefined hit instruction rule; the hit instruction rule includes: the instruction identification and the corresponding template file thereof; the hit instruction comprises an instruction identification;
the query statement rewriting module extracts a hit instruction from the SQL statement, acquires a template file corresponding to the hit instruction according to an instruction identifier of the hit instruction, and rewrites the SQL statement according to the template file;
and the query statement execution module queries data according to the rewritten statement.
Optionally, the query statement rewriting module is further configured to: after a hit instruction is extracted from the SQL statement, determining the instruction type of the hit instruction; the instruction type is a routing instruction or a control instruction;
the query statement rewriting module rewrites the SQL statement according to the template file, including: if the hit instruction is a routing instruction, rewriting the SQL statement according to the template file; if the hit instruction is a control instruction of a query class, generating a DSL statement according to the template file; if the hit instruction is a control instruction of a result type, acquiring a control rule of the control instruction according to the template file, and then binding the control rule with the SQL statement.
Optionally, the hit instruction includes an instruction value; the query statement rewriting module rewrites the SQL statement according to the template file, including:
positioning a target index in the SQL statement according to the template file, and replacing the target index with the command value to obtain the rewritten SQL statement; and/or the presence of a gas in the gas,
and determining the query conditions in the SQL sentences, and combining the query conditions in the SQL sentences or adjusting the sequence of the query conditions according to the instruction values and the template files to obtain the rewritten SQL sentences.
Optionally, the hit instruction rule includes: an instruction format rule; the query statement rewrite module is further configured to: after a hit instruction is extracted from the SQL statement, checking the SQL statement to determine that the syntax of the SQL statement is correct; and/or checking the hit instruction according to the instruction format rule to determine that the hit instruction format is correct.
Optionally, the hit instruction rule includes: instruction parameter rules;
the query statement rewrite module is further configured to: and before acquiring a template file corresponding to the hit instruction from a template configuration file according to the instruction identifier of the hit instruction, verifying the hit instruction according to the instruction parameter rule to determine that the hit instruction parameter is correct.
Optionally, the apparatus further includes an instruction rule configuration module, configured to: before the query statement rewriting module generates an SQL statement with a hit instruction according to a predefined hit instruction rule, writing the hit instruction rule into an etcd or zookeeper cluster.
Optionally, the instruction rule configuration module is further configured to: and after the hit instruction rule is written into the etcd or zookeeper cluster, updating the hit instruction rule based on a registration mechanism.
According to another aspect of the embodiments of the present invention, there is provided an electronic device for data query, including:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method provided by the first aspect of the embodiments of the present invention.
According to a further aspect of embodiments of the present invention, there is provided a computer readable medium, on which a computer program is stored, which when executed by a processor, implements the method provided by the first aspect of embodiments of the present invention.
One embodiment of the above invention has the following advantages or benefits: by compiling different hit instructions according to predefined hit instruction rules, the declarative encoding can replace the instruction encoding in the program encoding process, codes are decoupled, the query performance is improved, the grammar supported by the SQL query mode is expanded, and the expandability of the data query mode is improved.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main flow of a method of data query in an embodiment of the invention;
FIG. 2 is a schematic flow chart diagram of a method of data querying in some embodiments of the invention;
FIG. 3 is a flow diagram of rewriting SQL statements in some embodiments of the invention;
FIG. 4 is a schematic diagram of the main modules of an apparatus for data query according to an embodiment of the present invention;
FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 6 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
According to an aspect of an embodiment of the present invention, a data query method is provided. Fig. 1 is a schematic diagram of a main flow of a data query method according to an embodiment of the present invention, and as shown in fig. 1, the data query method includes:
s101, generating an SQL statement with a hit instruction according to a predefined hit instruction rule; the hit instruction rule includes: the instruction identification and the corresponding template file thereof; the hit instruction comprises an instruction identification;
step S102, extracting a hit instruction from the SQL statement, acquiring a template file corresponding to the hit instruction according to an instruction identifier of the hit instruction, and rewriting the SQL statement according to the template file;
and step S103, inquiring data according to the rewritten sentences.
An instruction identification is a unique representation of an instruction, such as an instruction name, instruction code, and the like. The template file refers to a specific implementation of an instruction, wherein an implementation function for rewriting the SQL statement is configured.
ES supports query modes based on DSL and SQL syntax, etc. DSL supports the query characteristics of all ES, and is assembled in a JSON format, and complex query Json is deeply nested, easy to make mistakes and inconvenient to eliminate. SQL supports the query characteristics of part of ES, and is based on SQL grammar, simple in writing and strong in readability. In data visualization and business BI scenarios, SQL-based query of ES is widely used due to the need to count a large number of data indexes and to consider maintainability and usability. The existing ES-SQL components are official release and gitHub (which is a managed platform facing to open source and private software projects) open source, and basically support SQL query under commonly used aggregation, non-aggregation and other scenes. But there are problems in the following scenarios: (1) the method does not support dimension index query: assuming that indexes are created with time as a dimension, such as 202012_ ORDER and 202011_ ORDER, the date dimension needs to be switched in program coding, and code scalability is poor; (2) control instruction for the non-support inquiry stage: such as ES supported conditional routing queries, which are supported in DSL but not usable in SQL; there are also cache controls, such as whether the query results need to be cached, etc.
The hit instruction is a rule constraint, and is used as a SQL supplementary syntax to allow a user to perform special optimization on SQL by inserting the hit instruction in an SQL statement to influence the execution mode of the SQL. In the present invention, different hit instructions are defined and written, and in the actual application process, a rule engine (for example, Drools, an open source business rule engine) may be adopted to implement step S102 to execute the hit instructions. The invention can solve the defects in the SQL query mode, different hit instructions are compiled according to the predefined hit instruction rules, and declarative encoding is used for replacing instruction encoding and decoupling codes in the program encoding process, so that the query performance is improved, the grammar supported by the SQL query mode is expanded, and the expandability of the data query mode is improved.
The instruction types include routing instructions and control instructions. Routing instructions are used to support index storage in a dimensional table format, such as by date, merchant, region, and the like. The control instruction is used for query cache control, SQL-to-DSL process optimization, query result batch processing operation and the like. Each hit instruction corresponds to a template file. In the actual application process, the hit instruction set can be maintained in advance, and the instruction set can be expanded according to different service scenes. Table 1 below shows the correspondence between part of the instructions and the template file. If a new instruction is needed subsequently, only the instruction name and the corresponding template file need to be defined.
TABLE 1 correspondence of instructions and templates
Figure BDA0002966207140000071
The control instructions can be divided into result-type control instructions and query-type control instructions according to the functions of the instructions. The control instruction of the query class refers to an instruction which affects the execution mode of the query, such as record-limit, scrolled, routing, etc. in table 1. The control instruction of the result class refers to an instruction which affects the query result, such as cach, timeout, null-convert, etc. in table 1. After the hit instruction is extracted from the SQL statement, the method may further include: the instruction type of the hit instruction is determined. When the SQL statement is rewritten according to the template file, if the hit instruction is a routing instruction, rewriting the SQL statement according to the template file; if the hit instruction is a control instruction of the query class, generating a DSL statement according to the template file; if the hit instruction is a control instruction of the result type, acquiring a control rule of the control instruction according to the template file, and then binding the control rule with the SQL statement. Illustratively, for the control instruction of the query class, a corresponding DSL segment is generated according to the control instruction of each query class in the SQL statement, and when the query is executed, all DSL segments are merged and then a query request is sent to the ES. By setting various control instructions, the query performance of the SQL query mode can be expanded.
Optionally, the hit instruction includes an instruction value. When the SQL statement is rewritten according to the template file, the following steps may be included: positioning a target index in the SQL statement according to the template file, and replacing the target index with the SQL statement of which the instruction value is rewritten; and/or determining the query conditions in the SQL sentences, and combining the query conditions in the SQL sentences or adjusting the sequence of the query conditions according to the instruction values and the template files to obtain the rewritten SQL sentences. The template file carries the realization function (namely the routing rule) of the routing instruction, and the SQL statement is rewritten according to the routing rule, so that the query efficiency can be improved on the basis of expanding the query performance of the SQL query mode, and the realization is easy.
The hit instruction rules may include instruction format rules. After the hit instruction is extracted from the SQL statement, the SQL statement may be checked to determine that the syntax of the SQL statement is correct, and/or the hit instruction may be checked according to the instruction format rule to determine that the format of the hit instruction is correct. The content of the command format rule can be selectively set according to the actual situation.
Illustratively, the prefix flag of the hint instruction is predefined to be "/. esHint:/", the instruction is in a format of a key-value pair, the key-value pair is a key value, the key is an instruction identifier, the value is an instruction value, and when a plurality of hint instructions are included in the SQL statement, the key-value pairs of the plurality of hint instructions are separated by # s. One index contains a prefix and a key value pair in the prefix, and when checking, the correctness of the instruction definition is verified, which comprises the following steps: whether the prefix is correct and whether the key-value pair format in the instruction is correct.
The method for verifying whether the syntax of the SQL statement is correct may be selectively set according to the actual situation, and is not specifically limited herein. Optionally, the SQL statement is parsed by a pipeline (a JDBC connection pool component), and the correctness of the SQL statement is verified according to an AST syntax tree (abstract syntax tree). For example, ES-SQL does not support JOIN (put all elements in an array into a string), sub-query queries, ES-SQL only supports select queries. If the SQL statement contains the query, the syntax is incorrect.
By checking the SQL statement and hit instruction format, the waste of computing resources caused by the fact that subsequent logic processing is still carried out when the SQL statement or hit instruction format is incorrect can be avoided, and the query efficiency is improved.
Optionally, the hit instruction rules include instruction parameter rules. Before the template file corresponding to the hit instruction is acquired from the template configuration file according to the instruction identifier of the hit instruction, the method further comprises the following steps: and checking the hit instruction according to the instruction parameter rule to determine that the hit instruction parameter is correct. Illustratively, the instruction value of the instruction record-limit in table 1 is an integer greater than 0, and if the instruction value in the fetched hit instruction is 0, the hit instruction parameter is incorrect and the hit instruction is illegal. By checking the format of the SQL statement and the hit instruction, the waste of computing resources caused by the fact that subsequent logic processing is still carried out when the hit instruction is illegal can be avoided, and the query efficiency is improved.
Before generating the SQL statement with the hit instruction according to the predefined hit instruction rule, the method may further include: the hit instruction rule is written to the etcd or zookeeper cluster. etcd is a highly available distributed key value database, and adopts a consistency algorithm. zookeeper is a distributed application coordination service that can provide consistency services for distributed applications. Illustratively, the method of the embodiment of the present invention is implemented based on an open source rule engine Drools as an underlying component, and when the component is initialized, the rule engine configuration is written into an etcd or zookeeper cluster, so that the unified maintenance of the rule metadata can be realized.
After writing the hit instruction rule into the etcd or zookeeper cluster, the method may further include: the hit instruction rules are updated based on the registration mechanism. Based on the registry, a hot update of the modification of the rules can be implemented.
The rule engine for implementing the embodiment of the invention can be integrated into the project through maven (project object model) in a jar (a software package file format) package form, and integrates the existing SQL query components (official version and open source version) through a component query adaptation interface.
The data query method according to the embodiment of the present invention is exemplarily described below with reference to fig. 2. In this embodiment, hit instruction rules are configured in advance. The embodiment mainly comprises the following steps:
one, SQL parsing
When the SQL statement is submitted to the rule engine, the rule engine firstly verifies the syntax of the SQL statement through SQL analysis, if the hit instruction exists in the SQL statement, the SQL statement is extracted from the SQL statement in the step and the format of the hit instruction is verified to be correct, and then the hit instruction is analyzed. And if the hit instruction does not exist in the SQL statement, directly performing query execution according to the SQL statement.
Two, hit instruction parsing
When the rules engine based query component starts, the component reads the Drools configuration file and initializes the verifier object that instructs the verifier.
And if the hit instruction exists, verifying whether the hit instruction is legal, obtaining a corresponding template file according to the hit name after the verification is passed, analyzing and executing the template file by the Drools, returning a result HintProcessInfo object, and transmitting the HintProcessInfo object serving as a parameter to the component context object for rewriting an SQL statement or optimizing a DSL statement.
Three, SQL statement rewrite
And for the routing instruction, positioning the target index according to the routing instruction rule, and modifying the SQL statement. Rewriting the where condition, merging the repeated inquiry condition and adjusting the condition sequence. And optimizing the SQL statement according to the control rule of the control instruction, wherein the SQL statement is converted into the DSL statement, and the control rule is bound to the SQL statement.
The specific logic is as follows:
1. the HintProcessInfo object is returned after the Drools processing, and three parameters of the instruction name, the instruction value (which is an incoming parameter or an actually called function during rewriting or optimization), and the instruction type are acquired from the object.
2. If the instruction type is route type, the SQL statement rewrite rule is as follows in Table 2:
TABLE 2 SQL statement rewrite rules
Figure BDA0002966207140000101
Figure BDA0002966207140000111
3. If the command type is control type, the optimization rule of the control command is as follows in table 3:
TABLE 3 optimization rules for control instructions
Figure BDA0002966207140000112
The control instruction of the query class refers to an instruction which affects the execution mode of the query, such as record-limit, scrolled, routing, etc. in table 1. The control instruction of the result class refers to an instruction which affects the query result, such as cach, timeout, null-convert, etc. in table 1. After the hit instruction is extracted from the SQL statement, the method may further include: the instruction type of the hit instruction is determined.
If the hit instruction is record-limit, scrolled or routing, generating a DSL statement according to the template file; if the hit instruction is the call, timeout, null-convert, the corresponding control rule is bound with the SQL statement.
Illustratively, suppose/# esHint: schema ═ 202012_ ORDER/select @ from ORDER, and/# esHint: schema ═ 202012_ ORDER/for hit instruction, select @ from ORDER is a query statement, where ORDER is an alias of the ES index. After the hit instruction parsing process, the rewritten SQL is select from 202012_ ORDER.
Fourth, query execution
And querying data according to the rewritten SQL statement or the optimized SQL statement or the generated DSL statement. The component can define a query adaptation Interface in an abstract way, supports adaptation to a bottom SQL query component in a Serial Peripheral Interface (SPI) mode, supports three versions of official version, open source version and self-realization version, and defaults to self-realization. When the source is opened for the official and community, the routing instruction is mainly used, and the routing instruction and the control instruction are simultaneously supported for the custom realization. The self-implemented SQL query component has the following characteristics: caching at SQL statement level; executing a plurality of SQL in batches, and simultaneously carrying out operations such as merging UNION, splicing JOIN and the like on a plurality of query result sets; distinct de-duplication processing on a single result set, and the like; and supporting a data source connection pool.
After the SQL statement is rewritten, or the SQL is optimized, or the DSL process is generated through the steps, the bottom SQL query component carries out query operation according to the rewritten SQL, the optimized SQL, or the generated DSL.
FIG. 3 is a flow diagram illustrating rewriting of SQL statements in some embodiments of the invention. In this embodiment, after receiving the SQL statement with hit instruction in step S301, in step S302, it is checked whether the SQL statement is correct according to the AST syntax tree. If yes, jumping to step S303, and extracting a hit instruction from the SQL statement, otherwise, ending the process. After the hit instruction is extracted, it is checked whether the hit instruction is correct in step S304. If so, go to step S305, otherwise, the process ends. In step S305, the type of hit instruction is determined, if yes, step S306 is skipped to rewrite SQL, otherwise step S307 is skipped to generate DSL or binding control rules. In practical application, the execution sequence of steps S302, S303 and S304 may be adjusted as needed.
According to still another aspect of an embodiment of the present invention, there is provided an apparatus for implementing the above method.
Fig. 4 is a schematic diagram of main modules of a data query apparatus according to an embodiment of the present invention. As shown in fig. 4, the data query apparatus 400 includes:
the query statement generation module 401 generates an SQL statement with a hit instruction according to a predefined hit instruction rule; the hit instruction rule includes: the instruction identification and the corresponding template file thereof; the hit instruction comprises an instruction identification;
the query statement rewriting module 402 is used for extracting a hit instruction from the SQL statement, acquiring a template file corresponding to the hit instruction according to an instruction identifier of the hit instruction, and rewriting the SQL statement according to the template file;
the query sentence execution module 403 queries data according to the rewritten sentence.
Optionally, the query statement rewriting module is further configured to: after a hit instruction is extracted from the SQL statement, determining the instruction type of the hit instruction; the instruction type is a routing instruction or a control instruction;
the query statement rewriting module rewrites the SQL statement according to the template file, including: if the hit instruction is a routing instruction, rewriting the SQL statement according to the template file; if the hit instruction is a control instruction of a query class, generating a DSL statement according to the template file; if the hit instruction is a control instruction of a result type, acquiring a control rule of the control instruction according to the template file, and then binding the control rule with the SQL statement.
Optionally, the hit instruction includes an instruction value; the query statement rewriting module rewrites the SQL statement according to the template file, including:
positioning a target index in the SQL statement according to the template file, and replacing the target index with the command value to obtain the rewritten SQL statement; and/or the presence of a gas in the gas,
and determining the query conditions in the SQL sentences, and combining the query conditions in the SQL sentences or adjusting the sequence of the query conditions according to the instruction values and the template files to obtain the rewritten SQL sentences.
Optionally, the hit instruction rule includes: an instruction format rule;
the query statement rewrite module is further configured to: after a hit instruction is extracted from the SQL statement, checking the SQL statement to determine that the syntax of the SQL statement is correct; and/or checking the hit instruction according to the instruction format rule to determine that the hit instruction format is correct.
Optionally, the hit instruction rule includes: instruction parameter rules;
the query statement rewrite module is further configured to: and before acquiring a template file corresponding to the hit instruction from a template configuration file according to the instruction identifier of the hit instruction, verifying the hit instruction according to the instruction parameter rule to determine that the hit instruction parameter is correct.
Optionally, the apparatus further includes an instruction rule configuration module, configured to: before the query statement rewriting module generates an SQL statement with a hit instruction according to a predefined hit instruction rule, writing the hit instruction rule into an etcd or zookeeper cluster.
Optionally, the instruction rule configuration module is further configured to: and after the hit instruction rule is written into the etcd or zookeeper cluster, updating the hit instruction rule based on a registration mechanism.
According to another aspect of the embodiments of the present invention, there is provided an electronic device for data query, including:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method provided by the first aspect of the embodiments of the present invention.
According to a further aspect of embodiments of the present invention, there is provided a computer readable medium, on which a computer program is stored, which when executed by a processor, implements the method provided by the first aspect of embodiments of the present invention.
Fig. 5 illustrates an exemplary system architecture 500 of a data query method or a data query apparatus to which embodiments of the present invention may be applied.
As shown in fig. 5, the system architecture 500 may include terminal devices 501, 502, 503, a network 504, and a server 505. The network 504 serves to provide a medium for communication links between the terminal devices 501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 501, 502, 503 to interact with a server 505 over a network 504 to receive or send messages or the like. The terminal devices 501, 502, 503 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 501, 502, 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 505 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 501, 502, 503. The backend management server may analyze and perform other processing on the received data such as the product information query request, and feed back a processing result (for example, target push information, product information — just an example) to the terminal device.
It should be noted that the method for querying data provided by the embodiment of the present invention is generally executed by the server 505, and accordingly, the apparatus for querying data is generally disposed in the server 505.
It should be understood that the number of terminal devices, networks, and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 6, a block diagram of a computer system 600 suitable for use with a terminal device implementing an embodiment of the invention is shown. The terminal device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 601.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a query statement generation module, a query statement rewriting module, and a query statement execution module. The names of these modules do not form a limitation on the module itself in some cases, for example, the query statement generation module may also be described as a "module rewriting the SQL statement according to the template file".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: generating an SQL statement with a hit instruction according to a predefined hit instruction rule; the hit instruction rule includes: the instruction identification and the corresponding template file thereof; the hit instruction comprises an instruction identification; extracting a hit instruction from the SQL statement, acquiring a template file corresponding to the hit instruction according to an instruction identifier of the hit instruction, and rewriting the SQL statement according to the template file; and inquiring data according to the rewritten sentences.
According to the technical scheme of the embodiment of the invention, different hit instructions are compiled according to predefined hit instruction rules, so that declarative encoding can be used for replacing instruction encoding in the program encoding process, codes are decoupled, the query performance is improved, the grammar supported by an SQL query mode is expanded, and the expandability of a data query mode is improved.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A method for querying data, comprising:
generating an SQL statement with a hit instruction according to a predefined hit instruction rule; the hit instruction rule includes: the instruction identification and the corresponding template file thereof; the hit instruction comprises an instruction identification;
extracting a hit instruction from the SQL statement, acquiring a template file corresponding to the hit instruction according to an instruction identifier of the hit instruction, and rewriting the SQL statement according to the template file;
and inquiring data according to the rewritten sentences.
2. The method of claim 1, after extracting hit instructions from the SQL statement, further comprising: determining an instruction type of the hit instruction; the instruction type is a routing instruction or a control instruction;
rewriting the SQL statement according to the template file, including: if the hit instruction is a routing instruction, rewriting the SQL statement according to the template file; if the hit instruction is a control instruction of a query class, generating a DSL statement according to the template file; if the hit instruction is a control instruction of a result type, acquiring a control rule of the control instruction according to the template file, and then binding the control rule with the SQL statement.
3. The method of claim 2, wherein the hit instruction includes an instruction value; rewriting the SQL statement according to the template file, including:
positioning a target index in the SQL statement according to the template file, and replacing the target index with the command value to obtain the rewritten SQL statement; and/or the presence of a gas in the gas,
and determining the query conditions in the SQL sentences, and combining the query conditions in the SQL sentences or adjusting the sequence of the query conditions according to the instruction values and the template files to obtain the rewritten SQL sentences.
4. The method of claim 1, wherein the hit instruction rule comprises: an instruction format rule;
after the hit instruction is extracted from the SQL statement, the method further includes: checking the SQL statement to determine that the syntax of the SQL statement is correct; and/or checking the hit instruction according to the instruction format rule to determine that the hit instruction format is correct.
5. The method of claim 1, wherein the hit instruction rule comprises: instruction parameter rules;
before acquiring the template file corresponding to the hit instruction from the template configuration file according to the instruction identifier of the hit instruction, the method further includes: and checking the hit instruction according to the instruction parameter rule to determine that the hit instruction parameter is correct.
6. The method of claim 1, wherein before generating the SQL statement with hit instructions according to the predefined hit instruction rules, further comprising: and writing the hit instruction rule into an etcd or zookeeper cluster.
7. The method of claim 6, wherein after writing the hit instruction rule to an etcd or zookeeper cluster, further comprising: updating the hit instruction rule based on a registration mechanism.
8. A data query apparatus, comprising:
the query statement generation module generates an SQL statement with a hit instruction according to a predefined hit instruction rule; the hit instruction rule includes: the instruction identification and the corresponding template file thereof; the hit instruction comprises an instruction identification;
the query statement rewriting module extracts a hit instruction from the SQL statement, acquires a template file corresponding to the hit instruction according to an instruction identifier of the hit instruction, and rewrites the SQL statement according to the template file;
and the query statement execution module queries data according to the rewritten statement.
9. An electronic device for data query, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-7.
CN202110251343.5A 2021-03-08 2021-03-08 Data query method and device Active CN113760949B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110251343.5A CN113760949B (en) 2021-03-08 2021-03-08 Data query method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110251343.5A CN113760949B (en) 2021-03-08 2021-03-08 Data query method and device

Publications (2)

Publication Number Publication Date
CN113760949A true CN113760949A (en) 2021-12-07
CN113760949B CN113760949B (en) 2023-09-22

Family

ID=78786708

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110251343.5A Active CN113760949B (en) 2021-03-08 2021-03-08 Data query method and device

Country Status (1)

Country Link
CN (1) CN113760949B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040243576A1 (en) * 1998-12-07 2004-12-02 Oracle International Corporation System and method for querying data for implicit hierarchies
CN106294468A (en) * 2015-06-02 2017-01-04 阿里巴巴集团控股有限公司 The method and apparatus processing business datum

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040243576A1 (en) * 1998-12-07 2004-12-02 Oracle International Corporation System and method for querying data for implicit hierarchies
CN106294468A (en) * 2015-06-02 2017-01-04 阿里巴巴集团控股有限公司 The method and apparatus processing business datum

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
魏彬;: "ORACLE SQL语句执行效率研究", 内蒙古科技与经济, no. 11 *

Also Published As

Publication number Publication date
CN113760949B (en) 2023-09-22

Similar Documents

Publication Publication Date Title
CN110019350B (en) Data query method and device based on configuration information
US10827036B2 (en) Version control machine
CN108846753B (en) Method and apparatus for processing data
CN109194714B (en) File pushing method and device, terminal device and storage medium
US11200231B2 (en) Remote query optimization in multi data sources
CN109522341A (en) Realize method, apparatus, the equipment of the stream data processing engine based on SQL
US20170024434A1 (en) Generating sql queries from declarative queries for semi-structured data
CN110555030A (en) SQL statement processing method and device
CN111666293A (en) Database access method and device
CN113760948A (en) Data query method and device
CN108694172B (en) Information output method and device
CN113568923A (en) Method and device for querying data in database, storage medium and electronic equipment
US11914560B2 (en) Systems and methods for creating a reorganization-immune blockchain index using mono-increasing sequence records
CN112947912A (en) Method and device for generating code, electronic equipment and storage medium
CN113760949B (en) Data query method and device
US11341154B2 (en) Normalizing encodings of requested data from a common data schema to a target data schema
CN113760969A (en) Data query method and device based on elastic search
CN113760240A (en) Method and device for generating data model
CN112988778A (en) Method and device for processing database query script
CN112307061A (en) Method and device for querying data
CN115994151B (en) Data request changing method, device, electronic equipment and computer readable medium
US11954114B2 (en) Systems and methods for improved blockchain data indexing by decoupling compute and storage layers
CN111177183B (en) Method and device for generating database access statement
US20240004866A1 (en) Systems and methods for supporting both batch processing and streaming data applications based on raw blockchain data
CN112783904A (en) Method and device for updating index data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant