CN114328577A - Data query method and device - Google Patents
Data query method and device Download PDFInfo
- Publication number
- CN114328577A CN114328577A CN202111545870.3A CN202111545870A CN114328577A CN 114328577 A CN114328577 A CN 114328577A CN 202111545870 A CN202111545870 A CN 202111545870A CN 114328577 A CN114328577 A CN 114328577A
- Authority
- CN
- China
- Prior art keywords
- query
- template
- cache
- data
- parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a data query method and device. The method comprises the following steps: after receiving a query request, acquiring a pre-configured query template from a cache; receiving input query parameters in the query template to obtain a parameter-carrying template; and inquiring from the cache according to the template with the parameters to obtain an inquiry result. After receiving a query request, the method acquires a pre-configured query template from a cache; receiving input query parameters in the query template to obtain a parameter-carrying template; and inquiring from the cache according to the template with the parameters to obtain an inquiry result. The utility model utilizes the local cache, the query module is stored in the local cache, the query result is obtained, and the local cache is repeatedly utilized, so that the efficiency and the speed of data query can be greatly improved.
Description
Technical Field
The application relates to the field of computers, in particular to a data query method and device.
Background
When the large data interface platform receives a data query request, the query efficiency is low, and the query response speed is slow.
Disclosure of Invention
The present application is mainly directed to providing a data query method and apparatus to solve the above problems.
In order to achieve the above object, according to an aspect of the present application, there is provided a data query method including:
after receiving a query request, acquiring a pre-configured query template from a cache;
receiving input query parameters in the query template to obtain a parameter-carrying template;
and inquiring from the cache according to the template with the parameters to obtain an inquiry result.
In one embodiment, the configuration of the query template includes one or more of: setting query parameters in a query template, setting a note prevention level, setting a template state, a template name and caching duration; the anti-note rating includes: basic protection, intermediate protection and high protection; selection of a data source is determined.
In one embodiment, before inputting the query parameters into the query template to obtain the parameter-bearing template, the method further comprises: analyzing the query template to generate a query link; the query link includes a template number.
In one embodiment, after the query is finished, the method further includes: generating a log file; the content recorded in the log file includes: error information occurring in the inquiry process and the inquiry result.
In one implementation, after receiving a query statement, judging whether the query statement is the same as a historical query statement in a preset time period; and if the query results are the same, obtaining the query results of the historical query statements from a cache.
In one embodiment, before judging whether a query template exists, starting a query system, initializing a connection pool of a data source providing query, and generating initial idle connection; and when the query is carried out, the initial idle connection is used for the query.
In one embodiment, for any item of query content, counting the response time of each query statement in a plurality of query statements of the query content; sequencing the query statements according to the sequence of response time from large to small; and determining the query statement with the shortest response time.
In order to achieve the above object, according to another aspect of the present application, there is provided a data query apparatus including:
the acquisition module is used for acquiring a pre-configured query template from the cache after receiving the query request;
the receiving module is used for receiving input query parameters in the query template to obtain a parameter-carrying template;
and the query module is used for querying from the cache according to the parameter-carrying template to obtain a query result.
In one embodiment, the configuration of the query template includes one or more of: setting query parameters in a query template, setting a note prevention level, setting a template state, a template name and caching duration; the anti-note rating includes: basic protection, intermediate protection and high protection; selection of a data source is determined.
In one embodiment, the query link module is configured to parse the query template to generate a query link before inputting the query parameter in the query template to obtain a template with parameters; the query link includes a template number.
In one embodiment, the log module is configured to generate a log file after the query is finished; the content recorded in the log file includes: error information occurring in the inquiry process and the inquiry result.
In one embodiment, the determining module is configured to determine whether a query statement is the same as a historical query statement in a preset time period after receiving the query statement; and if the query results are the same, obtaining the query results of the historical query statements from a cache.
In one embodiment, the system further comprises a connection module, configured to start the query system before determining whether there is a query template, perform connection pool initialization on a data source providing a query, and generate an initial idle connection; and when the query is carried out, the initial idle connection is used for the query.
In one embodiment, the system further comprises a sorting module, configured to count, for any one item of query content, a response time of each query statement in a plurality of query statements of the query content; sequencing the query statements according to the sequence of response time from large to small; and determining the query statement with the shortest response time.
In order to achieve the above object, according to a third aspect of the present application, there is provided an electronic apparatus; comprising at least one processor and at least one memory; the memory is to store one or more program instructions; the processor is configured to execute one or more program instructions to perform any of the above steps.
According to a fourth aspect of the present application, there is provided a computer readable storage medium having one or more program instructions embodied therein for performing the steps of any of the above.
In the embodiment of the application, after receiving a query request, obtaining a pre-configured query template from a cache; receiving input query parameters in the query template to obtain a parameter-carrying template; and inquiring from the cache according to the template with the parameters to obtain an inquiry result. The utility model utilizes the local cache, the query module is stored in the local cache, the query result is obtained, and the local cache is repeatedly utilized, so that the efficiency and the speed of data query can be greatly improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, serve to provide a further understanding of the application and to enable other features, objects, and advantages of the application to be more apparent. The drawings and their description illustrate the embodiments of the utility model and do not limit it. In the drawings:
FIG. 1 is a flow chart of a method of data query according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a query template interface according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a data query device according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Moreover, some of the above terms may be used to indicate other meanings besides the orientation or positional relationship, for example, the term "on" may also be used to indicate some kind of attachment or connection relationship in some cases. The specific meanings of these terms in the present invention can be understood by those skilled in the art as appropriate.
Furthermore, the terms "mounted," "disposed," "provided," "connected," and "sleeved" are to be construed broadly. For example, it may be a fixed connection, a removable connection, or a unitary construction; can be a mechanical connection, or an electrical connection; may be directly connected, or indirectly connected through intervening media, or may be in internal communication between two devices, elements or components. The specific meanings of the above terms in the present invention can be understood by those of ordinary skill in the art according to specific situations.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
As shown in fig. 1, the method includes steps S102 to S106 as follows:
a method of data query, comprising:
step S102, after receiving the query request, obtaining a pre-configured query template from a cache;
the query template is configured in advance and comprises one or more of the following parameters, the query parameter in the query template, the anti-annotation level, the template state, the template name and the cache duration; the anti-note rating includes: basic protection, intermediate protection and high protection.
Illustratively, the query template configuration includes the following: query parameters can be set in the template, for example:
SELECT A.a,A.b,A.c from A where A.a＝{a}and A.b＝{b}。
step S104, receiving input query parameters in the query template to obtain a parameter-carrying template;
illustratively, the parameters are as follows:
and step S106, inquiring from the cache according to the template with the parameters to obtain an inquiry result.
The query method can be applied to a large data query platform DSAPI.
Referring to FIG. 2, an interface diagram of a query template is shown; the status of the template includes enabled, disabled. The anti-note rating includes: a highest level, a medium level, and a lowest level. The user can make a selection. The data source may be selected from a drop down list; the data source management is used for showing a list and detailed description of available data sources, so that a user can know specific information of the data sources and select the correct data source when configuring the template.
The template number and the template name are user-defined. The template link includes a template number.
In order to accelerate data query, the query template, the query result and the query statement are cached: by caching the query template with the parameters, the interaction between the user and the database when the user queries the statement is reduced. After the cache template is analyzed, the cache result is obtained through interaction with a storage tool, the cache time of the cache is long, and the cache of the template cannot be overdue as long as the details of the template are not modified.
In one embodiment, the query template is parsed to generate a query link, and the query link includes a template number.
Illustratively, the query links are as follows: /api/v 2.0/queryData/123456. Wherein 123456 is the template number.
In one embodiment, after the query is finished, a log file is generated; the content recorded in the log file includes: error information occurring in the inquiry process and the inquiry result. The time of the query response.
For example, the log may record a specific error occurring in the query, where the error may be a query template error, a query parameter error, or a logical relationship error of a query statement;
the time of the query response may be a certain point in time. The time points of multiple queries for querying the same query content can be recorded, and the time duration for responding to each query is recorded;
for the same content in the cache, in principle, the response time for making a query is the same.
If the two response times are different, comparing the query parameters of the query sentences with the same content; and searching a specific difference value, and optimizing the setting of the query parameters and the setting of the query statement.
Further, for any item of query content, counting the response time of each query statement in a plurality of query statements of the query content; sequencing the query statements according to the sequence of response time from large to small; and determining the query statement with the shortest response time. The query parameters in the query statement are set as the preferred query parameters.
And in the set effective time of the cache, when the same SQL statement is queried, firstly obtaining a query result from the cache, and if the query result does not exist, continuously interacting with a storage engine to obtain data. In one embodiment, after receiving a query statement, judging whether the query statement is the same as a historical query statement in a preset time period; and if the query results are the same, obtaining the query results of the historical query statements from a cache.
Specifically, a flag may be set for each historical query statement; and establishing a corresponding relation table between each historical query statement and the query result, and finding the corresponding query result according to each historical query statement identifier and the corresponding relation table.
For the access of the relational database, a pooling technology is used, when the system is started, all data sources providing query are subjected to connection pool initialization to generate certain initial idle connections, and the connections always keep the connection of the system to the database, so that when a user queries, the existing connections can be directly used for directly querying, the time for waiting for connection initialization is saved, and the time consumed by the user for querying is reduced.
In one embodiment, before judging whether a query template exists, starting a query system, initializing a connection pool of a data source providing query, and generating initial idle connection; and when the query is carried out, the initial idle connection is used for the query.
The method supports user-defined SQL sentences, the system provides visual query template editing, users can define the SQL sentences to be queried by users, and query parameters can be specified in the sentences. The method supports the access of relational databases such as MySQL, Oracle and the like, and greatly improves the connection efficiency and the data query efficiency by adopting the connection pool technology. And analyzing the completed SQL sentence and the queried data result for the query template, and providing cache, wherein the query result can define the cache time by user. And recording the query behavior to obtain the response time of each query and the access information of the query. In order to monitor the platform.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
In a second aspect, the present application also proposes a data query device, see a schematic structural diagram of a data query device shown in fig. 3; the device includes:
the acquiring module 31 is configured to acquire a pre-configured query template from a cache after receiving a query request;
a receiving module 32, configured to receive an input query parameter in the query template to obtain a parameter-bearing template;
and the query module 33 is configured to query the cache according to the parameter template to obtain a query result.
In one embodiment, the configuration of the query template includes one or more of: setting query parameters in a query template, setting a note prevention level, setting a template state, a template name and caching duration; the anti-note rating includes: basic protection, intermediate protection and high protection; selection of a data source is determined.
In an embodiment, the query link module 34 is configured to parse the query template to generate a query link before inputting the query parameter into the query template to obtain a parameter-containing template; the query link includes a template number.
In an embodiment, the log module 35 is configured to generate a log file after the query is finished; the content recorded in the log file includes: error information occurring in the inquiry process and the inquiry result.
In an embodiment, the determining module 36 is configured to determine whether the query statement is the same as a historical query statement in a preset time period after receiving the query statement; and if the query results are the same, obtaining the query results of the historical query statements from a cache.
In an embodiment, the system further includes a connection module 37, configured to start the query system before determining whether there is a query template, perform connection pool initialization on a data source providing a query, and generate an initial idle connection; and when the query is carried out, the initial idle connection is used for the query.
In one embodiment, the system further comprises a sorting module 38, configured to count, for any one item of query content, a response time of each query statement in a plurality of query statements of the query content; sequencing the query statements according to the sequence of response time from large to small; and determining the query statement with the shortest response time.
According to a third aspect of the present application, there is provided an electronic device, see the schematic structural diagram of an electronic device shown in fig. 4; comprises at least one processor 41 and at least one memory 42; the memory 42 is for storing one or more program instructions; the processor 41 is configured to execute one or more program instructions to perform the following steps:
after receiving a query request, acquiring a pre-configured query template from a cache;
receiving input query parameters in the query template to obtain a parameter-carrying template;
and inquiring from the cache according to the template with the parameters to obtain an inquiry result.
In one embodiment, the processor 41 is further configured to configure the query template to include one or more of the following: setting query parameters in a query template, setting a note prevention level, setting a template state, a template name and caching duration; the anti-note rating includes: basic protection, intermediate protection and high protection; selection of a data source is determined.
In an embodiment, the processor 41 is further configured to, before inputting a query parameter into the query template to obtain a parameter-containing template, analyze the query template to generate a query link; the query link includes a template number.
In one embodiment, the processor 41 is further configured to, after the query is ended, further include: generating a log file; the content recorded in the log file includes: error information occurring in the inquiry process and the inquiry result.
In an embodiment, the processor 41 is further configured to, after receiving a query statement, determine whether the query statement is the same as a historical query statement in a preset time period; and if the query results are the same, obtaining the query results of the historical query statements from a cache.
In an embodiment, the processor 41 is further configured to, before determining whether there is a query template, start the query system, perform connection pool initialization on a data source providing a query, and generate an initial idle connection; and when the query is carried out, the initial idle connection is used for the query.
In one embodiment, the processor 41 is further configured to, for any one item of query content, count a response time of each query statement in a plurality of query statements of the query content; sequencing the query statements according to the sequence of response time from large to small; and determining the query statement with the shortest response time.
In a fourth aspect, the present application also proposes a computer-readable storage medium having one or more program instructions embodied therein, the one or more program instructions being configured to perform the steps of:
after receiving a query request, acquiring a pre-configured query template from a cache;
receiving input query parameters in the query template to obtain a parameter-carrying template;
and inquiring from the cache according to the template with the parameters to obtain an inquiry result.
In one embodiment, the configuration of the query template includes one or more of: setting query parameters in a query template, setting a note prevention level, setting a template state, a template name and caching duration; the anti-note rating includes: basic protection, intermediate protection and high protection; selection of a data source is determined.
In one embodiment, before inputting the query parameters into the query template to obtain the parameter-bearing template, the method further comprises: analyzing the query template to generate a query link; the query link includes a template number.
In one embodiment, after the query is finished, the method further includes: generating a log file; the content recorded in the log file includes: error information occurring in the inquiry process and the inquiry result.
In one implementation, after receiving a query statement, judging whether the query statement is the same as a historical query statement in a preset time period; and if the query results are the same, obtaining the query results of the historical query statements from a cache.
In one embodiment, before judging whether a query template exists, starting a query system, initializing a connection pool of a data source providing query, and generating initial idle connection; and when the query is carried out, the initial idle connection is used for the query.
In one embodiment, for any item of query content, counting the response time of each query statement in a plurality of query statements of the query content; sequencing the query statements according to the sequence of response time from large to small; and determining the query statement with the shortest response time.
The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The processor reads the information in the storage medium and completes the steps of the method in combination with the hardware.
The storage medium may be a memory, for example, which may be volatile memory or nonvolatile memory, or which may include both volatile and nonvolatile memory.
The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory.
The volatile Memory may be a Random Access Memory (RAM) which serves as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), SLDRAM (SLDRAM), and Direct Rambus RAM (DRRAM).
The storage media described in connection with the embodiments of the utility model are intended to comprise, without being limited to, these and any other suitable types of memory.
Those skilled in the art will appreciate that the functionality described in the present invention may be implemented in a combination of hardware and software in one or more of the examples described above. When software is applied, the corresponding functionality may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and they may alternatively be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, or fabricated separately as individual integrated circuit modules, or fabricated as a single integrated circuit module from multiple modules or steps. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A method for querying data, comprising:
after receiving a query request, acquiring a pre-configured query template from a cache;
receiving input query parameters in the query template to obtain a parameter-carrying template;
and inquiring from the cache according to the template with the parameters to obtain an inquiry result.
2. The data query method of claim 1,
the configuration of the query template includes one or more of the following: setting query parameters in a query template, setting a note prevention level, setting a template state, a template name and caching duration; the anti-note rating includes: basic protection, intermediate protection and high protection; selection of a data source is determined.
3. The data query method of claim 1,
before inputting query parameters in the query template to obtain a parameter-bearing template, the method further comprises: analyzing the query template to generate a query link; the query link includes a template number.
4. The data query method of claim 1, wherein after the query is finished, the method further comprises: generating a log file; the content recorded in the log file includes: error information occurring in the inquiry process and the inquiry result.
5. The data query method according to claim 1, wherein after receiving a query statement, determining whether the query statement is the same as a historical query statement in a preset time period; and if the query results are the same, obtaining the query results of the historical query statements from a cache.
6. The data query method of claim 1, wherein before determining whether there is a query template, a query system is started to initialize a connection pool for a data source providing a query, and an initial idle connection is generated; and when the query is carried out, the initial idle connection is used for the query.
7. The data query method according to claim 1, wherein for any one query content, a response time of each query statement in a plurality of query statements of the query content is counted; sequencing the query statements according to the sequence of response time from large to small; and determining the query statement with the shortest response time.
8. A data query apparatus, comprising;
the acquisition module is used for acquiring a pre-configured query template from the cache after receiving the query request;
the receiving module is used for receiving input query parameters in the query template to obtain a parameter-carrying template;
and the query module is used for querying from the cache according to the parameter-carrying template to obtain a query result.
9. An electronic device, comprising: at least one processor and at least one memory; the memory is to store one or more program instructions; the processor, configured to execute one or more program instructions to perform the method of any of claims 1-7.
10. A computer-readable storage medium having one or more program instructions embodied therein for performing the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111545870.3A CN114328577A (en) | 2021-12-15 | 2021-12-15 | Data query method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111545870.3A CN114328577A (en) | 2021-12-15 | 2021-12-15 | Data query method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114328577A true CN114328577A (en) | 2022-04-12 |
Family
ID=81053097
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111545870.3A Pending CN114328577A (en) | 2021-12-15 | 2021-12-15 | Data query method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114328577A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116521713A (en) * | 2023-06-30 | 2023-08-01 | 北京奥星贝斯科技有限公司 | Data query method, device, equipment and storage medium |
-
2021
- 2021-12-15 CN CN202111545870.3A patent/CN114328577A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116521713A (en) * | 2023-06-30 | 2023-08-01 | 北京奥星贝斯科技有限公司 | Data query method, device, equipment and storage medium |
| CN116521713B (en) * | 2023-06-30 | 2023-09-12 | 北京奥星贝斯科技有限公司 | Data query method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101925165B1 (en) | Enriching database query responses using data from external data sources | |
| CN112463886B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN112988782B (en) | Hive-supported interactive query method and device and storage medium | |
| CN106293891B (en) | Multidimensional investment index monitoring method | |
| CN109726244A (en) | Data lead-in method and device | |
| CN110659282A (en) | Data route construction method and device, computer equipment and storage medium | |
| CN108829651A (en) | A kind of method, apparatus of document treatment, terminal device and storage medium | |
| CN114020790A (en) | Data query method and device | |
| CN114116762A (en) | Offline data fuzzy search method, device, equipment and medium | |
| CN113407807A (en) | Query optimization method and device for search engine and electronic equipment | |
| CN114328577A (en) | Data query method and device | |
| CN111339720A (en) | Chip information generation method and device based on big data acquisition and computer equipment | |
| CN113342876B (en) | Data fuzzy query method and device of multi-tenant CRM system in SaaS environment | |
| CN110737432A (en) | script aided design method and device based on root list | |
| CN106250477B (en) | Paging method for data in sub-table | |
| CN113377791A (en) | Data processing method, system and computing equipment | |
| CN104750806A (en) | Large data query method and system | |
| CN107633094B (en) | Method and device for data retrieval in cluster environment | |
| CN114116646A (en) | Log data processing method, device, equipment and storage medium | |
| CN115114012B (en) | Task allocation method and device, electronic equipment and storage medium | |
| CN106446080B (en) | Data query method, query service equipment, client equipment and data system | |
| CN109344300A (en) | The data query of natural language is intended to determine method, apparatus and computer equipment | |
| CN111090670B (en) | Data pre-aggregation method, system, computing device and storage medium | |
| CN106933892B (en) | The storage method and device of description information | |
| CN109871372B (en) | Method for combining database tuning and program tuning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |