CN113918595A - Data query method and device - Google Patents

Data query method and device Download PDF

Info

Publication number
CN113918595A
CN113918595A CN202111271368.8A CN202111271368A CN113918595A CN 113918595 A CN113918595 A CN 113918595A CN 202111271368 A CN202111271368 A CN 202111271368A CN 113918595 A CN113918595 A CN 113918595A
Authority
CN
China
Prior art keywords
service
query
data
constructing
created
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111271368.8A
Other languages
Chinese (zh)
Inventor
杨治法
陈绎州
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202111271368.8A priority Critical patent/CN113918595A/en
Publication of CN113918595A publication Critical patent/CN113918595A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/242—Query formulation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing

Abstract

The invention discloses a data query method and a data query device, which can be used in the financial field or other technical fields, and the method comprises the following steps: sending a query request body of a user to a corresponding query engine so that the query engine determines structured data of a service corresponding to the query request body; generating SQL sentences of the service according to the structured data; and carrying out data query according to the SQL statement to obtain a data query result. The invention is beneficial to improving the arranging and releasing efficiency of the service.

Description

Data query method and device
Technical Field
The invention relates to the technical field of data query, in particular to a data query method and device.
Background
The existing service arrangement platform is formed by dragging a user to form a service through a visual interface and then publishing the service, wherein the arrangement of the service is formed through given fixed SQL. The existing arrangement mode of the service is too fixed and not flexible enough, SQL corresponding to the service needs to be fixed every time of arrangement, so that service arrangement personnel need to manually write SQL sentences, and if the query condition of the service changes, the corresponding SQL also needs to be changed. Therefore, the existing service arranging and issuing mode depends on manually writing SQL sentences, which is time-consuming, labor-consuming and inflexible.
Disclosure of Invention
The present invention provides a data query method and apparatus to solve at least one technical problem in the background art.
In order to achieve the above object, according to an aspect of the present invention, there is provided a data query method including:
sending a query request body of a user to a corresponding query engine so that the query engine determines structured data of a service corresponding to the query request body;
generating SQL sentences of the service according to the structured data;
and carrying out data query according to the SQL statement to obtain a data query result.
Optionally, the data query method further includes:
acquiring arrangement information of a service to be created, wherein the arrangement information is obtained by detecting the operation of a user on a service arrangement interface;
generating the structured data of the service to be created according to the arrangement information to complete the creation of the service;
and establishing a corresponding relation between the created service and the query engine according to the field of the created service.
Optionally, after the creating of the service is completed, the method further includes:
generating a service calling document of the created service so that a user can inquire the service according to the service calling document, wherein the service calling document comprises: the request address of the service, the query request body of the service, the return structure, the optional parameters and the optional parameters.
Optionally, the generating an SQL statement of the service according to the structured data specifically includes:
and analyzing the structured data according to the managed SQL and constructing an SQL statement.
Optionally, the constructing the SQL statement specifically includes: constructing a field list, constructing a field list prefix, constructing a query source list, constructing an association condition, constructing a where condition, constructing a packet field and condition, and constructing an order field.
Optionally, the structured data includes: service ID, reference service ID, query type, list of fields queried, table to query, query condition, and sort condition.
In order to achieve the above object, according to another aspect of the present invention, there is provided a data query apparatus including:
the structured data determining module is used for sending a query request body of a user to a corresponding query engine so as to enable the query engine to determine structured data of a service corresponding to the query request body;
the SQL statement generating module is used for generating an SQL statement of the service according to the structured data;
and the data query module is used for carrying out data query according to the SQL statement to obtain a data query result.
Optionally, the data query apparatus further includes:
the service editing device comprises an editing information acquisition module, a service editing module and a service editing module, wherein the editing information acquisition module is used for acquiring editing information of a service to be created, and the editing information is obtained by detecting the operation of a user on a service editing interface;
the service creation module is used for generating the structured data of the service to be created according to the arrangement information to complete the creation of the service;
and the query engine association module is used for establishing the corresponding relation between the created service and the query engine according to the field of the created service.
In order to achieve the above object, according to another aspect of the present invention, there is also provided a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the data query method when executing the computer program.
In order to achieve the above object, according to another aspect of the present invention, there is also provided a computer-readable storage medium storing a computer program which, when executed in a computer processor, implements the steps in the above data query method.
The invention has the beneficial effects that:
when the service query is carried out, the structured data of the service corresponding to the query request body of the user is determined by using the query engine, the SQL (structured query language) statement of the service is generated according to the structured data, and finally, the data query is carried out according to the SQL statement to obtain the data query result.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts. In the drawings:
FIG. 1 is a first flowchart of a data query method according to an embodiment of the present invention;
FIG. 2 is a second flowchart of a data query method according to an embodiment of the present invention;
FIG. 3 is a first block diagram of a data query device according to an embodiment of the present invention;
FIG. 4 is a second block diagram of the data query apparatus according to the embodiment of the present invention;
FIG. 5 is a schematic diagram of a computer apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
It should be noted that the terms "comprises" and "comprising," and any variations thereof, in the description and claims of the present invention and the above-described drawings, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
It should be noted that the data query method and apparatus of the present invention can be applied to the financial field, and can also be applied to other technical fields.
The invention aims at the defects of the existing service arrangement, the data structure of the service is not abstracted, in order to solve the development of repeated codes, the process of service arrangement and release is abstracted into a uniform data structure, SQL is constructed and analyzed through a uniform engine, and a universal query request body (query Schema) is dynamically constructed, the flexible data structure can be rapidly released to form own service, the query Schema can be flexibly defined according to the self requirements of users, the existing platform also needs to separately develop the specific logic of the service for the universal query, the query of the service is not flexible enough, the platform does not have the characteristic of flexible query, if the query condition is changed, the related logic codes need to be readjusted, therefore, the invention designs a universal service arrangement data structure, provides a method for rapidly releasing the service and flexibly querying and executing the service, and service authentication and related current limiting flexible configuration are realized, the convenience of application development is improved, and the operation and maintenance management cost of repeated codes is effectively reduced.
Fig. 1 is a first flowchart of a data query method according to an embodiment of the present invention, and as shown in fig. 1, in an embodiment of the present invention, the data query method according to the present invention includes steps S101 to S103.
Step S101, a query request body of a user is sent to a corresponding query engine, so that the query engine determines structured data of a service corresponding to the query request body.
The service of the present invention may be a query service. The invention designs a general data structure, and creates service based on the data structure, so that SQL sentences do not need to be written when creating service. The invention directly generates the structured data of the service when the service is created.
In an embodiment of the present invention, the structured data includes: service ID, reference service ID, type of query, list of fields queried, table to query, query conditions, and sort conditions.
In the embodiment of the present invention, the gateway receives query requests from different channels (mobile terminal, Web terminal, and other API calls), and the gateway routes the query requests to the corresponding query engine (DataFetcher) to perform data capture according to the query request body (query Schema) in the input query request.
Different DataFetchers correspond to different data sources, the DataFetcher of the general data service executes the structure of the SQL according to the transmitted query Schema data structure, carries out syntax check on the structured SQL and then executes the return data.
And step S102, generating SQL statements of the service according to the structured data.
In an embodiment of the present invention, the structured data can be parsed according to the managed SQL, and the SQL statement can be constructed.
In an embodiment of the present invention, constructing an SQL statement specifically includes: constructing a field list, constructing a field list prefix, constructing a query source list, constructing an association condition, constructing a where condition, constructing a packet field and condition, and constructing an order field.
And S103, performing data query according to the SQL statement to obtain a data query result.
In the embodiment of the invention, the data query is carried out by executing the SQL statement to obtain the data query result.
The query of the service is executed by the query engine, different query execution codes do not need to be developed for different queries, the result of service arrangement, namely the service, is uniformly abstracted, when the query is called, the service is uniformly routed to the query engine of the specific service from the gateway, the corresponding service is queried by the engine actuator, the SQL reverse construction is carried out on the corresponding service, the constructed SQL is transmitted to JdbcTemplate query, and the uniform query result is returned.
FIG. 2 is a second flowchart of a data query method according to an embodiment of the present invention, as shown in FIG. 2, in an embodiment of the present invention, the process of service orchestration according to the present invention includes steps S201 to S203.
Step S201, obtaining layout information of a service to be created, where the layout information is obtained by detecting an operation of a user on a service layout interface.
In the embodiment of the invention, the user performs service arrangement by performing dragging and combining of modules on the visualized service arrangement interface.
Step S202, generating the structured data of the service to be created according to the arrangement information, and completing the creation of the service.
Step S203, according to the field of the created service, establishing the corresponding relation between the created service and the query engine.
In one embodiment of the present invention, after completing the creation of the service, the present invention further comprises:
generating a service calling document of the created service so that a user can inquire the service according to the service calling document, wherein the service calling document comprises: the request address of the service, the query request body of the service, the return structure, the optional parameters and the optional parameters.
The calling of the service is uniformly routed through the gateway, the service is routed to different query engines according to the fields corresponding to different services, after the service is issued, a service calling document can be generated, the service customizes a data request format based on GraphQL grammar during query, flexible query and aggregated query of data can be met, data sources behind different services can be diversified during query, the data sources can be derived from a relational database MySQL, an analytic database ClickHouse and an index database Elasticissearch, the query engine which is independently designed can be applied according to different fields corresponding to different services and is routed to the corresponding data query sources, and the specific operation comprises the following steps:
the service arrangement is carried out through the visual platform, a user selects data sources in different fields, an index table in the field can be displayed, and then a data structure corresponding to the service can be formed by dragging the table, selecting table fields and filling table conditions.
And (3) issuing the service, namely, after issuing the layout result according to the result of the user layout in the last step, mapping the service to different query engines according to different fields, and enabling the query engines to be responsible for unified query.
The method comprises the steps of calling a service, checking a query result through simulation calling for the service which is already arranged, when the arrangement result of the service is inconsistent with an expected effect, rearranging, modifying and releasing the service again, saving graph information and data structure information of the result of each arrangement of the service, loading information corresponding to the service when modifying the service next time, automatically generating an interface document for the service before calling the service, and dynamically constructing a query Schema by a user.
It can be seen from the above embodiments that the present invention abstracts SQL based on an autonomously designed universal layout data structure, performs SQL reverse construction and analysis using the droid SQL, and performs a one-stop link for service layout and release and fast authorized use by autonomously developing a plurality of functional modules such as a gateway, an engine, and a constructor. The method and the system achieve quick arrangement, quick online and quick use of the service, well solve the problem that different applications need to develop redundant repeated codes due to the inflexibility of the service arrangement platform arrangement service at present, abstract a general data service structure, enable users to quickly release own service without knowing code logic, enable the service to be immediately verified and used, and customize service current limiting parameters such as the maximum request number of the service per second, TPS and QPS of the service, access tokens of the service and access authority of the service, reduce the query logic of the corresponding service required to be developed by different applications, improve the efficiency of quick arrangement and release online of the service, and enable the service to be lighter and more free.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
Based on the same inventive concept, the embodiment of the present invention further provides a data query apparatus, which can be used to implement the data query method described in the foregoing embodiment, as described in the following embodiment. Because the principle of the data query apparatus for solving the problem is similar to that of the data query method, the embodiment of the data query apparatus can be referred to as the embodiment of the data query method, and repeated details are not repeated. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 3 is a first structural block diagram of a data query apparatus according to an embodiment of the present invention, and as shown in fig. 3, in an embodiment of the present invention, the data query apparatus according to the embodiment of the present invention includes:
the structured data determining module 1 is configured to send a query request body of a user to a corresponding query engine, so that the query engine determines structured data of a service corresponding to the query request body;
the SQL sentence generating module 2 is used for generating an SQL sentence of the service according to the structured data;
and the data query module 3 is used for carrying out data query according to the SQL statement to obtain a data query result.
Fig. 4 is a second structural block diagram of the data query apparatus according to the embodiment of the present invention, as shown in fig. 4, in an embodiment of the present invention, the data query apparatus according to the embodiment of the present invention further includes:
the arrangement information acquisition module 4 is used for acquiring arrangement information of a service to be created, wherein the arrangement information is obtained by detecting the operation of a user on a service arrangement interface;
the service creation module 5 is used for generating the structured data of the service to be created according to the arrangement information to complete the creation of the service;
and the query engine association module 6 is used for establishing the corresponding relation between the created service and the query engine according to the field of the created service.
In an embodiment of the present invention, the data query apparatus in the embodiment of the present invention further includes:
a service call document generation module, configured to generate a service call document of a service that is created, so that a user performs service query according to the service call document, where the service call document includes: the request address of the service, the query request body of the service, the return structure, the optional parameters and the optional parameters.
In an embodiment of the present invention, the SQL statement generation module 2 is specifically configured to analyze the structured data according to the managed SQL and construct an SQL statement.
To achieve the above object, according to another aspect of the present application, there is also provided a computer apparatus. As shown in fig. 5, the computer device comprises a memory, a processor, a communication interface and a communication bus, wherein a computer program that can be run on the processor is stored in the memory, and the steps of the method of the above embodiment are realized when the processor executes the computer program.
The processor may be a Central Processing Unit (CPU). The Processor may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or a combination thereof.
The memory, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and units, such as the corresponding program units in the above-described method embodiments of the present invention. The processor executes various functional applications of the processor and the processing of the work data by executing the non-transitory software programs, instructions and modules stored in the memory, that is, the method in the above method embodiment is realized.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor, and the like. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory located remotely from the processor, and such remote memory may be coupled to the processor via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more units are stored in the memory and when executed by the processor perform the method of the above embodiments.
The specific details of the computer device may be understood by referring to the corresponding related descriptions and effects in the above embodiments, and are not described herein again.
In order to achieve the above object, according to another aspect of the present application, there is also provided a computer-readable storage medium storing a computer program which, when executed in a computer processor, implements the steps in the above data query method. It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD) or a Solid State Drive (SSD), etc.; the storage medium may also comprise a combination of memories of the kind described above.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and they may alternatively be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, or fabricated separately as individual integrated circuit modules, or fabricated as a single integrated circuit module from multiple modules or steps. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A method for querying data, comprising:
sending a query request body of a user to a corresponding query engine so that the query engine determines structured data of a service corresponding to the query request body;
generating SQL sentences of the service according to the structured data;
and carrying out data query according to the SQL statement to obtain a data query result.
2. The data query method of claim 1, further comprising:
acquiring arrangement information of a service to be created, wherein the arrangement information is obtained by detecting the operation of a user on a service arrangement interface;
generating the structured data of the service to be created according to the arrangement information to complete the creation of the service;
and establishing a corresponding relation between the created service and the query engine according to the field of the created service.
3. The data query method of claim 2, after the creating of the completion service, further comprising:
generating a service calling document of the created service so that a user can inquire the service according to the service calling document, wherein the service calling document comprises: the request address of the service, the query request body of the service, the return structure, the optional parameters and the optional parameters.
4. The data query method according to claim 1, wherein the generating the SQL statement of the service according to the structured data specifically includes:
and analyzing the structured data according to the managed SQL and constructing an SQL statement.
5. The data query method according to claim 4, wherein the constructing the SQL statement specifically includes: constructing a field list, constructing a field list prefix, constructing a query source list, constructing an association condition, constructing a where condition, constructing a packet field and condition, and constructing an order field.
6. The data query method of claim 1, wherein the structured data comprises: service ID, reference service ID, query type, list of fields queried, table to query, query condition, and sort condition.
7. A data query apparatus, comprising:
the structured data determining module is used for sending a query request body of a user to a corresponding query engine so as to enable the query engine to determine structured data of a service corresponding to the query request body;
the SQL statement generating module is used for generating an SQL statement of the service according to the structured data;
and the data query module is used for carrying out data query according to the SQL statement to obtain a data query result.
8. The data query apparatus of claim 7, further comprising:
the service editing device comprises an editing information acquisition module, a service editing module and a service editing module, wherein the editing information acquisition module is used for acquiring editing information of a service to be created, and the editing information is obtained by detecting the operation of a user on a service editing interface;
the service creation module is used for generating the structured data of the service to be created according to the arrangement information to complete the creation of the service;
and the query engine association module is used for establishing the corresponding relation between the created service and the query engine according to the field of the created service.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 6 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when executed in a computer processor, implements the method of any one of claims 1 to 6.
CN202111271368.8A 2021-10-29 2021-10-29 Data query method and device Pending CN113918595A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111271368.8A CN113918595A (en) 2021-10-29 2021-10-29 Data query method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111271368.8A CN113918595A (en) 2021-10-29 2021-10-29 Data query method and device

Publications (1)

Publication Number Publication Date
CN113918595A true CN113918595A (en) 2022-01-11

Family

ID=79243630

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111271368.8A Pending CN113918595A (en) 2021-10-29 2021-10-29 Data query method and device

Country Status (1)

Country Link
CN (1) CN113918595A (en)

Similar Documents

Publication Publication Date Title
CN109739604B (en) Page rendering method, device, server and storage medium
US9514184B2 (en) Systems and methods for a high speed query infrastructure
WO2020192141A1 (en) Method and device for simulating interface, computer apparatus, and storage medium
CN110781183A (en) Method and device for processing incremental data in Hive database and computer equipment
US20150269234A1 (en) User Defined Functions Including Requests for Analytics by External Analytic Engines
CN108255967B (en) Method and device for calling storage process, storage medium and terminal
CN111818175A (en) Enterprise service bus configuration file generation method, device, equipment and storage medium
CN111240772A (en) Data processing method and device based on block chain and storage medium
CN107491700B (en) Data access method and equipment
CN113987337A (en) Search method, system, equipment and storage medium based on componentized dynamic arrangement
CN113901083A (en) Heterogeneous data source operation resource analysis positioning method and equipment based on multiple analyzers
CN113918595A (en) Data query method and device
CN112817922B (en) Log dynamic printing method and device, computer equipment and storage medium
CN112130849B (en) Code automatic generation method and device
US20170344549A1 (en) Enhanced database query processing
CN114186958A (en) Method, computing device and storage medium for exporting list data as spreadsheet
CN113448985A (en) API (application program interface) interface generation method, calling method and device and electronic equipment
CN112052035A (en) Version packaging method and device based on bank back-line system
CN111506644B (en) Application data processing method and device and electronic equipment
CN113268483A (en) Request processing method and device, electronic equipment and storage medium
CN113553250A (en) Code testing method and device
CN112579705A (en) Metadata acquisition method and device, computer equipment and storage medium
CN110334098A (en) A kind of database combining method and system based on script
CN107451201B (en) Data access method and equipment
Schäfer et al. Sampling with incremental mapreduce

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination