CN111782923A - Data query method and device, electronic equipment and storage medium - Google Patents
Data query method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111782923A CN111782923A CN202010593984.4A CN202010593984A CN111782923A CN 111782923 A CN111782923 A CN 111782923A CN 202010593984 A CN202010593984 A CN 202010593984A CN 111782923 A CN111782923 A CN 111782923A
- Authority
- CN
- China
- Prior art keywords
- data
- data query
- query
- query request
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000012795 verification Methods 0.000 claims abstract description 97
- 238000004364 calculation method Methods 0.000 claims description 31
- 230000006870 function Effects 0.000 claims description 31
- 238000004422 calculation algorithm Methods 0.000 claims description 13
- 238000004458 analytical method Methods 0.000 claims description 10
- 230000001419 dependent effect Effects 0.000 claims description 8
- 238000013507 mapping Methods 0.000 claims description 8
- 238000000354 decomposition reaction Methods 0.000 claims description 6
- 238000004806 packaging method and process Methods 0.000 claims description 5
- 238000007616 round robin method Methods 0.000 claims 2
- 238000010200 validation analysis Methods 0.000 claims 2
- 238000011161 development Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 238000007726 management method Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000013508 migration Methods 0.000 description 3
- 230000005012 migration Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 239000003607 modifier Substances 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9532—Query formulation
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to big data, and provides a data query method, which comprises the following steps: after receiving a data query request, performing current-limiting verification and distribution on the data query request, after receiving the data query request, analyzing and verifying the data query request by a target query server, when the verification is passed, decomposing the data query request into a plurality of subtasks based on metadata, respectively generating query statements for the plurality of subtasks, converting the query statements into corresponding api for each data source execution engine to perform data query, receiving query results fed back by each data source execution engine, and returning the query results to a user. The invention also relates to blockchain techniques, the metadata being storable in blockchain nodes. The invention also provides a data query device, equipment and a storage medium. By using the invention, the efficiency of data query can be improved.
Description
Technical Field
The invention relates to the technical field of big data in artificial intelligence, in particular to a data query method, a data query device, electronic equipment and a computer readable storage medium.
Background
In the big data service, when an offline data interface and a real-time data interface need to be provided externally, code development, test operation and version release need to be carried out in the conventional interface development, the relationship between the interface and a calling party needs to be maintained, the mapping relationship between the interface and a bottom table field needs to be maintained, the development period is long, and the development is complex. In this information explosion era, data to be processed by a big data team every day is often calculated by TB, traditional interface development cannot well achieve unified data export and unified query logic, and the demand is often changeable. In the current interface development, only from each system, data storage often exists in different storage media, such as oracle, redis, cassandra, elastic search, and the like, however, the existing development method has the following two problems:
1) when data required by requirements are stored in different data sources, how to query the data in different databases through a uniform interface is difficult, and a calling party does not need to know the data sources.
2) When the needed data bottom database does not exist, but can be calculated through data in different databases, the traditional method needs to newly build a table or a field, and the obtained result is stored in a certain database through data migration backup and then written statement calculation, so that the database has large change and much data redundancy, and the storage space and the development cost are increased.
Therefore, it is desirable to provide a data query method.
Disclosure of Invention
In view of the above, the present invention provides a data query method, device, electronic device and computer readable storage medium, which mainly aims to improve the efficiency of data query.
To achieve the above object, the present invention provides a data query method, including:
the method comprises the steps that an Nginx server receives a data query request sent by a user through a client, wherein the data query request carries encrypted fields to be queried, interface entry and signature verification data;
the Nginx server performs current-limiting verification on the data query request, determines a target query server corresponding to the data query request from the plurality of query servers based on a load balancing algorithm when the current-limiting verification of the data query request is judged to pass, and distributes the data query request to the target query server;
when the target query server receives the data query request, analyzing the encrypted field to be queried, the interface access parameter and the signature verification data, and verifying the data query request based on an analysis result and a preset verification rule;
when the data query request passes the verification, decomposing the data query request into a plurality of subtasks based on pre-managed metadata, respectively filling parameters of the plurality of subtasks into an sql query statement template, and generating sql query statements corresponding to the plurality of subtasks, wherein the metadata is stored in a block chain;
converting the sql query statements corresponding to the multiple subtasks into interface functions which can be identified by data source execution engines corresponding to the multiple subtasks, and distributing the interface functions to the corresponding data source execution engines by adopting multithreading so that the data source execution engines can invoke the interface function query data in a multithreading manner; and
and receiving data query sub-results corresponding to the plurality of sub-tasks returned by the data source execution engine, writing the data query sub-results into a result set, and returning and displaying the query result set to the user.
Further, to achieve the above object, the present invention provides a data inquiry apparatus including:
the receiving module is used for controlling the Nginx server to receive a data query request sent by a user through a client, wherein the data query request carries encrypted fields to be queried and interface participation and signature verification data;
the distribution module is used for controlling the Nginx server to perform current-limiting verification on the data query request, determining a target query server corresponding to the data query request from the plurality of query servers based on a load balancing algorithm when the current-limiting verification of the data query request is judged to pass, and distributing the data query request to the target query server;
the verification module is used for controlling the data query request to be verified based on the analysis result and a preset verification rule by analyzing the encrypted field to be queried, the interface access parameter and the verification data when the target query server receives the data query request;
the decomposition module is used for decomposing the data query request into a plurality of subtasks based on pre-managed metadata when the data query request passes verification, filling parameters of the subtasks into an sql query statement template respectively, and generating sql query statements corresponding to the subtasks, wherein the metadata is stored in a block chain;
the query module is used for converting the sql query statements corresponding to the subtasks into interface functions which can be identified by the data source execution engines corresponding to the subtasks, and distributing the interface functions to the corresponding data source execution engines by adopting multithreading so that the data source execution engines can invoke the interface functions to query data in a multithreading manner; and
and the feedback module is used for receiving the data query sub-results corresponding to the plurality of sub-tasks returned by the data source execution engine, writing the data query sub-results into a result set, and returning and displaying the query result set to the user.
In addition, to achieve the above object, the present invention also provides an electronic device including: the data query method comprises the following steps of storing a data query program which can run on the processor in a memory, and realizing any steps of the data query method when the data query program is executed by the processor.
In addition, to achieve the above object, the present invention further provides a computer-readable storage medium, in which a data query program is stored, and when the data query program is executed by a processor, any step in the data query method can be implemented.
According to the data query method, the data query device, the electronic equipment and the computer readable storage medium, after the data query request is received, the data query request is subjected to current-limiting verification and distribution, after the data query request is received by the target query server, the data query request is analyzed and verified, when the verification is passed, the data query request is decomposed into a plurality of subtasks, query statements are respectively generated for the subtasks, the query statements are converted into corresponding interface functions, data query is carried out by each data source execution engine, query results fed back by each data source are received, and the query results are returned to a user. The data query method comprises the steps of establishing a thread pool for different data sources in advance, decomposing data query tasks according to the data sources in the data query process, and distributing sub tasks to the thread pool corresponding to each data source according to multiple threads. By performing task decomposition on the data query request, thread pools are established for different data sources, and subtasks are concurrently executed, so that data query between different data sources is not influenced mutually, and the data query efficiency is improved.
Drawings
FIG. 1 is a flowchart of a data query method according to an embodiment of the present invention;
FIG. 2 is a block diagram of a data query device according to a preferred embodiment of the present invention;
FIG. 3 is a diagram of an alternative hardware architecture of the electronic device of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
The invention provides a data query method. The method is applied to a server cluster, and the server cluster comprises a plurality of query servers and a Nginx server.
Referring to fig. 1, a flowchart of a data query method according to an embodiment of the present invention is shown.
In this embodiment, the data query method includes: step S1-step S6.
Step S1, the Nginx server receives a data query request sent by a user through a client, wherein the data query request carries encrypted fields to be queried and interface access and signature verification data;
for example, the user may be a user of an APP, a non-technical person such as a service person, and sends a data query request through the APP installed on the client.
Wherein the signature verification data is an encrypted string, e.g., an MD5 value, obtained by performing an encryption calculation on a URL, a caller ID (e.g., a fingered pin), a timestamp, a caller key, an interface name (e.g., structure), and the like.
Step S2, the Nginx server performs current-limiting verification on the data query request, determines a target query server corresponding to the data query request from the plurality of query servers based on a load balancing algorithm when the current-limiting verification of the data query request is judged to pass, and distributes the data query request to the target query server;
specifically, the performing the current-limiting verification on the data query request includes:
determining a calling party corresponding to the data query request, acquiring the real-time request number of the calling party, and judging whether the real-time request number of the calling party is less than or equal to a first preset threshold value; and
and when the real-time request number of the calling party is smaller than or equal to a first preset threshold value, acquiring the real-time request number of an interface corresponding to the calling party, and when the real-time request number of the interface is smaller than or equal to a second preset threshold value, judging that the data query request current limiting verification is passed.
The caller may include, but is not limited to, App name of the user sending out the data query request.
It will be appreciated that each caller corresponds to one interface, but that one interface may correspond to one or more callers. It should be noted that, the flow control parameters of the interface and its corresponding caller, that is, the access restriction, need to be configured in advance, include: rate is the number of concurrencies that the interface can reach, and burst is the concurrency buffer. The first preset threshold is the maximum concurrency number configured for the caller, for example, 1000, and the second preset threshold is the maximum concurrency number configured for the interface, for example, 2200, that is, when the real-time number of the caller/the real-time number of the interface corresponding to the data query request are respectively below 1000/2200, the interface returns data in real time through the request current-limiting verification.
Further, the performing current-limiting verification on the data query request further includes:
when the real-time request number of the calling party is larger than a first preset threshold value and smaller than or equal to a third preset threshold value, and the real-time request number of the interface is larger than a second preset threshold value and smaller than or equal to a fourth preset threshold value, writing the data query request into a buffer queue; or
And when the real-time request number of the calling party is greater than a third preset threshold value or the real-time request number of the interface is greater than a fourth preset threshold value, judging that the data query request current limiting verification fails.
The third preset threshold is the sum of the buffer configured for the caller and the maximum concurrency number, for example, the buffer is 500 and the buffer is 1500, the fourth preset threshold is the sum of the buffer configured for the interface and the maximum concurrency number, for example, the buffer is 800 and the buffer is 3000, that is, when the real-time request number of the caller/the real-time request number of the interface corresponding to the data query request is respectively below 1500/3000, the data query request is written into the buffer queue, and when the real-time request number of the caller/the real-time request number of the interface corresponding to the data query request exceeds 1500/3000, it is determined that the current-limiting verification of the data query request fails, and error-reporting data is returned.
Taking the calling party as an example, when the concurrence is more than 1000 and less than 1500, the current request enters the buffer according to the request time to queue and wait for the idle of the thread, and the data is returned. When the concurrency is more than 1500, the request which is not processed in time can directly return the abnormal information with the exceeded concurrency number to the calling party.
In this embodiment, after it is determined that the data query request passes the current-limiting verification, the real-time task number of each query server in the server cluster is read, the query server with the smallest task number is selected as the target query server, and the data query request is sent to the target query server.
In other embodiments, load balancing may also be performed according to a polling method, a weighted polling method, a random method, a weighted random method, and a consistent hash method, so as to determine a target query server corresponding to the data query request, which is not described herein again.
Step S3, when the target query server receives the data query request, analyzing the encrypted field to be queried, the interface access parameter and the signature verification data, and verifying the data query request based on the analysis result and a preset verification rule;
in this embodiment, the interface enrollment and caller id data is an encrypted string obtained by performing encryption calculation through the MD5 encryption algorithm. And decrypting the encrypted character string through a corresponding decryption algorithm to obtain interface access parameter and signature verification data in a plaintext form.
Specifically, the verifying the data query request based on the analysis result and the preset verification rule includes:
determining a table name corresponding to the field to be queried based on the metadata, determining the table name corresponding to the caller in the data query request, acquiring preset mapping data of the caller and table authority, and judging whether the caller passes the authority verification;
acquiring the interface access parameter, checking whether the interface access parameter contains the necessary filling parameter of the interface, and judging whether the data query request passes parameter verification according to the checking result; and
and when the data query request passes the permission verification and the parameter verification at the same time, judging that the data query request passes the verification, and if not, returning early warning information.
The preset mapping data of the caller and the table authority can be understood as an authority white list of different tables. For example, the caller corresponding to table a is caller Ai, and the caller corresponding to table B is caller Bi, …. The field name, the type, the table name, the data source, the IT caliber, the service caliber, the modifier and other information are managed in the pre-managed metadata, so that one or more table names corresponding to the data query request can be determined based on the field to be queried and the metadata, then authorized callers corresponding to one or more table names are queried from the mapping data respectively, whether data matched with the current caller exists or not is judged, if yes, the authority verification is judged to be passed, if not, the authority verification is not passed, and early warning information is returned.
In the interface editing process, the information to be filled in is checked by the entry. If the user must fill or select, parameters such as flow control, timeout time, dynamic parameters (for example, searching for an identity card number through a mobile phone number and completing parameter conversion) and the like are also set. The entry check is filled in a json format, key is a parameter of the entry, required in value is 1, the entry is necessary to be filled, and 0 is optional to be filled. And judging whether the parameters which need to be filled are filled, if so, passing parameter verification, and if the parameters which are not written exist, failing the parameter verification, and returning early warning information.
Further, when the table name is queried through the metadata, if the field to be queried is not managed by the metadata, the error information is directly returned.
Through metadata management and authority verification, data security is guaranteed.
Step S4, when the data query request passes the verification, decomposing the data query request into a plurality of subtasks based on the pre-managed metadata, respectively filling the parameters of the plurality of subtasks into an sql query statement template, and generating sql query statements corresponding to the plurality of subtasks;
specifically, the decomposing the data query request into a plurality of subtasks based on the pre-managed metadata includes:
when the data query request comprises a plurality of fields to be queried, respectively determining a table name and a data source corresponding to each field to be queried based on the metadata; and
and decomposing the data query request into a plurality of subtasks according to the data source corresponding to the field to be queried.
It is emphasized that the metadata may also be stored in a node of a blockchain in order to further ensure privacy and security of the metadata. It should be noted that the data corresponding to different fields to be queried is stored in different data sources, for example, cassandra, oracle, redis, es, and the like.
For example, the data query request R includes: a field to be queried R1, a field to be queried R2, a field to be queried R3, and a field to be queried R4, which respectively correspond to the database C, the database O, the database C, the database R, and the like, and then, according to the databases, subtasks corresponding to the databases are generated: TC, TO and TR, wherein the TC comprises fields R1 and R3 TO be queried.
The task decomposition is carried out according to the data source corresponding to the query statement configured in the background, the query field is not required to be transmitted by a calling party, after one interface is configured, the data source is automatically decomposed into different data source execution engines according to the dimensionality of the data source, different tables of the same subsequent data source cannot be further decomposed, and the execution tasks of the different tables are still executed in parallel.
Wherein, the sql query statement template is a select field from table name to be queried; the parameters of each subtask include: and filling parameters corresponding to each subtask into an sql statement template in sequence to obtain a standard sql statement corresponding to each subtask. And the number of the standard sql statements corresponds to the number of the fields to be queried in the subtasks.
Step S5, converting the sql query statements corresponding to the multiple subtasks into interface functions recognizable by the data source execution engines corresponding to the multiple subtasks, and distributing the interface functions to the corresponding data source execution engines by adopting multithreading so that the data source execution engines can invoke the interface function query data in a multithreading manner;
step S6, receiving the data query sub-results corresponding to the plurality of sub-tasks returned by the data source execution engine, writing the data query sub-results into a result set, and returning and displaying the query result set to the user.
Specifically, the task executor uses the sql engine to analyze sql for different data sources and converts the sql into an interface function (api) that can be directly called by different data sources. It can be understood that different data sources (databases) have respective execution engines, and the api that can be directly called by each execution engine is different, for example, Oracle and Mysq can directly call and execute sql, while Redis, Cassandra and ElasticSearch cannot recognize sql, so that for Redis, Cassandra and ElasticSearch, the standard sql needs to be automatically converted into the api of the corresponding data source query, thereby improving the efficiency of data query.
And multiple sub tasks are simultaneously distributed to corresponding execution engines through multiple threads, so that the time consumed by task distribution can be saved.
In other embodiments, different data sources bind different thread pools. The interface query is submitted to the corresponding thread pool aiming at the data query of different data sources, so that the query of each data source can be executed in the respective independent thread pool, the query of other databases cannot be influenced due to the large query pressure of a certain database, the thread quantity of the thread pool can be increased in real time, and the dynamic capacity expansion can be ensured after the concurrency quantity rises.
When the interface has a plurality of query sql or the data required by the interface is from different databases, after the interface is configured in the background, and the caller calls the interface, a plurality of query statements are executed in parallel in a multithread manner, the interface query time is not the time-consuming accumulation of the plurality of query statements, but depends on the statement with the longest time consumption, the statement result which is executed first is written into the result set, then the interface waits until all the query statements are executed, and the interface returns all the data.
In addition, aiming at the condition that interface data needs to inquire a plurality of databases, after the tasks are submitted to a thread pool, unified management and unified distribution of tasks executed by each data source are automatically carried out, standard sql can be automatically converted into api corresponding to data source inquiry, and after the tasks of each data source are executed, returned results are collected to a result set by a system in a unified mode.
In other embodiments, when the field to be queried includes a preset type field, the data querying method further includes:
acquiring bottom layer dependent data of the preset type field, and determining a plurality of subfields and logical relations corresponding to the bottom layer dependent data;
generating sql query statements corresponding to the plurality of subfields, and executing step S5;
packaging data returned by each data source execution engine into a uniform format to form a uniform memory temporary table;
and determining a secondary calculation sql statement according to the logical relationship, calculating data in the temporary memory table by using the secondary calculation sql statement, and taking an obtained calculation result as a query result of the preset type field.
For example, the preset type field is a field that cannot be directly read from a database and needs secondary calculation, for example, net assets-total assets-liabilities.
In this embodiment, for special requirements, for example, a situation that an interface needs to query a data base database, and data can be obtained only by performing calculation through a plurality of fields in different databases, this embodiment provides a secondary calculation service, and this embodiment uses Apache call (which is an SQL parsing and optimizing engine independent of storage and execution), first queries data dependent on the interface data to be provided, edits a plurality of query statements, stores results into a result set, automatically determines which data need to be subjected to secondary calculation by a system through hierarchical determination of execution SQL statement table structure dependence, modifies call, packages data queried in different base databases into a uniform format, and inputs the uniform format into call, for example, List < Map < String, Object > format, forms a uniform temporary memory table, and then performs secondary calculation through secondary calculation SQL configured in the background, and packaging the query result and the secondary calculation result according to a data structure defined by the interface and returning data.
When the required data base layer database does not exist, but can be calculated through data in different databases, the traditional method in the prior art is to create a table or a field, and store the obtained result in a certain database through data migration backup and then statement writing calculation. The embodiment can be inquired out only by being configured in the background, so that the change of the database table structure is reduced, the redundancy of data is also reduced, the space is saved, and the development cost can be saved.
The invention also provides a data query device.
Fig. 2 is a block diagram of a data query device according to a preferred embodiment of the present invention.
The data query apparatus 10 according to this embodiment may include, according to the implemented functions: module 110-module 170. A module according to the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and that can perform a fixed function, and that are stored in a memory of the electronic device.
In an embodiment of the data query device of the present invention, the device only includes: modules 110-160, the functions for each module/unit are as follows:
a receiving module 110, configured to control the Nginx server to receive a data query request sent by a user through a client, where the data query request carries an encrypted field to be queried and interface entry and signature verification data;
for example, the user may be a user of an APP, a non-technical person such as a service person, and sends a data query request through the APP installed on the client.
Wherein the signature verification data is an encrypted string, e.g., an MD5 value, obtained by performing an encryption calculation on a URL, a caller ID (e.g., a fingered pin), a timestamp, a caller key, an interface name (e.g., structure), and the like.
The distribution module 120 is configured to control the Nginx server to perform flow-limiting verification on the data query request, determine, when it is determined that the flow-limiting verification of the data query request passes, a target query server corresponding to the data query request from the plurality of query servers based on a load balancing algorithm, and distribute the data query request to the target query server;
specifically, the performing the current-limiting verification on the data query request includes:
determining a calling party corresponding to the data query request, acquiring the real-time request number of the calling party, and judging whether the real-time request number of the calling party is less than or equal to a first preset threshold value; and
and when the real-time request number of the calling party is smaller than or equal to a first preset threshold value, acquiring the real-time request number of an interface corresponding to the calling party, and when the real-time request number of the interface is smaller than or equal to a second preset threshold value, judging that the data query request current limiting verification is passed.
The caller may include, but is not limited to, App name of the user sending out the data query request.
It will be appreciated that each caller corresponds to one interface, but that one interface may correspond to one or more callers. It should be noted that, the flow control parameters of the interface and its corresponding caller, that is, the access restriction, need to be configured in advance, include: rate is the number of concurrencies that the interface can reach, and burst is the concurrency buffer. The first preset threshold is the maximum concurrency number configured for the caller, for example, 1000, and the second preset threshold is the maximum concurrency number configured for the interface, for example, 2200, that is, when the real-time number of the caller/the real-time number of the interface corresponding to the data query request are respectively below 1000/2200, the interface returns data in real time through the request current-limiting verification.
Further, the performing current-limiting verification on the data query request further includes:
when the real-time request number of the calling party is larger than a first preset threshold value and smaller than or equal to a third preset threshold value, and the real-time request number of the interface is larger than a second preset threshold value and smaller than or equal to a fourth preset threshold value, writing the data query request into a buffer queue; or
And when the real-time request number of the calling party is greater than a third preset threshold value or the real-time request number of the interface is greater than a fourth preset threshold value, judging that the data query request current limiting verification fails.
The third preset threshold is the sum of the buffer configured for the caller and the maximum concurrency number, for example, the buffer is 500 and the buffer is 1500, the fourth preset threshold is the sum of the buffer configured for the interface and the maximum concurrency number, for example, the buffer is 800 and the buffer is 3000, that is, when the real-time request number of the caller/the real-time request number of the interface corresponding to the data query request is respectively below 1500/3000, the data query request is written into the buffer queue, and when the real-time request number of the caller/the real-time request number of the interface corresponding to the data query request exceeds 1500/3000, it is determined that the current-limiting verification of the data query request fails, and error-reporting data is returned.
Taking the calling party as an example, when the concurrence is more than 1000 and less than 1500, the current request enters the buffer according to the request time to queue and wait for the idle of the thread, and the data is returned. When the concurrency is more than 1500, the request which is not processed in time can directly return the abnormal information with the exceeded concurrency number to the calling party.
In this embodiment, after it is determined that the data query request passes the current-limiting verification, the real-time task number of each query server in the server cluster is read, the query server with the smallest task number is selected as the target query server, and the data query request is sent to the target query server.
In other embodiments, load balancing may also be performed according to a polling method, a weighted polling method, a random method, a weighted random method, and a consistent hash method, so as to determine a target query server corresponding to the data query request, which is not described herein again.
The verification module 130 is configured to control, when the target query server receives the data query request, to analyze the encrypted field to be queried, the interface entry parameter and the signature verification data, and verify the data query request based on an analysis result and a preset verification rule;
in this embodiment, the interface enrollment and caller id data is an encrypted string obtained by performing encryption calculation through the MD5 encryption algorithm. And decrypting the encrypted character string through a corresponding decryption algorithm to obtain interface access parameter and signature verification data in a plaintext form.
Specifically, the verifying the data query request based on the analysis result and the preset verification rule includes:
determining a table name corresponding to the field to be queried based on the metadata, determining the table name corresponding to the caller in the data query request, acquiring preset mapping data of the caller and table authority, and judging whether the caller passes the authority verification;
acquiring the interface access parameter, checking whether the interface access parameter contains the necessary filling parameter of the interface, and judging whether the data query request passes parameter verification according to the checking result; and
and when the data query request passes the permission verification and the parameter verification at the same time, judging that the data query request passes the verification, and if not, returning early warning information.
The preset mapping data of the caller and the table authority can be understood as an authority white list of different tables. For example, the caller corresponding to table a is caller Ai, and the caller corresponding to table B is caller Bi, …. The field name, the type, the table name, the data source, the IT caliber, the service caliber, the modifier and other information are managed in the pre-managed metadata, so that one or more table names corresponding to the data query request can be determined based on the field to be queried and the metadata, then authorized callers corresponding to one or more table names are queried from the mapping data respectively, whether data matched with the current caller exists or not is judged, if yes, the authority verification is judged to be passed, if not, the authority verification is not passed, and early warning information is returned.
In the interface editing process, the information to be filled in is checked by the entry. If the user must fill or select, parameters such as flow control, timeout time, dynamic parameters (for example, searching for an identity card number through a mobile phone number and completing parameter conversion) and the like are also set. The entry check is filled in a json format, key is a parameter of the entry, required in value is 1, the entry is necessary to be filled, and 0 is optional to be filled. And judging whether the parameters which need to be filled are filled, if so, passing parameter verification, and if the parameters which are not written exist, failing the parameter verification, and returning early warning information.
Further, when the table name is queried through the metadata, if the field to be queried is not managed by the metadata, the error information is directly returned.
Through metadata management and authority verification, data security is guaranteed.
A decomposition module 140, configured to decompose the data query request into multiple subtasks based on pre-managed metadata when the data query request passes verification, and fill parameters of the multiple subtasks into an sql query statement template, respectively, to generate sql query statements corresponding to the multiple subtasks;
specifically, the decomposing the data query request into a plurality of subtasks based on the pre-managed metadata includes:
when the data query request comprises a plurality of fields to be queried, respectively determining a table name and a data source corresponding to each field to be queried based on the metadata;
and decomposing the data query request into a plurality of subtasks according to the data source corresponding to the field to be queried.
It is emphasized that the metadata may also be stored in a node of a blockchain in order to further ensure privacy and security of the metadata. It should be noted that the data corresponding to different fields to be queried is stored in different data sources, for example, cassandra, oracle, redis, es, and the like.
For example, the data query request R includes: a field to be queried R1, a field to be queried R2, a field to be queried R3, and a field to be queried R4, which respectively correspond to the database C, the database O, the database C, the database R, and the like, and then, according to the databases, subtasks corresponding to the databases are generated: TC, TO and TR, wherein the TC comprises fields R1 and R3 TO be queried.
The task decomposition is carried out according to the data source corresponding to the query statement configured in the background, the query field is not required to be transmitted by a calling party, after one interface is configured, the data source is automatically decomposed into different data source execution engines according to the dimensionality of the data source, different tables of the same subsequent data source cannot be further decomposed, and the execution tasks of the different tables are still executed in parallel.
Wherein, the sql query statement template is a select field from table name to be queried; the parameters of each subtask include: and filling parameters corresponding to each subtask into an sql statement template in sequence to obtain a standard sql statement corresponding to each subtask. And the number of the standard sql statements corresponds to the number of the fields to be queried in the subtasks.
The query module 150 is configured to convert the sql query statements corresponding to the multiple subtasks into interface functions that can be recognized by data source execution engines corresponding to the multiple subtasks, and distribute the interface functions to corresponding data source execution engines by using multiple threads, so that the data source execution engines invoke the interface functions to query data in parallel by using multiple threads;
specifically, the task executor uses the sql engine to analyze sql for different data sources and converts the sql into an interface function (api) that can be directly called by different data sources. It can be understood that different data sources (databases) have respective execution engines, and the api that can be directly called by each execution engine is different, for example, Oracle and Mysq can directly call and execute sql, while Redis, Cassandra and ElasticSearch cannot recognize sql, so that for Redis, Cassandra and ElasticSearch, the standard sql needs to be automatically converted into the api of the corresponding data source query, thereby improving the efficiency of data query.
And multiple sub tasks are simultaneously distributed to corresponding execution engines through multiple threads, so that the time consumed by task distribution can be saved.
In other embodiments, different data sources bind different thread pools. The interface query is submitted to the corresponding thread pool aiming at the data query of different data sources, so that the query of each data source can be executed in the respective independent thread pool, the query of other databases cannot be influenced due to the large query pressure of a certain database, the thread quantity of the thread pool can be increased in real time, and the dynamic capacity expansion can be ensured after the concurrency quantity rises.
And the feedback module 160 is configured to receive the data query sub-results corresponding to the plurality of sub-tasks returned by the data source execution engine, write the data query sub-results into a result set, and return and display the query result set to the user.
When the interface has a plurality of query sql or the data required by the interface is from different databases, after the interface is configured in the background, and the caller calls the interface, a plurality of query statements are executed in parallel in a multithread manner, the interface query time is not the time-consuming accumulation of the plurality of query statements, but depends on the statement with the longest time consumption, the statement result which is executed first is written into the result set, then the interface waits until all the query statements are executed, and the interface returns all the data.
In addition, aiming at the condition that interface data needs to inquire a plurality of databases, after the tasks are submitted to a thread pool, unified management and unified distribution of tasks executed by each data source are automatically carried out, standard sql can be automatically converted into api corresponding to data source inquiry, and after the tasks of each data source are executed, returned results are collected to a result set by a system in a unified mode.
In another embodiment of the data query apparatus of the present invention, the apparatus comprises: modules 110-170, wherein modules 110-160 are substantially similar to the embodiments described above, and the calculation module 170 is configured to:
acquiring bottom layer dependent data of the preset type field, and determining a plurality of subfields and logical relations corresponding to the bottom layer dependent data;
generating sql query statements corresponding to the plurality of subfields;
packaging data returned by each data source execution engine into a uniform format to form a uniform memory temporary table;
and determining a secondary calculation sql statement according to the logical relationship, calculating data in the temporary memory table by using the secondary calculation sql statement, and taking an obtained calculation result as a query result of the preset type field.
For example, the preset type field is a field that cannot be directly read from a database and needs secondary calculation, for example, net assets-total assets-liabilities.
In this embodiment, for special requirements, for example, a situation that an interface needs to query a data base database, and data can be obtained only by performing calculation through a plurality of fields in different databases, this embodiment provides a secondary calculation service, and this embodiment uses Apache call (which is an SQL parsing and optimizing engine independent of storage and execution), first queries data dependent on the interface data to be provided, edits a plurality of query statements, stores results into a result set, automatically determines which data need to be subjected to secondary calculation by a system through hierarchical determination of execution SQL statement table structure dependence, modifies call, packages data queried in different base databases into a uniform format, and inputs the uniform format into call, for example, List < Map < String, Object > format, forms a uniform temporary memory table, and then performs secondary calculation through secondary calculation SQL configured in the background, and packaging the query result and the secondary calculation result according to a data structure defined by the interface and returning data.
When the required data base layer database does not exist, but can be calculated through data in different databases, the traditional method in the prior art is to create a table or a field, and store the obtained result in a certain database through data migration backup and then statement writing calculation. The embodiment can be inquired out only by being configured in the background, so that the change of the database table structure is reduced, the redundancy of data is also reduced, the space is saved, and the development cost can be saved.
The embodiment of the invention also provides the electronic equipment.
Referring to fig. 3, a diagram of an alternative hardware architecture of the electronic device of the present invention is shown.
In the embodiment, the application electronic device 1 may include, but is not limited to, a memory 11, a processor 12, and a network interface 13, which may be communicatively connected to each other through a system bus.
The memory 11 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, and the like. The memory 11 may in some embodiments be an internal storage unit of the electronic device 1, e.g. a hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device 1. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device 1.
The memory 11 may be used not only to store application software installed in the electronic device 1 and various types of data, such as the data query program 110, but also to temporarily store data that has been output or is to be output.
The processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor or other data Processing chip in some embodiments, and is used for executing program codes stored in the memory 11 or Processing data, such as the data query program 110.
The network interface 13 may optionally comprise a standard wired interface, a wireless interface (e.g. WI-FI interface), and is typically used for establishing a communication connection between the electronic device 1 and other electronic devices, for example, a terminal (not shown).
It is noted that fig. 3 only shows the electronic device 1 with components 11-13, and that the structure shown in fig. 3 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or a combination of certain components, or a different arrangement of components, as will be appreciated by a person skilled in the art.
Optionally, the electronic device 1 may further comprise a user interface, the user interface may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface may further comprise a standard wired interface, a wireless interface.
Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an Organic Light-Emitting Diode (OLED) touch screen, or the like. The display, which may also be referred to as a display screen or display unit, is used for displaying information processed in the electronic device 1 and for displaying a visualized user interface.
In the embodiment of the electronic device 1 shown in fig. 3, the memory 11 as a computer storage medium stores the program code of the data query program 110, and the processor 12, when executing the program code of the data query program 10, may implement the following steps:
the Nginx server receives a data query request sent by a user through a client, wherein the data query request carries encrypted fields to be queried, interface entry parameter and signature verification data;
the Nginx server performs current-limiting verification on the data query request, determines a target query server corresponding to the data query request from the plurality of query servers based on a load balancing algorithm when the current-limiting verification of the data query request is judged to pass, and distributes the data query request to the target query server;
when the target query server receives the data query request, analyzing the encrypted field to be queried, the interface access parameter and the signature verification data, and verifying the data query request based on an analysis result and a preset verification rule;
when the data query request passes the verification, decomposing the data query request into a plurality of subtasks based on pre-managed metadata, respectively filling parameters of the plurality of subtasks into an sql query statement template, and generating sql query statements corresponding to the plurality of subtasks;
converting the sql query statements corresponding to the multiple subtasks into interface functions which can be identified by data source execution engines corresponding to the multiple subtasks, and distributing the interface functions to the corresponding data source execution engines by adopting multithreading so that the data source execution engines can invoke the interface function query data in a multithreading manner; and
and receiving data query sub-results corresponding to the plurality of sub-tasks returned by the data source execution engine, writing the data query sub-results into a result set, and returning and displaying the query result set to the user.
The specific implementation of the electronic device of the present invention is substantially the same as the method embodiments described above, and will not be described herein again.
Further, the integrated modules/units of the electronic device 1, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. The computer-usable storage medium may store an operating system, an application program required for at least one function, such as the data query program 110. The computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A data query method is applied to a server cluster, wherein the server cluster comprises a plurality of query servers and Nginx servers, and the method comprises the following steps:
the Nginx server receives a data query request sent by a user through a client, wherein the data query request carries encrypted fields to be queried, interface entry parameter and signature verification data;
the Nginx server performs current-limiting verification on the data query request, determines a target query server corresponding to the data query request from the plurality of query servers based on a load balancing algorithm when the current-limiting verification of the data query request is judged to pass, and distributes the data query request to the target query server;
when the target query server receives the data query request, analyzing the encrypted field to be queried, the interface access parameter and the signature verification data, and verifying the data query request based on an analysis result and a preset verification rule;
when the data query request passes the verification, decomposing the data query request into a plurality of subtasks based on pre-managed metadata, respectively filling parameters of the plurality of subtasks into an sql query statement template, and generating sql query statements corresponding to the plurality of subtasks;
converting the sql query statements corresponding to the multiple subtasks into interface functions which can be identified by data source execution engines corresponding to the multiple subtasks, and distributing the interface functions to the corresponding data source execution engines by adopting multithreading so that the data source execution engines can invoke the interface function query data in a multithreading manner; and
and receiving data query sub-results corresponding to the plurality of sub-tasks returned by the data source execution engine, writing the data query sub-results into a result set, and returning and displaying the query result set to the user.
2. The data query method of claim 1, wherein the performing current-limited validation on the data query request comprises:
determining a calling party corresponding to the data query request, acquiring the real-time request number of the calling party, and judging whether the real-time request number of the calling party is less than or equal to a first preset threshold value; and
and when the real-time request number of the calling party is smaller than or equal to a first preset threshold value, acquiring the real-time request number of an interface corresponding to the calling party, and when the real-time request number of the interface is smaller than or equal to a second preset threshold value, judging that the data query request current limiting verification is passed.
3. The data query method of claim 2, wherein the performing current-limited validation on the data query request further comprises:
when the real-time request number of the calling party is larger than a first preset threshold value and smaller than or equal to a third preset threshold value, and the real-time request number of the interface is larger than a second preset threshold value and smaller than or equal to a fourth preset threshold value, writing the data query request into a buffer queue; or
And when the real-time request number of the calling party is greater than a third preset threshold value or the real-time request number of the interface is greater than a fourth preset threshold value, judging that the data query request current limiting verification fails.
4. The data query method of claim 1, wherein the verifying the data query request based on the analysis result and a preset verification rule comprises:
determining a table name corresponding to the field to be queried based on the metadata, determining the table name corresponding to the caller in the data query request, acquiring preset mapping data of the caller and table authority, and judging whether the caller passes the authority verification;
acquiring the interface access parameter, checking whether the interface access parameter contains the necessary filling parameter of the interface, and judging whether the data query request passes parameter verification according to the checking result; and
and when the data query request passes the permission verification and the parameter verification at the same time, judging that the data query request passes the verification.
5. The data query method of claim 1, wherein the decomposing the data query request into a plurality of subtasks based on pre-managed metadata comprises:
when the data query request comprises a plurality of fields to be queried, respectively determining a table name and a data source corresponding to each field to be queried based on the metadata; and
and decomposing the data query request into a plurality of subtasks according to the data source corresponding to the field to be queried.
6. The data query method of claim 1, wherein the load balancing algorithm comprises: a minimum number of connections method, a round robin method, a weighted round robin method, a random method, a weighted random method or a consistent hashing method.
7. The data query method according to any one of claims 1 to 6, wherein when the field to be queried includes a preset type field, the data query method further includes:
acquiring bottom layer dependent data of the preset type field, and determining a plurality of subfields and logical relations corresponding to the bottom layer dependent data;
generating sql query statements corresponding to the subfields, converting the sql query statements corresponding to the subfields into interface functions recognizable by data source execution engines corresponding to the subfields, and distributing the interface functions to the corresponding data source execution engines by adopting multithreading so that the data source execution engines can call the interface function query data in a multithreading manner;
packaging data returned by each data source execution engine into a uniform format to form a uniform memory temporary table; and
and determining a secondary calculation sql statement according to the logical relationship, calculating data in the temporary memory table by using the secondary calculation sql statement, and taking an obtained calculation result as a query result of the preset type field.
8. A data query apparatus, comprising:
the receiving module is used for controlling the Nginx server to receive a data query request sent by a user through a client, wherein the data query request carries encrypted fields to be queried and interface participation and signature verification data;
the distribution module is used for controlling the Nginx server to perform current-limiting verification on the data query request, determining a target query server corresponding to the data query request from the plurality of query servers based on a load balancing algorithm when the current-limiting verification of the data query request is judged to pass, and distributing the data query request to the target query server;
the verification module is used for controlling the data query request to be verified based on the analysis result and a preset verification rule by analyzing the encrypted field to be queried, the interface access parameter and the verification data when the target query server receives the data query request;
the decomposition module is used for decomposing the data query request into a plurality of subtasks based on pre-managed metadata when the data query request passes verification, filling parameters of the subtasks into an sql query statement template respectively, and generating sql query statements corresponding to the subtasks;
the query module is used for converting the sql query statements corresponding to the subtasks into interface functions which can be identified by the data source execution engines corresponding to the subtasks, and distributing the interface functions to the corresponding data source execution engines by adopting multithreading so that the data source execution engines can invoke the interface functions to query data in a multithreading manner; and
and the feedback module is used for receiving the data query sub-results corresponding to the plurality of sub-tasks returned by the data source execution engine, writing the data query sub-results into a result set, and returning and displaying the query result set to the user.
9. An electronic device, comprising a memory and a processor, wherein the memory stores a data query program operable on the processor, and the data query program, when executed by the processor, implements the steps of the data query method according to any one of claims 1 to 7.
10. A computer-readable storage medium, in which a data query program is stored, and when the data query program is executed by a processor, the data query program can implement the data query method according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010593984.4A CN111782923A (en) | 2020-06-24 | 2020-06-24 | Data query method and device, electronic equipment and storage medium |
| PCT/CN2020/122107 WO2021151316A1 (en) | 2020-06-24 | 2020-10-20 | Method and apparatus for querying data, electronic device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010593984.4A CN111782923A (en) | 2020-06-24 | 2020-06-24 | Data query method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111782923A true CN111782923A (en) | 2020-10-16 |
Family
ID=72760876
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010593984.4A Pending CN111782923A (en) | 2020-06-24 | 2020-06-24 | Data query method and device, electronic equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111782923A (en) |
| WO (1) | WO2021151316A1 (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112306648A (en) * | 2020-10-29 | 2021-02-02 | 平安科技（深圳）有限公司 | Transaction calling method and device, electronic equipment and storage medium |
| CN112597162A (en) * | 2020-12-25 | 2021-04-02 | 平安银行股份有限公司 | Data set acquisition method, system, device and storage medium |
| CN112597187A (en) * | 2021-03-03 | 2021-04-02 | 北京冠新医卫软件科技有限公司 | GraphQL query method and device based on FHIR standard |
| CN112669076A (en) * | 2020-12-30 | 2021-04-16 | 平安证券股份有限公司 | Data distribution method based on rule engine, server and storage medium |
| CN112860737A (en) * | 2021-03-11 | 2021-05-28 | 中国平安财产保险股份有限公司 | Data query method and device, electronic equipment and readable storage medium |
| CN113094387A (en) * | 2021-04-08 | 2021-07-09 | 杭州数梦工场科技有限公司 | Data query method and device, electronic equipment and machine-readable storage medium |
| CN113094401A (en) * | 2021-05-07 | 2021-07-09 | 北京京东振世信息技术有限公司 | Target source data generation method, device, equipment and storage medium |
| WO2021151316A1 (en) * | 2020-06-24 | 2021-08-05 | 平安科技（深圳）有限公司 | Method and apparatus for querying data, electronic device, and storage medium |
| CN113239044A (en) * | 2021-05-18 | 2021-08-10 | 安翰科技(武汉)股份有限公司 | Management device and application system of SQL records for query |
| CN113239069A (en) * | 2021-05-18 | 2021-08-10 | 上海跨境电子商务公共服务有限公司 | Data query method and data query system |
| CN113284586A (en) * | 2021-05-24 | 2021-08-20 | 康键信息技术（深圳）有限公司 | Medical service processing method, device, equipment and storage medium |
| CN113672651A (en) * | 2021-08-24 | 2021-11-19 | 杭州海康威视数字技术股份有限公司 | Task execution method and device and electronic equipment |
| CN113742385A (en) * | 2021-09-15 | 2021-12-03 | 第四范式（北京）技术有限公司 | Data query method and device |
| CN113760993A (en) * | 2021-08-18 | 2021-12-07 | 北京健康之家科技有限公司 | Service data query method and device and electronic equipment |
| CN113901088A (en) * | 2021-10-14 | 2022-01-07 | 中国银行股份有限公司 | Parameter checking method, device and equipment |
| CN114666165A (en) * | 2022-05-20 | 2022-06-24 | 深圳华锐分布式技术股份有限公司 | Query request response method, device, equipment and medium |
| WO2023115252A1 (en) * | 2021-12-20 | 2023-06-29 | Boe Technology Group Co., Ltd. | Data query method, data query apparatus, and computer-program product |
| CN116680061A (en) * | 2023-08-02 | 2023-09-01 | 腾讯科技（深圳）有限公司 | Task execution method, device, equipment and storage medium |
| CN116955416A (en) * | 2023-09-18 | 2023-10-27 | 腾讯科技（深圳）有限公司 | Data processing method, system, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180004556A1 (en) * | 2016-06-29 | 2018-01-04 | Amazon Technologies, Inc. | Adjusting variable limit on concurrent code executions based on communication between frontends |
| CN108073465A (en) * | 2017-12-29 | 2018-05-25 | 中国平安人寿保险股份有限公司 | Dynamic current limiting method, Nginx servers, storage medium and device |
| CN110191063A (en) * | 2019-06-13 | 2019-08-30 | 北京百度网讯科技有限公司 | Processing method, device, equipment and the storage medium of service request |
| CN110516142A (en) * | 2019-08-29 | 2019-11-29 | 深圳前海微众银行股份有限公司 | Data query method, apparatus, equipment and medium |
| CN110580258A (en) * | 2019-09-20 | 2019-12-17 | 中国银行股份有限公司 | big data free query method and device |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9729525B1 (en) * | 2015-06-29 | 2017-08-08 | EMC IP Holding Company LLC | Secure data analytics |
| CN110990473B (en) * | 2019-11-28 | 2023-11-03 | 京东科技信息技术有限公司 | Tag data processing system and method |
| CN111782923A (en) * | 2020-06-24 | 2020-10-16 | 平安科技（深圳）有限公司 | Data query method and device, electronic equipment and storage medium |
-
2020
- 2020-06-24 CN CN202010593984.4A patent/CN111782923A/en active Pending
- 2020-10-20 WO PCT/CN2020/122107 patent/WO2021151316A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180004556A1 (en) * | 2016-06-29 | 2018-01-04 | Amazon Technologies, Inc. | Adjusting variable limit on concurrent code executions based on communication between frontends |
| CN108073465A (en) * | 2017-12-29 | 2018-05-25 | 中国平安人寿保险股份有限公司 | Dynamic current limiting method, Nginx servers, storage medium and device |
| CN110191063A (en) * | 2019-06-13 | 2019-08-30 | 北京百度网讯科技有限公司 | Processing method, device, equipment and the storage medium of service request |
| CN110516142A (en) * | 2019-08-29 | 2019-11-29 | 深圳前海微众银行股份有限公司 | Data query method, apparatus, equipment and medium |
| CN110580258A (en) * | 2019-09-20 | 2019-12-17 | 中国银行股份有限公司 | big data free query method and device |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021151316A1 (en) * | 2020-06-24 | 2021-08-05 | 平安科技（深圳）有限公司 | Method and apparatus for querying data, electronic device, and storage medium |
| CN112306648A (en) * | 2020-10-29 | 2021-02-02 | 平安科技（深圳）有限公司 | Transaction calling method and device, electronic equipment and storage medium |
| CN112597162A (en) * | 2020-12-25 | 2021-04-02 | 平安银行股份有限公司 | Data set acquisition method, system, device and storage medium |
| CN112597162B (en) * | 2020-12-25 | 2023-08-08 | 平安银行股份有限公司 | Data set acquisition method, system, equipment and storage medium |
| CN112669076A (en) * | 2020-12-30 | 2021-04-16 | 平安证券股份有限公司 | Data distribution method based on rule engine, server and storage medium |
| CN112597187A (en) * | 2021-03-03 | 2021-04-02 | 北京冠新医卫软件科技有限公司 | GraphQL query method and device based on FHIR standard |
| CN112597187B (en) * | 2021-03-03 | 2021-06-29 | 北京冠新医卫软件科技有限公司 | GraphQL query method and device based on FHIR standard |
| CN112860737B (en) * | 2021-03-11 | 2022-08-12 | 中国平安财产保险股份有限公司 | Data query method and device, electronic equipment and readable storage medium |
| CN112860737A (en) * | 2021-03-11 | 2021-05-28 | 中国平安财产保险股份有限公司 | Data query method and device, electronic equipment and readable storage medium |
| CN113094387A (en) * | 2021-04-08 | 2021-07-09 | 杭州数梦工场科技有限公司 | Data query method and device, electronic equipment and machine-readable storage medium |
| CN113094401A (en) * | 2021-05-07 | 2021-07-09 | 北京京东振世信息技术有限公司 | Target source data generation method, device, equipment and storage medium |
| CN113094401B (en) * | 2021-05-07 | 2023-08-08 | 北京京东振世信息技术有限公司 | Target source data generation method, device, equipment and storage medium |
| CN113239044A (en) * | 2021-05-18 | 2021-08-10 | 安翰科技(武汉)股份有限公司 | Management device and application system of SQL records for query |
| CN113239069A (en) * | 2021-05-18 | 2021-08-10 | 上海跨境电子商务公共服务有限公司 | Data query method and data query system |
| CN113284586B (en) * | 2021-05-24 | 2024-05-14 | 康键信息技术（深圳）有限公司 | Medical service processing method, device, equipment and storage medium |
| CN113284586A (en) * | 2021-05-24 | 2021-08-20 | 康键信息技术（深圳）有限公司 | Medical service processing method, device, equipment and storage medium |
| CN113760993A (en) * | 2021-08-18 | 2021-12-07 | 北京健康之家科技有限公司 | Service data query method and device and electronic equipment |
| CN113672651A (en) * | 2021-08-24 | 2021-11-19 | 杭州海康威视数字技术股份有限公司 | Task execution method and device and electronic equipment |
| CN113672651B (en) * | 2021-08-24 | 2024-06-04 | 杭州海康威视数字技术股份有限公司 | Task execution method and device and electronic equipment |
| CN113742385A (en) * | 2021-09-15 | 2021-12-03 | 第四范式（北京）技术有限公司 | Data query method and device |
| CN113901088A (en) * | 2021-10-14 | 2022-01-07 | 中国银行股份有限公司 | Parameter checking method, device and equipment |
| WO2023115252A1 (en) * | 2021-12-20 | 2023-06-29 | Boe Technology Group Co., Ltd. | Data query method, data query apparatus, and computer-program product |
| CN114666165B (en) * | 2022-05-20 | 2022-08-09 | 深圳华锐分布式技术股份有限公司 | Query request response method, device, equipment and medium |
| CN114666165A (en) * | 2022-05-20 | 2022-06-24 | 深圳华锐分布式技术股份有限公司 | Query request response method, device, equipment and medium |
| CN116680061A (en) * | 2023-08-02 | 2023-09-01 | 腾讯科技（深圳）有限公司 | Task execution method, device, equipment and storage medium |
| CN116680061B (en) * | 2023-08-02 | 2024-03-15 | 腾讯科技（深圳）有限公司 | Task execution method, device, equipment and storage medium |
| CN116955416A (en) * | 2023-09-18 | 2023-10-27 | 腾讯科技（深圳）有限公司 | Data processing method, system, equipment and storage medium |
| CN116955416B (en) * | 2023-09-18 | 2023-12-08 | 腾讯科技（深圳）有限公司 | Data processing method, system, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021151316A1 (en) | 2021-08-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111782923A (en) | Data query method and device, electronic equipment and storage medium | |
| US20230281192A1 (en) | Method of generating globally verifiable unique identifiers using a scalable interlinked blockchain structure | |
| CN109474578B (en) | Message checking method, device, computer equipment and storage medium | |
| CN106874461B (en) | A kind of workflow engine supports multi-data source configuration security access system and method | |
| CN109255622B (en) | Traceable anti-fake data storage system | |
| CN108492195B (en) | Rule engine package parameter generalization method, equipment and storage medium | |
| US11442933B2 (en) | Function semantic based partition-wise SQL execution and partition pruning | |
| US20220300487A1 (en) | Blockchain database management system | |
| CN112860778B (en) | Database management method, device, equipment and medium for desktop application program | |
| CN112540811A (en) | Cache data detection method and device, computer equipment and storage medium | |
| CN112650759A (en) | Data query method and device, computer equipment and storage medium | |
| CN112446022A (en) | Data authority control method and device, electronic equipment and storage medium | |
| CN112328486A (en) | Interface automation test method and device, computer equipment and storage medium | |
| CN113672692B (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN112363997B (en) | Data version management method, device and storage medium | |
| CN113221154A (en) | Service password obtaining method and device, electronic equipment and storage medium | |
| CN111522881B (en) | Service data processing method, device, server and storage medium | |
| CN111339193B (en) | Category encoding method and device | |
| CN112860954A (en) | Real-time computing method and real-time computing system | |
| CN115086047B (en) | Interface authentication method and device, electronic equipment and storage medium | |
| CN111143399A (en) | Data processing method, data processing device, storage medium and computer equipment | |
| CN115687384A (en) | UUID (user identifier) identification generation method, device, equipment and storage medium | |
| CN112988888B (en) | Key management method, device, electronic equipment and storage medium | |
| CN112118290B (en) | Program analysis-based data resource management and control method | |
| CN115033605A (en) | Data query method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201016 |
|
| RJ01 | Rejection of invention patent application after publication |