CN115168398A

CN115168398A - Data query method and device, electronic equipment and storage medium

Info

Publication number: CN115168398A
Application number: CN202210952900.0A
Authority: CN
Inventors: 柴瑞
Original assignee: Beijing Jingdong Zhenshi Information Technology Co Ltd
Current assignee: Beijing Jingdong Zhenshi Information Technology Co Ltd
Priority date: 2022-08-09
Filing date: 2022-08-09
Publication date: 2022-10-11

Abstract

The disclosure provides a data query method, which can be applied to the technical field of big data. The method comprises the following steps: responding to the first query request, and acquiring a field to be queried carried in the first query request; determining a target field information table corresponding to a field to be queried from a data source; determining a first target query engine associated with a field to be queried based on a target field information table; constructing a first query statement based on the first target query engine; and executing the first query statement to obtain first target data associated with the field to be queried. In addition, the present disclosure also provides a data query device, a device and a storage medium.

Description

Data query method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of big data technologies, and in particular, to a data query method and apparatus, an electronic device, and a storage medium.

Background

With the development of data technology, in order to meet the market requirements for data retrieval and analysis, more and more kinds of query engines are applied to business systems of enterprises. For the same query request, query efficiency, resource occupation and query results expressed by different query engines may be completely different, so that the selection of the query engine is inevitable when data query is performed.

In implementing the disclosed concept, the inventors found that there are at least the following problems in the related art: when data is queried in the related technology, the selection of a query engine is carried out depending on the personal experience of technicians, so that the labor cost is high, and the query efficiency is low.

Disclosure of Invention

In view of the above, the present disclosure provides a data query method, a data query apparatus, an electronic device, a readable storage medium, and a computer program product.

One aspect of the present disclosure provides a data query method, including: responding to a first query request, and acquiring a field to be queried carried in the first query request; determining a target field information table corresponding to the field to be queried from a data source; determining a first target query engine associated with the field to be queried based on the target field information table; constructing a first query statement based on the first target query engine; and executing the first query statement to obtain first target data associated with the field to be queried.

According to the embodiment of the present disclosure, the first query request further carries query period information, and the data source includes a dictionary information table and a plurality of field information tables; wherein, the determining the target field information table corresponding to the field to be queried from the data source includes: determining a target query type of the first query request based on the query time period and response time information of the first query request; determining a reference field associated with the field to be queried based on the dictionary information table; and determining the target field information table from a plurality of field information tables based on the target query type and the reference field.

According to an embodiment of the present disclosure, the determining, based on the target field information table, a first target query engine associated with the field to be queried includes: determining at least one first query engine associated with the reference field based on the target field information table; and determining a first target query engine from at least one of the first query engines.

According to the embodiment of the disclosure, the field information table is configured with a query engine attribute value; wherein the determining at least one first query engine associated with the reference field based on the target field information table includes: and analyzing the attribute value of the query engine of the target field information table to determine at least one first query engine.

According to an embodiment of the present disclosure, the method further includes: determining the target query dimension of the reference field; determining at least one second query engine associated with the target query dimension; and determining at least one third query engine based on at least one of said first query engine and at least one of said second query engines; wherein said determining a first target query engine from at least one of said first query engines comprises: the first target query engine is determined from at least one of the third query engines.

According to an embodiment of the present disclosure, the first query engine is configured with a priority; wherein said determining a first target query engine from at least one of said first query engines comprises: sequencing at least one first query engine from high to low in priority to obtain a query engine sequence; and determining the first query engine positioned at the head in the query engine sequence as the first target query engine.

According to an embodiment of the present disclosure, the method further includes: determining the preset resource consumption of the first target query engine under the condition that the first target query engine is a preset query engine; analyzing a query plan based on the first query statement to obtain predicted resource consumption; determining a second target query engine from the sequence of query engines when the predicted resource consumption is greater than the predetermined resource consumption; and constructing a second query statement aiming at the field to be queried based on the second target query engine.

According to an embodiment of the present disclosure, the method further includes: carrying out serialization processing on the first target data to obtain serialized data; and taking the field to be queried as a cache main key and the serialized data as a cache value, and storing the field to be queried and the serialized data into a cache.

According to an embodiment of the present disclosure, the method further includes: responding to a second query request, and generating a target cache main key based on a target query field carried in the second query request; inquiring the cache by using the target cache main key to obtain an inquiry result; and when the query result is characterized in that a target cache value corresponding to the target cache primary key exists in the cache, performing deserialization processing on the target cache value to obtain second target data associated with the target query field.

Another aspect of the present disclosure provides a data query apparatus including: the device comprises a first acquisition module, a first determination module, a second determination module, a first construction module and a first processing module. The first obtaining module is used for responding to a first query request and obtaining a field to be queried carried in the first query request; the first determining module is used for determining a target field information table corresponding to the field to be inquired from a data source; a second determining module, configured to determine, based on the target field information table, a first target query engine associated with the field to be queried; a first constructing module, configured to construct a first query statement based on the first target query engine; and the first processing module is used for executing the first query statement to obtain first target data associated with the field to be queried.

Another aspect of the present disclosure provides an electronic device including: one or more processors; a memory for storing one or more instructions, wherein the one or more instructions, when executed by the one or more processors, cause the one or more processors to implement the method as described above.

Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions for implementing the method as described above when executed.

Another aspect of the disclosure provides a computer program product comprising computer executable instructions for implementing the method as described above when executed.

According to the embodiment of the disclosure, the query engines supported by the fields may be configured in advance and stored in the data source, when data query is performed, a first target query engine may be determined according to the fields to be queried and the field information table in the data source, and a first query statement is constructed based on the first target query engine, and then the first query statement may be executed to query to obtain first target data. Through the technical means, the technical problems that the selection of the query engine is carried out depending on the personal experience of technicians when data query is carried out in the related technology, the labor cost is high, and the query efficiency is low can be at least partially solved, the learning cost of data query is effectively reduced, and the query efficiency is improved.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:

fig. 1 schematically illustrates an exemplary system architecture to which the data query method and apparatus may be applied, according to an embodiment of the present disclosure.

Fig. 2 schematically shows a flow chart of a data query method according to an embodiment of the present disclosure.

FIG. 3 schematically shows a flow diagram of a first target query engine determination method according to another embodiment of the present disclosure.

FIG. 4 schematically illustrates a flow diagram of a first target query engine determination method according to yet another embodiment of the present disclosure.

Fig. 5 schematically shows a flow chart of a data query method according to another embodiment of the present disclosure.

Fig. 6 schematically shows a block diagram of a data querying device according to an embodiment of the present disclosure.

FIG. 7 schematically illustrates a block diagram of an electronic device suitable for implementing a data query method according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that these descriptions are illustrative only and are not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.

Where a convention analogous to "A, B and at least one of C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include, but not be limited to, systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a convention analogous to "A, B or at least one of C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B or C" would include, but not be limited to, systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

With the development of data technology, in order to meet the market requirements for data retrieval and analysis, more and more kinds of query engines are introduced and widely used in the market. Query engines commonly used in the related art may include traditional relational databases such as Oracle, DB2, mySQL, etc., and may also include HIVE, impala, draid, etc. that are pushed for big data processing. Various query engines have the same place and have respective characteristics and adaptive scenes, so in order to adapt to different business requirements, enterprises generally deploy various query systems to adapt to different query engines. However, for the same query request, the query efficiency, the resource occupation, and the performance of the query result in different query engines may be completely different, and therefore, when performing data query, the selection of the query engine is inevitable.

The selection method of the query engine adopted in the related art generally comprises the steps of data source screening, query engine screening, query code compiling and the like, however, the steps mostly depend on personal experience of business personnel, and because the same query can be executed by different query engines, the execution efficiency and the resource occupation of different query engines are different, the manual selection mode is likely to select the query engine which is less than the optimal one, thereby influencing the query efficiency.

In view of the above, embodiments of the present disclosure provide a data query method, apparatus, electronic device, readable storage medium, and computer program product. The method comprises the following steps: responding to the first query request, and acquiring a field to be queried carried in the first query request; determining a target field information table corresponding to a field to be queried from a data source; determining a first target query engine associated with a field to be queried based on a target field information table; constructing a first query statement based on the first target query engine; and executing the first query statement to obtain first target data associated with the field to be queried.

It should be noted that, unless explicitly stated that there is an execution sequence between different operations or there is an execution sequence between different operations in technical implementation, the execution sequence between multiple operations may not be sequential, or multiple operations may be executed simultaneously in the flowchart in this disclosure.

Fig. 1 schematically illustrates an exemplary system architecture to which the data query method and apparatus may be applied, according to an embodiment of the present disclosure. It should be noted that fig. 1 is only an example of a system architecture to which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, and does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios.

As shown in fig. 1, the system architecture 100 according to this embodiment may include

terminal devices

101, 102, 103, a network 104 and a server 105.

The

terminal devices

101, 102, 103 may be various electronic devices having a display screen including, but not limited to, smart phones, tablet computers, laptop portable computers, desktop computers, and the like. The user can perform data query through the

terminal devices

101, 102, 103, and the result of the data query can be displayed to the user in a display screen.

The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired and/or wireless communication links, and so forth.

The server 105 may be a server that provides various services, for example, a background management server that provides support for query requests initiated by users through the

terminal devices

101, 102, 103. The background management server can process the received query request and feed back the data query structure to the terminal equipment.

It should be noted that the data query method provided by the embodiment of the present disclosure may be generally executed by the server 105. Accordingly, the data query device provided by the embodiment of the present disclosure may be generally disposed in the server 105. The data query method provided by the embodiment of the present disclosure may also be executed by a server or a server cluster different from the server 105 and capable of communicating with the

terminal devices

101, 102, 103 and/or the server 105. Correspondingly, the data query apparatus provided in the embodiments of the present disclosure may also be disposed in a server or a server cluster that is different from the server 105 and is capable of communicating with the

terminal devices

101, 102, 103 and/or the server 105. Alternatively, the data query method provided by the embodiment of the present disclosure may also be executed by the

terminal device

101, 102, or 103, or may also be executed by another terminal device different from the

terminal device

101, 102, or 103. Correspondingly, the data query apparatus provided in the embodiment of the present disclosure may also be disposed in the

terminal device

101, 102, or 103, or in another terminal device different from the

terminal device

101, 102, or 103.

For example, the terminal device 101 may generate an inquiry request in response to an input operation of a user, and then the terminal device 101 may locally execute the data inquiry method provided by the embodiment of the present disclosure, and process the inquiry request to obtain the target data. Alternatively, the terminal device 101 may send the query request to other terminal devices, servers, or server clusters, and have other terminal devices, servers, or server clusters receiving the query request to execute the data query method provided by the embodiment of the present disclosure.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

As shown in fig. 2, the method includes operations S201 to S205.

In operation S201, in response to the first query request, a field to be queried carried in the first query request is obtained.

In operation S202, a target field information table corresponding to a field to be queried is determined from a data source.

In operation S203, a first target query engine associated with a field to be queried is determined based on the target field information table.

In operation S204, a first query statement is constructed based on the first target query engine.

In operation S205, a first query statement is executed to obtain first target data associated with a field to be queried.

According to the embodiment of the disclosure, the first query request can be generated according to the selection operation of a user on a selection frame, a drop-down frame and other components on the display interface. The first query request may include a field to be queried, a data period range, configured filtering conditions, and the like. The configured filtering conditions may include a filtering field, a filtering relation, a filtering standard value, etc., and are not limited herein.

According to the embodiment of the disclosure, the number of the fields to be queried carried in the first query request may be one or more.

According to the embodiment of the disclosure, a plurality of field information tables which are related to a plurality of fields stored in the query engine one by one can be included in the data source, and the field information tables can include information of field IDs, field names, supported query engines and the like. When each query engine writes a field, it may generate a field information table corresponding to the field, or modify the content of the existing field information table according to the query engine used when writing the field.

According to the embodiment of the disclosure, determining the target field information table corresponding to the field to be queried from the data source may refer to: and matching the fields to be inquired with attributes such as field IDs, field names and the like in the plurality of field information tables, and determining the matched field information table as a target field information table.

According to embodiments of the present disclosure, each field may have a corresponding plurality of query engines supporting the field. Accordingly, the first target query engine may be a query engine that supports the field to be queried.

According to an embodiment of the present disclosure, constructing the first query statement based on the first target query engine may be implemented by extracting a relevant field from a field to be queried and then replacing a query statement template with the relevant field; the query statement generation tool may also be implemented by any query statement generation tool, such as SqlMetal, and the like, which is not limited herein.

The method shown in fig. 2 is further described with reference to fig. 3-5 in conjunction with specific embodiments.

According to an embodiment of the present disclosure, a dictionary information table and a plurality of field information tables may be included in the data source.

According to the embodiment of the disclosure, the first query request may carry query period information.

According to an embodiment of the present disclosure, operation S202 may include the following operations:

determining a target query type of the first query request based on the query time interval and the response time information of the first query request; determining a reference field associated with a field to be queried based on the dictionary information table; and determining a target field information table from the plurality of field information tables based on the target query type and the reference field.

According to an embodiment of the present disclosure, the query type of the current query may include a real-time query, an offline query, a historical query, and the like. The classification of the query type may be determined based on a time period difference between the query time period and the response time information. For example, if the starting time of the query period is not more than 3 days from the current time, it may be determined that the query type of the current query is a real-time query; if the distance between the starting time of the query period and the current time is more than 3 days and less than 3 months, determining that the query type of the current query is offline query; if the starting time of the query period is more than 3 months from the current time, it can be determined that the query type of the current query is a history query. The classification of the query type may also be determined according to some preset conditions. For example, in the case that the time represented by the response time information does not exceed 6 am, it may be determined that the query type of the current query is a real-time query, and the like.

According to embodiments of the present disclosure, the same field may have different field names in different query engines. The dictionary information table can record field names of the same field in all the query engines, and the dictionary information table can be used for converting the field to be queried into a uniform reference field. The reference field may refer to a field name of the field to be queried in a reference query engine, and the reference query engine may refer to any one of the query engines, which is not limited herein.

According to an embodiment of the present disclosure, the field information table may include information such as a field ID, a reference field name, a query type attribute value, and a query engine attribute value, as shown in table 1.

TABLE 1

Column name	Type (B)	Remarks for note
			ID	int	Field ID
Name	varchar	Name of reference field
			Query_Type	int	Query type attribute values
Query_Engine	int	Query engine attribute values

According to the embodiment of the disclosure, the determining of the target field information table from the plurality of field information tables may be, based on the target query type and the reference field, respectively matching the target query type with the query type attribute value in each field information table, matching the reference field with the reference field name in each field information table, and determining the matched field information table as the target field information table when both are matched.

According to an embodiment of the present disclosure, operation S203 may include the operations of:

determining at least one first query engine associated with the reference field based on the target field information table; and determining a first target query engine from the at least one first query engine.

According to an embodiment of the present disclosure, determining at least one first query engine associated with a reference field may include the operations of:

the query engine attribute values of the target field information table are parsed to determine at least one first query engine.

According to embodiments of the present disclosure, a query engine attribute value may be characterized as the sum of the values characterized by all query engines supported by the reference field. For example, the values characterized by the respective query engines are 1: impala,2: hive,4: kylin,8: druid and 16: mysql. The parsing of the query engine attribute value may be in the form of decomposing the query engine attribute value into a sum of exponential powers of 2, and taking the query engine attribute value of the reference field as 21 as an example, the query engine attribute value may be parsed as 21=1 × 2 ⁰ +0×2 ¹ +1×2 ² +0×2 ³ +1×2 ⁴ Based on the result of the parsing, it can be determined that the reference field supports the query engines of Impala, kylin and mysql, i.e., theA query engine includes Impala, kylin, and mysql.

According to embodiments of the present disclosure, the first query engine may be configured with a priority. The priority may be pre-configured by a user, for example, for three query engines, namely Impala, kylin and mysql, the query engines may be mysql, kylin and Impala in a preset order from high to low; the priority may also be automatically adjusted according to the resource occupation of each query engine, for example, a query engine with more remaining resources is configured with a higher priority, which is not limited herein.

According to an embodiment of the present disclosure, determining a first target query engine from among at least one first query engine may include the operations of:

sequencing at least one first query engine from high to low in priority to obtain a query engine sequence; and determining a first query engine positioned at the head in the query engine sequence as a first target query engine.

As shown in fig. 3, the method includes operations S301 to S306.

In operation S301, a target query type of the first query request is determined based on the query period and the response time information of the first query request.

In operation S302, at least one first query engine corresponding to a target query type is determined.

In operation S303, a target query dimension of the reference field is determined.

In operation S304, at least one second query engine related to the target query dimension is determined.

In operation S305, at least one third query engine is determined based on the at least one first query engine and the at least one second query engine.

In operation S306, a first target query engine is determined from at least one third query engine.

According to embodiments of the present disclosure, a target query dimension of the first query request may be determined from the field name of the base field. For example, when the field name of the base field includes a keyword such as "frequency", the target query dimension of the first query request may be determined as a frequency query dimension; for another example, when the field name of the reference field includes a keyword of a specific scenario such as "traffic", the target query dimension of the first query request may be determined to be a traffic query dimension.

In accordance with embodiments of the present disclosure, there may be a supported query engine for each query dimension. For example, the query engine supporting the frequency query dimension may be Hive, and the query engine supporting the traffic query dimension may be Impala, and the like.

According to an embodiment of the present disclosure, determining at least one third query engine, based on the at least one first query engine and the at least one second query engine, may be understood as: the same one of the at least one first query engine and the at least one second query engine is determined to be a third query engine. For example, each query engine may be represented in the form of an exponential power of 2, such as 1: impala,2: hive,4: kylin,8: druid and 16: mysql; the query engines supported by the respective query types and query dimensions can be represented as corresponding exponential power sums, e.g., the first query engine supported by the target query type can be represented as (10101) ₂ I.e., the first query engine includes Impala, kylin, and mysql, the second query engine supported by the target query dimension may be represented as (01101) ₂ I.e., the second query engine includes Hive, kylin, and mysql, the third query engine may be represented as (10101) ₂ And (01101) ₂ That is, the third query engine may be represented as (00101) ₂ That is, the third query engine includes Kylin and mysql.

In some embodiments, after the first target query engine is determined, if the first target query engine is a special query engine, such as impala, the query statement constructed based on the first query requirement may be further analyzed, and then, according to the analysis result, whether the first target query engine meets the requirement is determined, and if not, the first query engine may be re-determined after the query engine is excluded.

FIG. 4 schematically shows a flow diagram of a first target query engine determination method according to yet another embodiment of the present disclosure.

As shown in fig. 4, the method includes operations S401 to S406.

In operation S401, a first query statement is constructed based on a first target query engine.

In operation S402, it is determined whether the first target query requirement is a preset query engine. In a case where it is determined that the first target query request is a preset query engine, performing operation S403; in a case where it is determined that the first target query requirement is not the preset query engine, operation S406 is performed.

In operation S403, query plan analysis is performed on the first query statement to obtain the predicted resource consumption.

In operation S404, it is determined whether the predicted resource consumption amount is greater than a preset resource consumption amount. In a case where it is determined that the predicted resource consumption amount is greater than the preset resource consumption amount, performing operation S405; in the case where it is determined that the predicted resource consumption amount is less than or equal to the preset resource consumption amount, operation S406 is performed.

In operation S405, the first target query engine is redetermined from the sequence of query engines. After operation S405 is completed, operation S401 is returned to be performed.

In operation S406, the first target query engine is returned.

According to an embodiment of the present disclosure, the preset query engine may be a query engine with memory limitation, such as Impala.

According to an embodiment of the present disclosure, the step of re-determining the first target query engine from the query engine sequence may be to take the query engine located at the second position in the query engine sequence as the first target query engine, and then delete the query engine located at the first position from the query engine sequence.

As shown in fig. 5, the method includes operations S501 to S507.

In operation S501, a query request is received.

In operation S502, it is determined whether there is a same query in the cache. In case it is determined that there is no identical query in the cache, operation S503 is performed; in case it is determined that there is the same query in the cache, operation S506 is performed.

In operation S503, the query engine is adapted according to the query request.

In operation S504, a query statement is constructed based on the query engine.

In operation S505, a query statement is executed.

In operation S506, a query result is returned.

In operation S507, the cache is updated based on the query result.

According to the embodiment of the disclosure, data in the cache can be stored in a form of key value pairs, wherein a cache primary key in a key value pair can be generated according to a field needing to be queried, and a cache value in the key value pair can be generated according to data returned by querying the field.

According to the embodiment of the disclosure, the query request may carry a target query field, and accordingly, the query result may include target data corresponding to the target query field.

According to an embodiment of the present disclosure, operation S507 may include the following operations:

carrying out serialization processing on the target data to obtain serialized data; and storing the target query field and the serialized data into a cache by taking the target query field as a cache main key and the serialized data as a cache value.

According to the embodiment of the disclosure, the serialization processing can reduce the storage space occupied by the target data, so that the cache can store more key-value pairs.

According to an embodiment of the present disclosure, operation S502 may include the operations of:

firstly, a target query field carried in a query request can be converted into a target cache main key; then, the target cache main key is used for inquiring the cache, namely the target cache main key is compared with the cache main key of each key value pair in the cache in sequence, and therefore an inquiry result is obtained; then, in the case that the query result is characterized in that the cache main key same as the target cache main key exists in the cache, it can be determined that the same query exists in the cache.

According to the embodiment of the disclosure, under the condition that the query result is characterized in that the cache main key same as the target cache main key exists in the cache, the target cache value corresponding to the target cache main key exists in the cache can be determined, then, the target cache value can be subjected to deserialization, and the obtained target data is the query result.

As shown in fig. 6, the data query apparatus 600 includes a first obtaining module 610, a first determining module 620, a second determining module 630, a first constructing module 640, and a first processing module 650.

The first obtaining module 610 is configured to, in response to the first query request, obtain a field to be queried, where the field to be queried is carried in the first query request.

The first determining module 620 is configured to determine a target field information table corresponding to a field to be queried from a data source.

A second determining module 630, configured to determine, based on the target field information table, the first target query engine associated with the field to be queried.

A first construction module 640 for constructing a first query statement based on the first target query engine.

The first processing module 650 is configured to execute the first query statement to obtain first target data associated with the field to be queried.

According to the embodiment of the disclosure, the first query request further carries query period information, and the data source includes a dictionary information table and a plurality of field information tables.

According to an embodiment of the present disclosure, the first determination module 620 includes a first determination unit, a second determination unit, and a third determination unit.

And the first determining unit is used for determining the target query type of the first query request based on the query time interval and the response time information of the first query request.

And the second determining unit is used for determining the reference field associated with the field to be inquired based on the dictionary information table.

And the third determining unit is used for determining a target field information table from the plurality of field information tables based on the target query type and the reference field.

According to an embodiment of the present disclosure, the second determination module 630 includes a fourth determination unit and a fifth determination unit.

And the fourth determining unit is used for determining at least one first query engine associated with the reference field based on the target field information table.

A fifth determining unit, configured to determine a first target query engine from the at least one first query engine.

According to an embodiment of the present disclosure, a query engine attribute value is configured in the field information table.

According to an embodiment of the present disclosure, the fourth determination unit includes a first determination subunit.

And the first determining subunit is used for analyzing the query engine attribute value of the target field information table to determine at least one first query engine.

According to an embodiment of the present disclosure, the second determining module 630 further includes a sixth determining unit, a seventh determining unit, and an eighth determining unit.

And the sixth determining unit is used for determining the target query dimension of the reference field.

A seventh determining unit for determining at least one second query engine related to the target query dimension.

An eighth determining unit for determining at least one third query engine based on the at least one first query engine and the at least one second query engine.

According to an embodiment of the present disclosure, the fifth determining unit includes a second determining subunit.

A second determining subunit for determining the first target query engine from the at least one third query engine.

According to an embodiment of the present disclosure, the first query engine is configured with a priority.

According to an embodiment of the present disclosure, the fifth determination unit includes a third determination subunit and a fourth determination subunit.

And the third determining subunit is used for sequencing at least one first query engine from high to low according to the priority to obtain a query engine sequence.

And the fourth determining subunit is used for determining that the first query engine positioned at the head in the query engine sequence is the first target query engine.

According to an embodiment of the present disclosure, the data query apparatus 600 further includes a third determination module, a second processing module, a fourth determination module, and a second construction module.

And the third determining module is used for determining the preset resource consumption of the first target query engine under the condition that the first target query engine is the preset query engine.

And the second processing module is used for carrying out query plan analysis on the basis of the first query statement to obtain the predicted resource consumption.

And the fourth determining module is used for determining a second target query engine from the query engine sequence under the condition that the predicted resource consumption is greater than the preset resource consumption.

And the second construction module is used for constructing a second query statement aiming at the field to be queried based on the second target query engine.

According to an embodiment of the present disclosure, the data query apparatus 600 further includes a third processing module and a storage module.

And the third processing module is used for carrying out serialization processing on the first target data to obtain serialized data.

And the storage module is used for storing the field to be queried and the serialized data into a cache by taking the field to be queried as a cache main key and the serialized data as a cache value.

According to the embodiment of the present disclosure, the data query apparatus 600 further includes a generation module, a query module, and a fourth processing module.

And the generating module is used for responding to the second query request and generating a target cache main key based on the target query field carried in the second query request.

And the query module is used for querying the cache by using the target cache main key to obtain a query result.

And the fourth processing module is used for performing deserialization processing on the target cache value under the condition that the query result is characterized in that the target cache value corresponding to the target cache primary key exists in the cache, so as to obtain second target data associated with the target query field.

Any number of modules, sub-modules, units, sub-units, or at least part of the functionality of any number thereof according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be implemented by being split into a plurality of modules. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented at least partially as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or by any other reasonable means of hardware or firmware for integrating or packaging a circuit, or by any one of or a suitable combination of any of software, hardware, and firmware. Alternatively, one or more of the modules, sub-modules, units, sub-units according to embodiments of the disclosure may be at least partially implemented as a computer program module, which when executed may perform the corresponding functions.

For example, any plurality of the first obtaining module 610, the first determining module 620, the second determining module 630, the first constructing module 640, and the first processing module 650 may be combined and implemented in one module/unit/sub-unit, or any one of the modules/units/sub-units may be split into a plurality of modules/units/sub-units. Alternatively, at least part of the functionality of one or more of these modules/units/sub-units may be combined with at least part of the functionality of other modules/units/sub-units and implemented in one module/unit/sub-unit. According to an embodiment of the present disclosure, at least one of the first obtaining module 610, the first determining module 620, the second determining module 630, the first constructing module 640, and the first processing module 650 may be at least partially implemented as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or implemented by any one of three implementations of software, hardware, and firmware, or by a suitable combination of any of them. Alternatively, at least one of the first obtaining module 610, the first determining module 620, the second determining module 630, the first constructing module 640 and the first processing module 650 may be at least partially implemented as a computer program module, which when executed, may perform a corresponding function.

It should be noted that the data query device portion in the embodiment of the present disclosure corresponds to the data query method portion in the embodiment of the present disclosure, and the description of the data query device portion specifically refers to the data query method portion, which is not described herein again.

FIG. 7 schematically illustrates a block diagram of an electronic device suitable for implementing a data query method according to an embodiment of the present disclosure. The electronic device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 7, a computer electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. The processor 701 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 701 may also include on-board memory for caching purposes. The processor 701 may comprise a single processing unit or a plurality of processing units for performing the different actions of the method flows according to embodiments of the present disclosure.

In the RAM 703, various programs and data necessary for the operation of the electronic apparatus 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other by a bus 704. The processor 701 performs various operations of the method flows according to the embodiments of the present disclosure by executing programs in the ROM 702 and/or the RAM 703. It is noted that the programs may also be stored in one or more memories other than the ROM 702 and RAM 703. The processor 701 may also perform various operations of method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.

Electronic device 700 may also include input/output (I/O) interface 705, which input/output (I/O) interface 705 also connects to bus 704, according to an embodiment of the present disclosure. The electronic device 700 may also include one or more of the following components connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.

According to embodiments of the present disclosure, method flows according to embodiments of the present disclosure may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711. The computer program, when executed by the processor 701, performs the above-described functions defined in the system of the embodiment of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.

The present disclosure also provides a computer-readable storage medium, which may be embodied in the device/apparatus/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement a method according to an embodiment of the disclosure.

According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples may include, but are not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM 702 and/or the RAM 703 and/or one or more memories other than the ROM 702 and the RAM 703 described above.

Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the method provided by the embodiments of the present disclosure, when the computer program product is run on an electronic device, the program code being adapted to cause the electronic device to carry out the data query method provided by the embodiments of the present disclosure.

When executed by the processor 701, performs the above-described functions defined in the system/apparatus of the embodiments of the present disclosure. The systems, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.

In one embodiment, the computer program may be hosted on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted in the form of a signal on a network medium, distributed, downloaded and installed via the communication section 709, and/or installed from the removable medium 711. The computer program containing program code may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

In accordance with embodiments of the present disclosure, program code for executing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, these computer programs may be implemented using high level procedural and/or object oriented programming languages, and/or assembly/machine languages. The programming language includes, but is not limited to, programming languages such as Java, C + +, python, the "C" language, or the like. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In situations involving remote computing devices, the remote computing devices may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to external computing devices (e.g., through the internet using an internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It will be appreciated by those skilled in the art that various combinations and/or combinations of the features recited in the various embodiments of the disclosure and/or the claims may be made even if such combinations or combinations are not explicitly recited in the disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.

The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.

Claims

1. A method of data query, comprising:

responding to a first query request, and acquiring a field to be queried carried in the first query request;

determining a target field information table corresponding to the field to be queried from a data source;

determining a first target query engine associated with the field to be queried based on the target field information table;

constructing a first query statement based on the first target query engine; and

and executing the first query statement to obtain first target data associated with the field to be queried.

2. The method according to claim 1, wherein the first query request further carries query period information, and the data source includes a dictionary information table and a plurality of field information tables;

wherein, the determining the target field information table corresponding to the field to be queried from the data source includes:

determining a target query type of the first query request based on the query time period and response time information of the first query request;

determining a reference field associated with the field to be queried based on the dictionary information table; and

determining the target field information table from a plurality of the field information tables based on the target query type and the reference field.

3. The method of claim 2, wherein the determining a first target query engine associated with the field to be queried based on the target field information table comprises:

determining at least one first query engine associated with the reference field based on the target field information table; and

a first target query engine is determined from at least one of the first query engines.

4. The method of claim 3, wherein a query engine attribute value is configured in the field information table;

wherein the determining at least one first query engine associated with the reference field based on the target field information table comprises:

and analyzing the query engine attribute value of the target field information table to determine at least one first query engine.

5. The method of claim 3, further comprising:

determining a target query dimension for the reference field;

determining at least one second query engine that is relevant to the target query dimension; and

determining at least one third query engine based on at least one of the first query engine and at least one of the second query engines;

wherein said determining a first target query engine from at least one of said first query engines comprises:

determining the first target query engine from at least one of the third query engines.

6. The method of claim 3, wherein the first query engine is configured with a priority;

sequencing at least one first query engine from high to low in priority to obtain a query engine sequence; and

determining the first query engine in the query engine sequence located at the head as the first target query engine.

7. The method of claim 6, further comprising:

determining the preset resource consumption of the first target query engine under the condition that the first target query engine is a preset query engine;

analyzing a query plan based on the first query statement to obtain predicted resource consumption;

determining a second target query engine from the sequence of query engines if the predicted resource consumption is greater than the preset resource consumption; and

and constructing a second query statement aiming at the field to be queried based on the second target query engine.

8. The method of claim 1, further comprising:

carrying out serialization processing on the first target data to obtain serialized data; and

and taking the field to be queried as a cache main key and the serialized data as a cache value, and storing the field to be queried and the serialized data into a cache.

9. The method of claim 8, further comprising:

responding to a second query request, and generating a target cache main key based on a target query field carried in the second query request;

querying the cache by using the target cache main key to obtain a query result; and

and when the query result is characterized in that a target cache value corresponding to the target cache main key exists in the cache, performing deserialization processing on the target cache value to obtain second target data associated with the target query field.

10. A data query apparatus, comprising:

the first acquisition module is used for responding to a first query request and acquiring a field to be queried carried in the first query request;

the first determining module is used for determining a target field information table corresponding to the field to be inquired from a data source;

the second determination module is used for determining a first target query engine associated with the field to be queried based on the target field information table;

a first construction module for constructing a first query statement based on the first target query engine; and

and the first processing module is used for executing the first query statement to obtain first target data associated with the field to be queried.

11. An electronic device, comprising:

one or more processors;

a memory to store one or more instructions that,

wherein the one or more instructions, when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-9.

12. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to carry out the method of any one of claims 1 to 9.

13. A computer program product comprising computer executable instructions for implementing the method of any one of claims 1 to 9 when executed.