CN114185898A - Data query method and device and terminal equipment - Google Patents
Data query method and device and terminal equipment Download PDFInfo
- Publication number
- CN114185898A CN114185898A CN202111549720.XA CN202111549720A CN114185898A CN 114185898 A CN114185898 A CN 114185898A CN 202111549720 A CN202111549720 A CN 202111549720A CN 114185898 A CN114185898 A CN 114185898A
- Authority
- CN
- China
- Prior art keywords
- data
- query
- real
- historical
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/103—Formatting, i.e. changing of presentation of documents
- G06F40/114—Pagination
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application is applicable to the technical field of data processing, and provides a data query method, a data query device and terminal equipment, wherein the data query method comprises the following steps: determining a routing policy according to an archive field in the query command; when the routing strategy is a first routing strategy, acquiring corresponding real-time data from a real-time database and returning the real-time data; when the routing strategy is a second routing strategy, acquiring corresponding historical data from the historical database and returning the historical data; when the routing strategy is a third routing strategy, corresponding real-time data is obtained from the real-time database, corresponding historical data is obtained from the historical database, the real-time data and the historical data are returned, the routing strategy is determined through the filed fields, namely, which database table the order data needing to be inquired is correspondingly stored in is determined, and then the corresponding order data is obtained based on the routing strategy, so that a data system for separating cold data from hot data is realized, and the requirements of the historical data and the real-time data are inquired.
Description
Technical Field
The present application belongs to the technical field of data processing, and in particular, to a data query method, apparatus, terminal device, and computer-readable storage medium.
Background
With the increase of user quantity and the gradual improvement of services, the order data volume in the order system is larger and larger, and after the order system operates for a period of time, the order system faces the pressure of high concurrent read-write requests when inquiring data due to the accumulation of the stored and managed order data volume, so that the inquiry efficiency is seriously influenced.
For such a situation, at present, the data amount in each table is usually reduced by a way of database partitioning and table partitioning to improve the query efficiency, for example, a database (a database storing data after an order) is horizontally partitioned according to a certain dimension, so that the data amount of each data table is controlled within a certain range, the single table query performance is kept within a relatively good range, however, the database partitioning needs to be greatly modified for an original database, and the query performance of data of different tables during aggregated paging query is poor, so that the way of improving the query efficiency by a way of database partitioning and table partitioning has the problems of high implementation difficulty and poor query performance for aggregated paging query.
In order to improve the query efficiency and the query performance, another way is to store data in a history table by regular filing, so as to separate cold and hot data, that is, the history data and the real-time data are stored in different databases, however, the databases in which the cold and hot data are stored separately usually can only query the real-time data, and cannot query the history data.
Disclosure of Invention
The embodiment of the application provides a data query method, a data query device, terminal equipment and a computer readable storage medium, and can solve the problem that the existing database with cold and hot data stored separately can only query real-time data and cannot query historical data.
In a first aspect, an embodiment of the present application provides a data query method, including:
determining a routing policy according to an archive field in the query command;
when the routing strategy is a first routing strategy, acquiring corresponding real-time data from a real-time database and returning the real-time data;
when the routing strategy is a second routing strategy, acquiring corresponding historical data from a historical database and returning the historical data;
and when the routing strategy is a third routing strategy, acquiring corresponding real-time data from a real-time database, acquiring corresponding historical data from a historical database, and returning the real-time data and the historical data.
In an implementation manner of the first aspect, the determining a routing policy according to an archive field in the query command includes:
determining the data distribution condition corresponding to the query command according to the filing field;
and setting a routing strategy according to the data distribution condition corresponding to the query command.
In an implementation manner of the first aspect, the determining, according to the archive field, a data distribution condition corresponding to the query command specifically includes:
and determining the data distribution condition according to the archive field and the archive field threshold value contained in the query command.
In an implementation manner of the first aspect, when the routing policy is a second routing policy, acquiring corresponding historical data from a historical database and returning the historical data, includes:
determining a data table name according to the query command;
replacing the data table name with a corresponding historical data table name;
and acquiring corresponding historical data from the historical data table corresponding to the historical data table name and returning the historical data.
In an implementation manner of the first aspect, when the routing policy is a third routing policy, the obtaining of corresponding real-time data from a real-time database, obtaining corresponding historical data from a historical database, and returning the real-time data and the historical data includes:
acquiring corresponding real-time data from the real-time database;
determining a data table name according to the query command, replacing the data table name with a corresponding historical data table name, and acquiring corresponding historical data from a historical data table corresponding to the historical data table name;
fusing the real-time data and the historical data to obtain target data;
and returning the target data.
In an implementation manner of the first aspect, when the query command is a paging query command, and when the routing policy is a third routing policy, acquiring corresponding real-time data from a real-time database, acquiring corresponding historical data from a historical database, and returning the real-time data and the historical data, includes:
determining the total data number of real-time data and the total data number of historical data according to the paging query command;
determining query initial offset and query line number of a real-time data table corresponding to a data table name and query initial offset and query line number of a historical data table corresponding to the historical data table name during paging query according to the line number and page number of each page of paging;
inquiring real-time data according to the initial offset and the number of inquiry lines of the real-time data table corresponding to the data table name;
and querying the historical data according to the initial offset and the query line number of the historical data table corresponding to the name of the historical data table.
In an implementation manner of the first aspect, before determining the routing policy according to the archive field in the query command, the method further includes:
setting an interceptor to intercept the query command;
judging whether the intercepted query command contains an archive field or not through the interceptor;
if the intercepted query command contains an archive field, determining a routing strategy according to the archive field in the query command; otherwise, data query is carried out from the real-time database.
In a second aspect, an embodiment of the present application provides a data query apparatus, including:
the strategy determining unit is used for determining a routing strategy according to the filing field in the query command;
the first query unit is used for acquiring corresponding real-time data from a real-time database and returning the real-time data when the routing strategy is the first routing strategy;
the second query unit is used for acquiring corresponding historical data from a historical database and returning the historical data when the routing strategy is a second routing strategy;
and the third query unit is used for acquiring corresponding real-time data from a real-time database and corresponding historical data from a historical database when the routing policy is the third routing policy, and returning the real-time data and the historical data.
In a third aspect, an embodiment of the present application provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor, when executing the computer program, implements the steps of the data query method according to any one of the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps of the data query method according to any one of the first aspect.
In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a terminal device, enables the terminal device to execute the data query method according to any one of the above first aspects.
Compared with the prior art, the embodiment of the application has the advantages that:
according to the data query method, the data query device, the terminal equipment, the computer readable storage medium and the computer program product, the routing strategy is determined through the filed field, namely, it is determined in which database table the order data to be queried is correspondingly stored, the data needs to be acquired from the real-time database or the historical database, and the corresponding order data is acquired based on the routing strategy, so that a data system for separating cold data from hot data can be realized, the requirements of historical data and real-time data are simultaneously queried, and the problem that the existing database for separating cold data from hot data can only query the real-time data and cannot query the historical data is solved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a flowchart illustrating an implementation of a data query method according to an embodiment of the present application;
fig. 2 is a flowchart illustrating a specific implementation of S11 in the data query method according to an embodiment of the present application;
fig. 3 is a flowchart illustrating a specific implementation of S13 in the data query method according to an embodiment of the present application;
fig. 4 is a flowchart illustrating a specific implementation of S14 in the data query method according to an embodiment of the present application;
FIG. 5 is a flow chart of another implementation of a data query method provided by an embodiment of the present application;
FIG. 6 is a schematic structural diagram of a data query device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.
Referring to fig. 1, fig. 1 is a flowchart illustrating an implementation of a data query method according to an embodiment of the present application. In the embodiment of the present application, the execution subject of the data query method may be a device with data processing capability, such as a terminal device or a server. The terminal device may be an electronic device such as a smart phone, a tablet computer, a desktop computer, and a smart wearable device, which is not limited in this embodiment of the application.
As shown in fig. 1, the data query method provided in an embodiment of the present application may include S11 to S14, which are detailed as follows:
at S11, a routing policy is determined based on the archive field in the query command.
In the embodiment of the application, the terminal device can generate a corresponding query command according to the query operation of the user.
In a specific application, a user can input a query operation through an interactive interface of an order management system installed in a terminal device, and the terminal device obtains the query operation input by the user and then generates a query command corresponding to the query operation.
For example, assuming that a user inputs a query operation for selecting data related to an order generated from 11/month 1/2021 to 11/month 15/2021 in order to query in the order management system, the order management system may, upon receiving the query operation, query corresponding order data from a database storing the order data in response to the query operation, and return the queried order data to the user.
In a specific implementation, when the order management system receives a query operation input by a user, an inquiry command corresponding to the query operation may be generated and sent to an executor (executor) of the database, and the executor (executor) executes the query operation corresponding to the inquiry command, that is, inquires order data required by the user and returns the order data required by the user.
In a specific application, the query command may be an sql command. It is understood that the query command may be other names capable of making the database perform data query, and the present application is not limited thereto.
In an embodiment of the present application, an interceptor may be configured to intercept a query command sent by an order management system to an executor of a database, and then determine whether the query command has an archive field, and if the query command includes the archive field, determine a routing policy according to the archive field.
In the embodiment of the present application, the archive field refers to a key field used when data is archived, that is, when data is archived, the key archive field is used as a tag to archive the data, and the archive field may be a creation date, a primary key ID, or the like.
For example, the process of archiving data with creation date as the archiving field may be: and archiving the order data with the creation date earlier than the preset time.
As another example, the process of data archiving with the primary key ID as the archiving field may be: and archiving the order data with the primary key ID smaller than the preset primary key ID.
It should be noted that, when generating the primary key ID, the primary key ID may be generated according to an increment rule, that is, the primary key ID of the order generated later is larger than the primary key ID of the order generated earlier; the primary key ID may also be generated according to a decremental rule, i.e., the primary key ID of a later generated order is less than the primary key ID of an earlier generated order.
The primary key ID is a unique index of the order, and is, for example, an order number.
It is easy to understand that when data is archived, the data is archived based on the archive field, and when the user inputs a query operation, the data can be queried by selecting the archive field, for example, querying order data generated within a certain date or order data with an order number within a certain range.
In a specific implementation, the archive field may be selected according to the content of multiple aspects such as the business requirement and the database performance, and the order data may be archived based on different archive fields.
It can be understood that the archived data is historical data, and the unarchived data is real-time data. And storing the archived historical data in a historical database, and storing unarchived real-time data in a real-time database to realize cold and hot data separation.
In a specific implementation, the routing policy is used to describe a query path corresponding to the query command, that is, a database from which order data needs to be queried/obtained corresponding to the query command.
In this embodiment of the present application, the routing policy may include a first routing policy, a second routing policy, and a third routing policy, where the first routing policy is to obtain order data only from a real-time database, the second routing policy is to obtain order data only from a historical database, and the third routing policy is to obtain order data from the real-time database and the historical database at the same time.
In a specific application, the above routing policy may implement routing of the read request to the main library (i.e. the database corresponding to the query command) through a thread local variable (threaded local).
Referring to fig. 2, fig. 2 is a schematic diagram illustrating an implementation flow of S11 in the data query method according to the embodiment of the present application. As shown in fig. 2, in an embodiment of the present application, the step S11 may include the following steps:
s21: and determining the data distribution condition corresponding to the query command according to the filing field.
S22: and setting a routing strategy according to the data distribution condition corresponding to the query command.
In the embodiment of the present application, since the query command includes the archive field, the corresponding data distribution condition can be determined according to the archive field in the query command, where the data distribution condition refers to in which database table the data to be queried is stored.
In an embodiment of the present application, the step S21 may specifically be determining the data distribution according to the archive field and the archive field threshold included in the query command.
In a specific application, the archive field information contained in the query command may be compared with the archive field threshold to determine a data distribution situation corresponding to the query command, that is, if the archive field information contained in the query command is before the archive field threshold, it indicates that the order data to be queried in the query command is stored in the historical database (that is, all the order data to be queried is already archived), and if the archive field information contained in the query command is after the archive field threshold, it indicates that the order data to be queried in the query command is stored in the real-time database (that is, all the order data to be queried is not archived), otherwise, it indicates that the data to be queried in the query command is present in both the real-time database and the historical database. If the order data to be inquired in the inquiry command is stored in the real-time database, correspondingly setting the routing strategy as a first routing strategy; if the order data to be inquired in the inquiry command is stored in the historical database, correspondingly setting the routing strategy as a second routing strategy; if the order data to be queried in the query command exists in both the real-time database and the historical database, the routing policy correspondence is set as a third routing policy.
For example, assume that the query command is "query about data about orders generated from 11/1/2021 to 11/15/2021", wherein the archive field is the creation date of the order. Assuming that the archive field threshold is 2021 year 10 month 31 day, it indicates that the order data before 2021 year 10 month 31 day (including 2021 year 10 month 31 day) has been archived and stored in the historical database, and the order data after 2021 year 10 month 31 day has not been archived, then it can be determined that all the order data to be queried are stored in the real-time database corresponding to the query command, so that the routing policy is determined to be the first routing policy; assuming that the archive field threshold is 2021, 11, 16 days, it indicates that the order data before 11, 16 days of 2021 (including 11, 16 days of 2021) has been archived and stored in the historical database, and the order data after 11, 16 days of 2021 has not been archived, then it may be determined that all the order data to be queried is stored in the historical database corresponding to the query command, and thus the routing policy is determined to be the second routing policy; assuming that the archive field threshold is 2021 year 11 month 5 day, it indicates that order data before 2021 year 11 month 5 day (including 2021 year 11 month 5 day) has been archived and stored in the history database, and order data after 2021 year 11 month 5 day is not archived, then it can be determined that order data to be queried for order data 2021 year 11 month 1 day to 2021 month 5 day of milk, and order data from 2021 year 11 month 6 day to 2021 year 11 month 15 day are stored in the real-time database in response to the query command, and thus the routing policy is determined to be the third routing policy.
In S12, when the routing policy is the first routing policy, acquiring corresponding real-time data from the real-time database and returning the real-time data.
In this embodiment of the application, when the routing policy is the first routing policy, it indicates that the data to be queried in the query command only exists in the real-time database, so that the order management system directly obtains the corresponding real-time data from the real-time database, and returns the queried real-time data to the user.
In a specific application, when it is determined that the routing policy is the first routing policy, the order management system sends a command for acquiring corresponding real-time data from the real-time database to the executor (executor), and the executor (executor) queries the real-time database and returns a query result.
In S13, when the routing policy is the second routing policy, corresponding history data is obtained from the history database and returned.
In this embodiment of the application, when the routing policy is the second routing policy, it indicates that the data to be queried in the query command only exists in the historical database, and therefore the order management system needs to obtain corresponding historical data from the historical database and return the queried historical data to the user.
In a specific application, when it is determined that the routing policy is the second routing policy, the order management system sends a command for acquiring corresponding historical data from the historical database to the executor, and the executor queries the historical database and then returns a query result.
Referring to fig. 3, fig. 3 is a schematic diagram illustrating an implementation flow of S13 in the data query method according to the embodiment of the present application. As shown in fig. 3, in an embodiment of the present application, the step S13 may include the following steps:
s31: and determining the name of the data table according to the query command.
S32: the data table name is replaced with the corresponding historical data table name.
S33: and acquiring corresponding historical data from the historical data table corresponding to the historical data table name and returning the historical data.
In this embodiment, since the data table name of the historical data table is set based on the data table name of the real-time data table, usually, a key character indicating the historical data is added to the data table name of the real-time data table, for example, _ LIS, when querying the historical data, the data table name of the historical data table needs to be determined first, so that the data table name in the query command, which is usually the table name of the real-time data table, is determined according to the query command, and then the data table name is replaced by the corresponding historical data table name, so that the corresponding historical data can be queried from the historical data table corresponding to the historical data table name, and then the queried historical data is returned to the user.
It should be noted that the query command may include a field for describing the table name, and the data table name may be determined through the field.
For example, when the interceptor reads the field for describing the table name from the query command, the read data table name is a, and the history data table name obtained by replacement is a _ LIS, at this time, the interceptor sends an instruction for querying data from the a _ LIS data table to the executor, and the executor queries corresponding history data from the a _ LIS data table.
In S14, when the routing policy is the third routing policy, corresponding real-time data is obtained from the real-time database, corresponding historical data is obtained from the historical database, and the real-time data and the historical data are returned.
In this embodiment of the application, when the routing policy is the third routing policy, it indicates that the data to be queried in the query command exists in both the real-time database and the historical database, and therefore, the corresponding data acquired from the real-time database and the historical database needs to be simultaneously acquired, and the queried data is returned to the user.
It can be understood that, when the routing policy is the third routing policy, the order management system sends a command to the executor to obtain corresponding historical data from the historical database and corresponding real-time data from the real-time database, and the executor queries the historical database and the real-time data respectively and returns a query result.
Referring to fig. 4, fig. 4 is a schematic diagram illustrating an implementation flow of S14 in the data query method according to the embodiment of the present application. As shown in fig. 4, in an embodiment of the present application, the step S14 may include the following steps:
s41: and acquiring corresponding real-time data from the real-time database.
S42: determining a data table name according to the query command, replacing the data table name with a corresponding historical data table name, and acquiring corresponding historical data from a historical data table corresponding to the historical data table name.
S43: and fusing the real-time data and the historical data to obtain target data.
S44: and returning the target data.
In the embodiment of the application, the queried data is obtained from different base tables, so that the queried real-time data and the historical data can be fused first, and then the fused data is returned, the fusion process can be based on a plurality of information such as creation time of order data, order codes, order contents and the like, for example, taking the creation time as an example, the queried data is arranged by the method of ascending the creation time and then fused in a page table, and data fusion is realized.
Since the query command is a paging query in many cases, in an embodiment of the present invention, when the query command is a paging query command, the step S14 may include the following steps:
determining the total data number of the real-time data and the total data number of the historical data according to the paging query command;
determining query initial offset and query line number of a real-time data table corresponding to a data table name and query initial offset and query line number of a historical data table corresponding to the historical data table name during paging query according to the line number and page number of each page of paging;
inquiring real-time data according to the initial offset and the number of inquiry lines of the real-time data table corresponding to the data table name;
and querying the historical data according to the initial offset and the query line number of the historical data table corresponding to the name of the historical data table.
In the embodiment of the application, because the query command is a paging query command, the queried data needs to be displayed in a paging manner, so that the total data amount of the real-time data and the total data amount of the historical data need to be determined first, and then the query start offset and the query line number of the real-time data table and the query start offset and the query line number of the historical data table during paging query are determined according to the line number per page and the page number of pages in the paging query command.
For example, it is assumed that the total number of data in the real-time data table is 15 and the total number of data in the historical data table is 15 in the paging query command, and the paging query command is "10 rows per page" and the number of pages to page is 3. "then it can be determined that the query start offset of the first page of the real-time data table is 0, 10 queries are performed, and the query start offset of the second page is 11, 5 queries are performed; the query starting offset in the historical database table is 0, 5 queries are performed, the query starting offset of the third page is 6, 10 queries are performed, and accordingly, corresponding data are queried from the real-time data table and the historical data table and written into a corresponding page (a certain row of a certain page), and page query can be achieved.
It can be seen from the above that, in the data query method provided in the embodiment of the present application, the routing policy is determined by the filed field, that is, it is determined in which database table the order data to be queried is stored, data needs to be acquired from the real-time database or the historical database, and then the corresponding order data is acquired based on the routing policy, so that a data system with separated cold and hot data can be implemented, and the requirements for historical data and real-time data are simultaneously queried, thereby solving the problem that the existing database with separated cold and hot data stored can only query the real-time data and cannot query the historical data.
Referring to fig. 5, fig. 5 is a schematic view illustrating an implementation flow of another data query method provided in the embodiment of the present application, and as shown in fig. 5, different from the embodiment shown in fig. 1, the data query method provided in the embodiment of the present application further includes, before S11, the following steps:
s15: and setting an interceptor to intercept the query command.
S16: judging whether the intercepted query command contains an archive field or not through the interceptor, and if the intercepted query command contains the archive field, executing S11; otherwise, data query is carried out from the real-time database.
In the embodiment of the application, an interceptor (MergeInterreceptor) is arranged in front of an order management system and a database for storing order data, and once the interceptor intercepts a query command, the query command is analyzed, whether the query command contains an archive field or not is judged, if the archive field exists, the interceptor can determine a corresponding routing policy according to the archive field, and sends an operation instruction to an executor of a corresponding database based on the determined routing policy, so that the executor of the corresponding database queries the order data corresponding to the query command and returns the queried order data.
In the embodiment of the application, if the query command does not include an archive field, it is unclear where the data to be queried is stored, so that the query can be performed only according to the existing query mode, that is, data query is performed from the real-time database, at this time, the interceptor directly sends the query command to the executor of the real-time database, and the executor of the real-time database performs data query from the real-time database.
Therefore, in the data query method provided by the embodiment of the application, the query command is intercepted by the interceptor, and the query command is analyzed by the interceptor to determine whether the query command contains the filed field, if the filed field exists, the routing policy corresponding to the query command can be determined according to the filed field, that is, the order data to be queried is correspondingly stored in which database table, the order data needs to be acquired from the real-time database or the historical database, and then the corresponding order data is acquired based on the routing policy, so that the order management system for separating cold data from hot data can be realized, and the requirements of the historical data and the real-time data can be queried.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Fig. 6 shows a block diagram of a data query device provided in an embodiment of the present application, which corresponds to a data query method described in the above embodiment, and only shows portions related to the embodiment of the present application for convenience of description. Referring to fig. 6, the data query apparatus 60 includes: a policy determining unit 61, a first querying unit 62, a second querying unit 63 and a third querying unit 64. Wherein:
the policy determination unit 61 is configured to determine a routing policy according to the archive field in the query command.
The first query unit 62 is configured to, when the routing policy is the first routing policy, obtain corresponding real-time data from a real-time database and return the real-time data.
The second query unit 63 is configured to, when the routing policy is the second routing policy, obtain corresponding historical data from a historical database and return the historical data.
The third query unit 64 is configured to, when the routing policy is a third routing policy, obtain corresponding real-time data from a real-time database, obtain corresponding historical data from a historical database, and return the real-time data and the historical data.
In an embodiment of the present application, the policy determining unit 61 specifically includes: a distribution determining unit and a setting unit. Wherein:
and the distribution determining unit is used for determining the data distribution condition corresponding to the query command according to the filing field.
The setting unit is used for setting a routing strategy according to the data distribution condition corresponding to the query command.
In an embodiment of the application, the distribution determining unit is specifically configured to determine a data distribution according to an archive field and an archive field threshold included in the query command.
In an embodiment of the present application, the second querying unit 63 specifically includes: the device comprises a table name determining unit, a replacing unit and a first acquiring unit. Wherein:
the table name determining unit is used for determining the name of the data table according to the query command;
the replacing unit is used for replacing the data table name with a corresponding historical data table name;
the first acquisition unit is used for acquiring corresponding history data from the history data table corresponding to the history data table name and returning the history data.
In an embodiment of the present application, the third query unit 64 is specifically configured to obtain corresponding real-time data from the real-time database; determining a data table name according to the query command, replacing the data table name with a corresponding historical data table name, and acquiring corresponding historical data from a historical data table corresponding to the historical data table name; fusing the real-time data and the historical data to obtain target data; and returning the target data.
In an embodiment of the present application, when the query command is a paging query command, the third query unit 64 is specifically configured to determine a total data amount of the real-time data and a total data amount of the historical data according to the paging query command; determining query initial offset and query line number of a real-time data table corresponding to a data table name and query initial offset and query line number of a historical data table corresponding to the historical data table name during paging query according to the line number and page number of each page of paging; inquiring real-time data according to the initial offset and the number of inquiry lines of the real-time data table corresponding to the data table name; and querying the historical data according to the initial offset and the query line number of the historical data table corresponding to the name of the historical data table.
In one embodiment of the present application, the data query device 60 further includes: the device comprises an interception unit and a judgment unit.
Wherein:
the interception unit is used for setting an interceptor to intercept the query command.
The judging unit is used for judging whether the intercepted query command contains an archive field or not through the interceptor; if the intercepted query command contains an archive field, determining a routing strategy according to the archive field in the query command; otherwise, data query is carried out from the real-time database.
It can be seen from the above that, according to the data query device provided in the embodiment of the present application, the routing policy is determined by the filed field, that is, it is determined in which database table the order data to be queried is stored, data needs to be acquired from the real-time database or the historical database, and then the corresponding order data is acquired based on the routing policy, so that a data system for separating cold data from hot data can be implemented, and the requirements for querying the historical data and the real-time data are met, thereby solving the problem that the existing database for separating cold data from hot data can only query the real-time data and cannot query the historical data.
Fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application. As shown in fig. 7, the terminal device 7 of this embodiment includes: at least one processor 70 (only one shown in fig. 7), a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, wherein the processor 70 implements the steps of any of the above-described embodiments of the data query method when executing the computer program 72.
Those skilled in the art will appreciate that fig. 7 is only an example of the terminal device 7, and does not constitute a limitation to the terminal device 7, and may include more or less components than those shown, or combine some components, or different components, for example, and may further include input/output devices, network access devices, and the like.
The Processor 70 may be a Central Processing Unit (CPU), and the Processor 70 may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 71 may in some embodiments be an internal storage unit of the terminal device 7, such as a hard disk or a memory of the terminal device 7. In other embodiments, the memory 71 may also be an external storage device of the terminal device 7, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the terminal device 7. The memory 71 is used for storing an operating system, an application program, a BootLoader (BootLoader), data, and other programs, such as program codes of the computer program. The memory 71 may also be used to temporarily store data that has been output or is to be output.
Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored, and when being executed by a processor, the computer program may implement the steps in any of the above-mentioned data query method embodiments.
The embodiments of the present application provide a computer program product, which when running on a terminal device, enables the terminal device to implement the steps in any of the above data query method embodiments when executed.
It should be noted that, for the information interaction, execution process, and other contents between the above-mentioned devices/units, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and specific reference may be made to the part of the embodiment of the method, which is not described herein again.
Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored, and when being executed by a processor, the computer program may implement the steps in any of the above-mentioned data query method embodiments.
The embodiment of the present application provides a computer program product, which when running on a terminal device, enables the terminal device to implement the steps in any of the above data query method embodiments when executed.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one first processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed data query apparatus and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A method for querying data, comprising:
determining a routing policy according to an archive field in the query command;
when the routing strategy is a first routing strategy, acquiring corresponding real-time data from a real-time database and returning the real-time data;
when the routing strategy is a second routing strategy, acquiring corresponding historical data from a historical database and returning the historical data;
and when the routing strategy is a third routing strategy, acquiring corresponding real-time data from a real-time database, acquiring corresponding historical data from a historical database, and returning the real-time data and the historical data.
2. The data query method of claim 1, wherein determining a routing policy based on an archive field in the query command comprises:
determining the data distribution condition corresponding to the query command according to the filing field;
and setting a routing strategy according to the data distribution condition corresponding to the query command.
3. The data query method of claim 2, wherein the determining the data distribution corresponding to the query command according to the archive field specifically includes:
and determining the data distribution condition according to the archive field and the archive field threshold value contained in the query command.
4. The data query method of claim 1, wherein when the routing policy is a second routing policy, acquiring corresponding historical data from a historical database and returning the historical data, comprises:
determining a data table name according to the query command;
replacing the data table name with a corresponding historical data table name;
and acquiring corresponding historical data from the historical data table corresponding to the historical data table name and returning the historical data.
5. The data query method of claim 1, wherein when the routing policy is a third routing policy, obtaining corresponding real-time data from a real-time database, obtaining corresponding historical data from a historical database, and returning the real-time data and the historical data comprises:
acquiring corresponding real-time data from the real-time database;
determining a data table name according to the query command, replacing the data table name with a corresponding historical data table name, and acquiring corresponding historical data from a historical data table corresponding to the historical data table name;
fusing the real-time data and the historical data to obtain target data;
and returning the target data.
6. The data query method of claim 1, wherein when the query command is a paging query command, and when the routing policy is a third routing policy, the method further comprises obtaining corresponding real-time data from a real-time database, obtaining corresponding historical data from a historical database, and returning the real-time data and the historical data, comprising:
determining the total data number of real-time data and the total data number of historical data according to the paging query command;
determining query initial offset and query line number of a real-time data table corresponding to a data table name and query initial offset and query line number of a historical data table corresponding to the historical data table name during paging query according to the line number and page number of each page of paging;
inquiring real-time data according to the initial offset and the number of inquiry lines of the real-time data table corresponding to the data table name;
and querying the historical data according to the initial offset and the query line number of the historical data table corresponding to the name of the historical data table.
7. The data query method of any one of claims 1-6, wherein prior to determining the routing policy based on the archived field in the query command, further comprising:
setting an interceptor to intercept the query command;
judging whether the intercepted query command contains an archive field or not through the interceptor;
if the intercepted query command contains an archive field, determining a routing strategy according to the archive field in the query command; otherwise, data query is carried out from the real-time database.
8. A data query apparatus, comprising:
the strategy determining unit is used for determining a routing strategy according to the filing field in the query command;
the first query unit is used for acquiring corresponding real-time data from a real-time database and returning the real-time data when the routing strategy is the first routing strategy;
the second query unit is used for acquiring corresponding historical data from a historical database and returning the historical data when the routing strategy is a second routing strategy;
and the third query unit is used for acquiring corresponding real-time data from a real-time database and corresponding historical data from a historical database when the routing policy is the third routing policy, and returning the real-time data and the historical data.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the data query method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the data query method according to any one of claims 1 to 7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111549720.XA CN114185898A (en) | 2021-12-17 | 2021-12-17 | Data query method and device and terminal equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111549720.XA CN114185898A (en) | 2021-12-17 | 2021-12-17 | Data query method and device and terminal equipment |
Publications (1)
Publication Number | Publication Date |
---|---|
CN114185898A true CN114185898A (en) | 2022-03-15 |
Family
ID=80544290
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111549720.XA Pending CN114185898A (en) | 2021-12-17 | 2021-12-17 | Data query method and device and terminal equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114185898A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115617860A (en) * | 2022-12-19 | 2023-01-17 | 廊坊奎达信息技术有限公司 | Intelligent query method and system based on big data |
-
2021
- 2021-12-17 CN CN202111549720.XA patent/CN114185898A/en active Pending
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115617860A (en) * | 2022-12-19 | 2023-01-17 | 廊坊奎达信息技术有限公司 | Intelligent query method and system based on big data |
CN115617860B (en) * | 2022-12-19 | 2023-05-05 | 廊坊奎达信息技术有限公司 | Intelligent query method and system based on big data |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8868484B2 (en) | Efficiently updating rows in a data warehouse | |
US20140046928A1 (en) | Query plans with parameter markers in place of object identifiers | |
JP2004070403A (en) | File storage destination volume control method | |
CN105740295B (en) | A kind of processing method and processing device of distributed data | |
US10929370B2 (en) | Index maintenance management of a relational database management system | |
US9514170B1 (en) | Priority queue using two differently-indexed single-index tables | |
US20140101132A1 (en) | Swapping expected and candidate affinities in a query plan cache | |
CN112597153A (en) | Data storage method and device based on block chain and storage medium | |
CN110046181B (en) | Data routing method and device based on database distributed storage | |
CN111026761B (en) | Financial data storage system, processing method and device | |
CN111104426A (en) | Data query method and system | |
CN114185898A (en) | Data query method and device and terminal equipment | |
CN111753141B (en) | Data management method and related equipment | |
EP2662783A1 (en) | Data archiving approach leveraging database layer functionality | |
CN107784091B (en) | Operation authority query method and terminal device | |
US11580123B2 (en) | Columnar techniques for big metadata management | |
CN113326281A (en) | Logistics order data processing method, device, equipment and storage medium | |
US9773034B1 (en) | Large-scale log index | |
CN116450607A (en) | Data processing method, device and storage medium | |
US9092472B1 (en) | Data merge based on logical segregation | |
CN113625967B (en) | Data storage method, data query method and server | |
CN113779120A (en) | Data query method, device, equipment and storage medium | |
CN113761053A (en) | Data query method and device, electronic equipment and storage medium | |
CN114385657A (en) | Data storage method, device and storage medium | |
CN113419792A (en) | Event processing method and device, terminal equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |