CN111797134A - Data query method and device of distributed database and storage medium - Google Patents
Data query method and device of distributed database and storage medium Download PDFInfo
- Publication number
- CN111797134A CN111797134A CN202010583096.4A CN202010583096A CN111797134A CN 111797134 A CN111797134 A CN 111797134A CN 202010583096 A CN202010583096 A CN 202010583096A CN 111797134 A CN111797134 A CN 111797134A
- Authority
- CN
- China
- Prior art keywords
- target
- field
- data
- query
- primary key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000013507 mapping Methods 0.000 claims abstract description 75
- 230000007246 mechanism Effects 0.000 claims description 15
- 230000010076 replication Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 14
- 238000005516 engineering process Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 241000288902 Lemur catta Species 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure relates to a data query method, device and storage medium for a distributed database, and relates to the technical field of electronic information, wherein the method comprises the following steps: the method comprises the steps of obtaining a query request, wherein the query request comprises at least one query field and query conditions corresponding to each query field, if a target query field exists in the query request, querying a target key value of a primary key field matched with the query conditions corresponding to the target query field according to a pre-established target index, wherein the target query field is an appointed non-primary key field, the target index is established when data are written into a distributed database and comprises an index of a mapping relation between the appointed non-primary key field and the primary key field, the target index is taken out of the distributed database, the primary key field is target data of the target key value, and the target data are used as query results corresponding to the query request. According to the method and the device, the non-primary key field can be quickly inquired in the distributed database through the pre-established target index, and the inquiry efficiency is improved.
Description
Technical Field
The present disclosure relates to the field of electronic information technologies, and in particular, to a data query method and apparatus for a distributed database, and a storage medium.
Background
With the development of electronic information technologies such as internet and cloud computing, the Data volume brought correspondingly also shows explosive growth, but the traditional database can not meet the storage requirement of mass Data, so a Distributed database (english: Distributed Data Base, abbreviation: DDB) is usually used for storing mass Data. For example, the Hbase database is a distributed, column-oriented open-source database suitable for unstructured data storage. One row of data is read in the Hbase database by a primary key field (i.e., row key, english: Rowkey) that can uniquely identify one row of data.
In the related technology, the Hbase database only provides query based on the primary key field, the index of the primary key field is single, multi-dimensional query cannot be achieved, and query of non-primary key fields can only be achieved through a Filter or an offline task and the like, so that computing resources are wasted, delay is caused, and query efficiency is low.
Disclosure of Invention
To overcome the problems in the related art, the present disclosure provides a data query method, apparatus, and storage medium for a distributed database.
According to a first aspect of the embodiments of the present disclosure, there is provided a data query method for a distributed database, the method including:
acquiring a query request, wherein the query request comprises at least one query field and a query condition corresponding to each query field;
if the query request has a target query field, querying a target key value of a primary key field matched with a query condition corresponding to the target query field according to a pre-established target index; the target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field;
and taking out the data from the distributed database, wherein the primary key field is the target data of the target key value, and the target data is used as the query result corresponding to the query request.
Optionally, the method further comprises:
writing first data to the distributed database;
if a target establishing field exists in the first data, acquiring the value of the target establishing field in the first data and the key value of a primary key field in the first data, wherein the target establishing field is a designated non-primary key field;
and taking the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field, and storing the mapping in the target index.
Optionally, the distributed database is an Hbase database, and the writing the first data into the distributed database includes:
writing the first data into a pre-written log of the Hbase database;
if the first data has a target establishing field, acquiring a value of the target establishing field in the first data and a key value of a primary key field in the first data, including:
judging whether the target establishing field exists in the first data or not through a Replication mechanism;
and if the first data has a target establishing field, reading the value of the target establishing field in the first data and the key value of the primary key field in the first data from the pre-written log.
Optionally, the using the value of the target creation field in the first data and the key value of the primary key field in the first data as a mapping between the target creation field and the primary key field, and storing the mapping into the target index includes:
and sending the value of the target establishing field in the first data and the key value of the primary key field in the first data to a search server so that the search server takes the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field and stores the mapping in the target index in the search server.
Optionally, the querying, according to a pre-established target index, a target key value of a primary key field matched with a query condition corresponding to the target query field includes:
sending the target query field and the query condition corresponding to the target query field to a search server storing the target index, so that the search server queries a target mapping matched with the query condition corresponding to the target query field in the target index;
and obtaining key values of the primary key fields in the target mapping from the search server, and taking the key values of the primary key fields in the target mapping as the target key values.
According to a second aspect of the embodiments of the present disclosure, there is provided a data query apparatus for a distributed database, the apparatus including:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is configured to acquire a query request, and the query request comprises at least one query field and a query condition corresponding to each query field;
the query module is configured to query a target key value of a primary key field matched with a query condition corresponding to a target query field according to a pre-established target index if the target query field exists in the query request; the target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field;
and the determining module is configured to take out the data from the distributed database, wherein the primary key field is target data of the target key value, and the target data is used as a query result corresponding to the query request.
Optionally, the apparatus further comprises:
a write module configured to write first data to the distributed database;
a second obtaining module, configured to, if a target establishment field exists in the first data, obtain a value of the target establishment field in the first data and a key value of a primary key field in the first data, where the target establishment field is a specified non-primary key field;
and the establishing module is configured to take the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field, and store the mapping in the target index.
Optionally, the writing module is configured to:
writing the first data into a pre-written log of the Hbase database;
the second acquisition module includes:
the judgment sub-module is configured to judge whether the target establishment field exists in the first data through a Replication mechanism;
and the obtaining sub-module is configured to read a value of the target establishment field in the first data and a key value of a primary key field in the first data from the pre-written log if the target establishment field exists in the first data.
Optionally, the establishing module is configured to:
and sending the value of the target establishing field in the first data and the key value of the primary key field in the first data to a search server so that the search server takes the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field and stores the mapping in the target index in the search server.
Optionally, the query module includes:
the sending sub-module is configured to send the target query field and the query condition corresponding to the target query field to a search server storing the target index, so that the search server queries a target mapping matched with the query condition corresponding to the target query field in the target index;
and the query submodule is configured to acquire key values of the primary key fields in the target mapping from the search server and take the key values of the primary key fields in the target mapping as the target key values.
According to a third aspect of the embodiments of the present disclosure, there is provided a data query apparatus for a distributed database, including:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to:
acquiring a query request, wherein the query request comprises at least one query field and a query condition corresponding to each query field;
if the query request has a target query field, querying a target key value of a primary key field matched with a query condition corresponding to the target query field according to a pre-established target index; the target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field;
and taking out the data from the distributed database, wherein the primary key field is the target data of the target key value, and the target data is used as the query result corresponding to the query request.
According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium on which computer program instructions are stored, which program instructions, when executed by a processor, implement the steps of the data query method of the distributed database provided by the first aspect of the present disclosure.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
the method comprises the steps of firstly obtaining a query request comprising at least one query field and query conditions corresponding to each query field, under the condition that a target query field exists in the query request, querying a target key value of a primary key field matched with the query conditions corresponding to the target query field according to a pre-established target index, wherein the target query field is an appointed non-primary key field, the target index is established when data are written into a distributed database and comprises an index of a mapping relation between the appointed non-primary key field and the primary key field, and finally determining target data of which the primary key field is the target key value from the distributed database and using the target data as a query result corresponding to the query request. According to the method and the device, the non-primary key field can be rapidly inquired in the distributed database through the pre-established target index comprising the mapping relation between the non-primary key field and the primary key field, and the inquiry efficiency is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a flow diagram illustrating a method of data querying a distributed database in accordance with an exemplary embodiment;
FIG. 2 is a flow diagram illustrating another method of data querying a distributed database in accordance with an illustrative embodiment;
FIG. 3 is a flow diagram illustrating another method of data querying a distributed database in accordance with an illustrative embodiment;
FIG. 4 is a flow diagram illustrating another method of data querying a distributed database in accordance with an illustrative embodiment;
FIG. 5 is a block diagram illustrating a data querying device of a distributed database in accordance with an exemplary embodiment;
FIG. 6 is a block diagram illustrating another data querying device of a distributed database in accordance with an exemplary embodiment;
FIG. 7 is a block diagram illustrating another data querying device of a distributed database in accordance with an exemplary embodiment;
FIG. 8 is a block diagram illustrating another data querying device of a distributed database in accordance with an exemplary embodiment;
fig. 9 is a block diagram illustrating a data querying device of a distributed database according to an example embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
Before introducing the data query method, apparatus, and storage medium of the distributed database provided by the present disclosure, an application scenario related to various embodiments of the present disclosure is first introduced. The application scenario may include a server and a terminal, and the terminal may be capable of communicating with the server through a wired network or a wireless network. The server is deployed with a distributed database (for example, an Hbase database), and is capable of receiving a query request sent by the terminal and returning a corresponding query result to the terminal. The server may be a local server, or may also be a cloud server or a server cluster, which is not specifically limited in this disclosure. The terminal may be a mobile terminal such as a smart phone, a tablet computer, a smart television, a smart watch, a PDA (Personal Digital Assistant, chinese), a portable computer, or a fixed terminal such as a desktop computer. It should be noted that the execution subject of each embodiment in the present disclosure may be the server described above.
Fig. 1 is a flowchart illustrating a data query method of a distributed database, according to an exemplary embodiment, as shown in fig. 1, the method including the steps of:
in step 101, a query request is obtained, where the query request includes at least one query field and a query condition corresponding to each query field.
For example, when a user needs to search for data in the distributed database, a query request may be sent to the server through the terminal, where the query request may include at least one query field to be queried by the user and a query condition corresponding to each query field. For example, a user needs to search for data in a distributed database, wherein the gender of the data is female and the age of the data is less than 35 years old, and a query request can be sent to a server through a terminal, and the query request comprises two query fields: "sex" and "age", "sex" corresponds to a query condition of "female", and "age" corresponds to a query condition of "less than 35 years". After receiving the query request, the server may analyze the query request to obtain each query field and a query condition corresponding to each query field.
In step 102, if a target query field exists in the query request, a target key value of a primary key field matched with a query condition corresponding to the target query field is queried according to a pre-established target index. The target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field.
For example, it may be determined whether a non-primary key field is present in at least one query field included in the query request. In the distributed database, the primary key field is a row key (Rowkey), and the non-primary key field can be understood as other fields except the row key. If a certain query field in the query request is a primary key field, the query can be directly performed in the distributed database according to the query condition corresponding to the query field, that is, the query is performed according to the primary key field, so as to obtain key values (one or more) of the primary key field matching the query condition corresponding to the query field, which can be recorded as initial key values.
If the query request has a target query field, wherein the target query field is a designated non-primary key field, then the target key values of the primary key fields matched with the query conditions corresponding to the target query field can be queried according to a pre-established target index, and the target key values can be one or more. The target index stores a mapping relationship between a designated non-primary key field and a primary key field, and can be understood as that a large number of mappings are stored in the target index, and each mapping comprises a designated non-primary key field, a value of the designated non-primary key field and a key value corresponding to the primary key field corresponding to the designated non-primary key field. Therefore, the mapping including the target query field in the target index can be queried, then, the mapping matching the query condition corresponding to the target query field is screened out from the mapping including the target query field (that is, the value of the target query field included in the mapping satisfies the corresponding query condition), and the key value corresponding to the primary key field included in the screened mapping is used as the target key value.
It should be noted that the designated non-primary key field may be one or more. For example, the specified non-primary key field may be pre-specified by a user when creating or modifying a table in a distributed database, 100 fields exist in a certain table, wherein the a field is the primary key field, and the B field and the C field are considered by the user to be frequently queried in addition to the primary key field, so that the B field and the C field may be set as the specified non-primary key field. The query behavior of the user can be analyzed statistically, and the field frequently queried by the user is used as the designated non-primary key field, which is not limited in the disclosure. The target index may be understood as, when data is written into the distributed database, if the data to be written includes a designated non-primary key field, the mapping relationship between the designated non-primary key field and the primary key field in the data to be written may be stored in the target index to establish the target index, or may be understood as updating an existing target index, so that the mapping relationship between the designated non-primary key field and the primary key field is stored in the target index.
In step 103, the target data with the primary key field as the target key value is taken out from the distributed database, and the target data is used as the query result corresponding to the query request.
For example, according to the target key value, data with the primary key field as the target key value may be taken out from the distributed database as target data, and then the target data may be used as a query result corresponding to the query request. It should be noted that, in the case that only one query field is included in the query request, and the query field is a target query field (i.e. a specified non-primary key field), the target data whose primary key field is a target key value may be directly used as the query result. If the query request includes a plurality of query fields, and the query fields are all target query fields, then in step 102, a target key value corresponding to each target query field may be determined, and then intersection is performed on the target key values corresponding to each target query field, that is, repeated target key values (which may be one or more) in the target key values corresponding to each target query field are determined, and then target data with the primary key field being the repeated target key values are taken out from the distributed database according to the repeated target key values and serve as a query result. In the case that the query request includes a plurality of query fields, and one of the query fields is a primary key field, and the other query fields are target query fields, the query may be performed in the distributed database according to the query condition corresponding to the primary key field, so as to obtain the key value of the primary key field matching the corresponding query condition, that is, the initial key value. Then, the target key values corresponding to other target query fields are determined through step 102, and then the target key value corresponding to each target query field is intersected with the initial key value, that is, repeated key values (which may be one or more) in the target key value and the initial key value corresponding to each target query field are determined. And then, according to the repeated key values, taking out the data with the primary key fields as the repeated key values from the distributed database, and taking the data as a query result.
In summary, the present disclosure first obtains a query request including at least one query field and a query condition corresponding to each query field, and queries, in a case that a target query field exists in the query request, a target key value of a primary key field matching the query condition corresponding to the target query field according to a pre-established target index, where the target query field is an assigned non-primary key field, the target index is an index that is established when data is written in a distributed database and includes a mapping relationship between the assigned non-primary key field and the primary key field, and finally determines, from the distributed database, target data whose primary key field is the target key value, and uses the target data as a query result corresponding to the query request. According to the method and the device, the non-primary key field can be rapidly inquired in the distributed database through the pre-established target index comprising the mapping relation between the non-primary key field and the primary key field, and the inquiry efficiency is improved.
Fig. 2 is a flowchart illustrating another data query method for a distributed database according to an exemplary embodiment, where the method may further include the following steps, as shown in fig. 2:
in step 104, the first data is written to the distributed database.
In step 105, if the target establishment field exists in the first data, a value of the target establishment field in the first data and a key value of a primary key field in the first data are obtained, and the target establishment field is a designated non-primary key field.
In step 106, the value of the target establishment field in the first data and the key value of the primary key field in the first data are used as a mapping of the target establishment field and the primary key field, and are stored in the target index.
In one implementation, the target index may be established when new data to be written is written to the distributed database. Taking the new data to be written as the first data, for example, the first data is first written into the distributed database, for example, a pre-written Log (Write Ahead Log, abbreviated as WAL) of the distributed database may be written. Meanwhile, whether a target establishing field exists in the first data or not is judged, wherein the target establishing field is a field which belongs to a specified non-primary key field in the first data. If the first data has the target establishing field, acquiring the value of the target establishing field in the first data and the key value of the primary key field in the first data, then taking the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field, and storing the mapping into a target index to establish the target index, which can also be understood as updating the existing target index.
Fig. 3 is a flowchart illustrating another data query method for a distributed database, which may be an Hbase database, as shown in fig. 3, according to an example embodiment.
The implementation of step 104 may be:
writing the first data into a pre-written log of the Hbase database.
Accordingly, step 105 may be implemented by:
in step 1051, it is determined whether a target setup field exists in the first data through a Replication mechanism.
In step 1052, if the target setup field exists in the first data, the value of the target setup field in the first data and the key value of the primary key field in the first data are read from the pre-write log.
For example, for Hbase database, the target index can be established (or the existing target index can be updated) by two mechanisms: a coreprocessor mechanism and a Replication mechanism. Taking a coreprocesser mechanism as an example, before writing first data into a pre-written log of an Hbase database, judging whether a target establishing field exists in the first data or not, if the target establishing field exists in the first data, obtaining a value of the target establishing field in the first data and a key value of a primary key field in the first data, so as to establish a target index, and simultaneously writing the first data into the pre-written log. It can be understood that, in the coreprocesser mechanism, the establishment of the target index and the writing of the first data into the pre-write log are synchronous, both operations must return success to complete the establishment of the target index and the writing of the first data, and if either operation (establishing the target index or writing the first data into the pre-write log) returns failure, neither the update of the target index nor the writing of the first data can be completed.
If the Replication mechanism is used, the first data may be written into the pre-written log of the Hbase database, and then the first data is read from the pre-written log through the Replication mechanism, so as to determine whether the target creation field exists in the first data. If the target establishing field exists in the first data, reading the value of the target establishing field in the first data and the key value of the primary key field in the first data from the pre-written log through a Replication mechanism, and thus establishing the target index. It can be understood that, in the Replication mechanism, the establishment of the target index and the writing of the first data into the pre-written log are asynchronous, and whether the target index is established successfully or not does not affect the writing of the first data, thereby ensuring the reliability of the Hbase database.
In one implementation, the implementation of step 106 may be:
and sending the value of the target establishing field in the first data and the key value of the primary key field in the first data to a search server so that the search server takes the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field and stores the mapping in a target index in the search server.
For example, a specific implementation manner of establishing the target index may be that a server deploying the distributed database first sends a value of the target establishment field in the first data and a key value of the primary key field in the first data to a search server, so that the search server takes the value of the target establishment field in the first data and the key value of the primary key field in the first data as a mapping between the target establishment field and the primary key field, and stores the mapping in the target index stored in the search server. The search server may be understood as any search engine capable of building an inverted index, for example, an open source search engine such as apacesor, elastic search, IndexTank, Katta, Compass, and Summa, which is not limited in this disclosure.
Fig. 4 is a flowchart illustrating another data query method for a distributed database according to an exemplary embodiment, and as shown in fig. 4, the implementation of step 102 may include:
in step 1021, the target query field and the query condition corresponding to the target query field are sent to the search server storing the target index, so that the search server queries the target index for a target map matching the query condition corresponding to the target query field.
In step 1022, the key values of the primary key fields in the target map are obtained from the search server, and the key values of the primary key fields in the target map are used as the target key values.
In an implementation manner, the manner of searching for the target key value according to the target index may be that a server deploying the distributed database first sends the target query field and the query condition corresponding to the target query field to a search server storing the target index. The search server then completes the operation of finding the target key value. Taking the example that the search server is an elastic search (hereinafter abbreviated as ES), first, the ES query target index includes a mapping of a target query field. And then screening out a target mapping matched with the query condition corresponding to the target query field from the mapping containing the target query field, namely, the value of the target query field contained in the target mapping meets the corresponding query condition. Finally, the ES can actively send the key values of the primary key fields in the target mapping to a server for deploying the distributed database, and the server can also actively obtain the key values of the primary key fields in the target mapping from the ES. And after obtaining the key values of the primary key fields in the target mapping, the server takes the key values of the primary key fields in the target mapping as target key values.
In summary, the present disclosure first obtains a query request including at least one query field and a query condition corresponding to each query field, and queries, in a case that a target query field exists in the query request, a target key value of a primary key field matching the query condition corresponding to the target query field according to a pre-established target index, where the target query field is an assigned non-primary key field, the target index is an index that is established when data is written in a distributed database and includes a mapping relationship between the assigned non-primary key field and the primary key field, and finally determines, from the distributed database, target data whose primary key field is the target key value, and uses the target data as a query result corresponding to the query request. According to the method and the device, the non-primary key field can be rapidly inquired in the distributed database through the pre-established target index comprising the mapping relation between the non-primary key field and the primary key field, and the inquiry efficiency is improved.
Fig. 5 is a block diagram illustrating a data query apparatus of a distributed database according to an exemplary embodiment, and as shown in fig. 5, the apparatus 200 includes:
the first obtaining module 201 is configured to obtain a query request, where the query request includes at least one query field and a query condition corresponding to each query field.
The query module 202 is configured to query, according to a pre-established target index, a target key value of a primary key field matching a query condition corresponding to a target query field if the target query field exists in the query request. The target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field.
And the determining module 203 is configured to take out the target data with the primary key field as the target key value from the distributed database, and take the target data as a query result corresponding to the query request.
Fig. 6 is a block diagram illustrating another data query apparatus for a distributed database according to an exemplary embodiment, where the apparatus 200 further includes, as shown in fig. 6:
a write module 204 configured to write the first data to the distributed database.
The second obtaining module 205 is configured to, if the target establishment field exists in the first data, obtain a value of the target establishment field in the first data and a key value of the primary key field in the first data, where the target establishment field is a specified non-primary key field.
The creating module 206 is configured to use the value of the target creating field in the first data and the key value of the primary key field in the first data as a mapping between the target creating field and the primary key field, and store the mapping in the target index.
Fig. 7 is a block diagram illustrating another data querying device for a distributed database according to an example embodiment, where, as shown in fig. 7, the writing module 204 is configured to:
writing the first data into a pre-written log of the Hbase database.
The second obtaining module 205 includes:
the determining sub-module 2051 is configured to determine whether the target establishment field exists in the first data through a Replication mechanism.
The obtaining sub-module 2052 is configured to, if the target establishment field exists in the first data, read a value of the target establishment field in the first data and a key value of a primary key field in the first data from the pre-write log.
In one implementation, the setup module 206 is configured to:
and sending the value of the target establishing field in the first data and the key value of the primary key field in the first data to a search server so that the search server takes the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field and stores the mapping in a target index in the search server.
Fig. 8 is a block diagram illustrating another data query apparatus for a distributed database according to an exemplary embodiment, where, as shown in fig. 8, the query module 202 includes:
the sending sub-module 2021 is configured to send the target query field and the query condition corresponding to the target query field to the search server storing the target index, so that the search server queries the target index for a target map matching the query condition corresponding to the target query field.
The query submodule 2022 is configured to obtain key values of the primary key fields in the target mapping from the search server, and use the key values of the primary key fields in the target mapping as target key values.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
In summary, the present disclosure first obtains a query request including at least one query field and a query condition corresponding to each query field, and queries, in a case that a target query field exists in the query request, a target key value of a primary key field matching the query condition corresponding to the target query field according to a pre-established target index, where the target query field is an assigned non-primary key field, the target index is an index that is established when data is written in a distributed database and includes a mapping relationship between the assigned non-primary key field and the primary key field, and finally determines, from the distributed database, target data whose primary key field is the target key value, and uses the target data as a query result corresponding to the query request. According to the method and the device, the non-primary key field can be rapidly inquired in the distributed database through the pre-established target index comprising the mapping relation between the non-primary key field and the primary key field, and the inquiry efficiency is improved.
The present disclosure also provides a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the data query method of the distributed database provided by the present disclosure.
Fig. 9 is a block diagram illustrating a data query apparatus 1900 for a distributed database according to an example embodiment. For example, the apparatus 1900 may be provided as a server. Referring to fig. 9, the device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by the processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the data query method of the distributed database described above.
The device 1900 may also include a power component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input/output (I/O) interface 1958. The device 1900 may operate based on an operating system, such as Windows Server, stored in memory 1932TM，MacOS XTM，UnixTM，LinuxTM，FreeBSDTMOr the like.
In another exemplary embodiment, a computer program product is also provided, which comprises a computer program executable by a programmable apparatus, the computer program having code portions for performing the above-mentioned data query method of the distributed database when executed by the programmable apparatus.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. A data query method for a distributed database, the method comprising:
acquiring a query request, wherein the query request comprises at least one query field and a query condition corresponding to each query field;
if the query request has a target query field, querying a target key value of a primary key field matched with a query condition corresponding to the target query field according to a pre-established target index; the target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field;
and taking out the data from the distributed database, wherein the primary key field is the target data of the target key value, and the target data is used as the query result corresponding to the query request.
2. The method of claim 1, further comprising:
writing first data to the distributed database;
if a target establishing field exists in the first data, acquiring the value of the target establishing field in the first data and the key value of a primary key field in the first data, wherein the target establishing field is a designated non-primary key field;
and taking the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field, and storing the mapping in the target index.
3. The method of claim 2, wherein the distributed database is an Hbase database, and wherein writing the first data to the distributed database comprises:
writing the first data into a pre-written log of the Hbase database;
if the first data has a target establishing field, acquiring a value of the target establishing field in the first data and a key value of a primary key field in the first data, including:
judging whether the target establishing field exists in the first data or not through a Replication mechanism;
and if the first data has a target establishing field, reading the value of the target establishing field in the first data and the key value of the primary key field in the first data from the pre-written log.
4. The method of claim 2, wherein said mapping the value of the target-established field in the first data and the key value of the primary key field in the first data into the target index as a mapping between the target-established field and the primary key field comprises:
and sending the value of the target establishing field in the first data and the key value of the primary key field in the first data to a search server so that the search server takes the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field and stores the mapping in the target index in the search server.
5. The method of claim 1, wherein the querying, according to the pre-established target index, target key values of primary key fields matching query conditions corresponding to the target query fields comprises:
sending the target query field and the query condition corresponding to the target query field to a search server storing the target index, so that the search server queries a target mapping matched with the query condition corresponding to the target query field in the target index;
and obtaining key values of the primary key fields in the target mapping from the search server, and taking the key values of the primary key fields in the target mapping as the target key values.
6. An apparatus for querying data in a distributed database, the apparatus comprising:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is configured to acquire a query request, and the query request comprises at least one query field and a query condition corresponding to each query field;
the query module is configured to query a target key value of a primary key field matched with a query condition corresponding to a target query field according to a pre-established target index if the target query field exists in the query request; the target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field;
and the determining module is configured to take out the data from the distributed database, wherein the primary key field is target data of the target key value, and the target data is used as a query result corresponding to the query request.
7. The apparatus of claim 6, further comprising:
a write module configured to write first data to the distributed database;
a second obtaining module, configured to, if a target establishment field exists in the first data, obtain a value of the target establishment field in the first data and a key value of a primary key field in the first data, where the target establishment field is a specified non-primary key field;
and the establishing module is configured to take the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field, and store the mapping in the target index.
8. The apparatus of claim 7, wherein the write module is configured to:
writing the first data into a pre-written log of the Hbase database;
the second acquisition module includes:
the judgment sub-module is configured to judge whether the target establishment field exists in the first data through a Replication mechanism;
and the obtaining sub-module is configured to read a value of the target establishment field in the first data and a key value of a primary key field in the first data from the pre-written log if the target establishment field exists in the first data.
9. A data query apparatus for a distributed database, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to:
acquiring a query request, wherein the query request comprises at least one query field and a query condition corresponding to each query field;
if the query request has a target query field, querying a target key value of a primary key field matched with a query condition corresponding to the target query field according to a pre-established target index; the target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field;
and taking out the data from the distributed database, wherein the primary key field is the target data of the target key value, and the target data is used as the query result corresponding to the query request.
10. A computer-readable storage medium, on which computer program instructions are stored, which program instructions, when executed by a processor, carry out the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010583096.4A CN111797134A (en) | 2020-06-23 | 2020-06-23 | Data query method and device of distributed database and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010583096.4A CN111797134A (en) | 2020-06-23 | 2020-06-23 | Data query method and device of distributed database and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111797134A true CN111797134A (en) | 2020-10-20 |
Family
ID=72803744
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010583096.4A Pending CN111797134A (en) | 2020-06-23 | 2020-06-23 | Data query method and device of distributed database and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111797134A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112364014A (en) * | 2020-10-26 | 2021-02-12 | 北京达佳互联信息技术有限公司 | Data query method, device, server and storage medium |
| CN112463827A (en) * | 2020-11-16 | 2021-03-09 | 北京达佳互联信息技术有限公司 | Query method and device, electronic equipment and storage medium |
| CN112905600A (en) * | 2021-03-19 | 2021-06-04 | 腾讯科技（深圳）有限公司 | Data query method and device, storage medium and electronic equipment |
| CN112988666A (en) * | 2021-03-22 | 2021-06-18 | 湖南大学 | Distributed log condition query method and system based on cuckoo filter |
| CN112989171A (en) * | 2021-03-26 | 2021-06-18 | 广东电网有限责任公司 | Data query method, device, equipment and medium |
| CN113723907A (en) * | 2021-08-05 | 2021-11-30 | 中核武汉核电运行技术股份有限公司 | ES quick query method and system based on nuclear power spare part management |
| CN114168589A (en) * | 2021-10-28 | 2022-03-11 | 阿里巴巴（中国）有限公司 | Index construction method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010048789A1 (en) * | 2008-11-03 | 2010-05-06 | 中国移动通信集团公司 | Index building, querying method, device, and system for distributed column memory database |
| CN104102710A (en) * | 2014-07-15 | 2014-10-15 | 浪潮(北京)电子信息产业有限公司 | Massive data query method |
| US20140317093A1 (en) * | 2013-04-22 | 2014-10-23 | Salesforce.Com, Inc. | Facilitating dynamic creation of multi-column index tables and management of customer queries in an on-demand services environment |
| WO2017067117A1 (en) * | 2015-10-21 | 2017-04-27 | 华为技术有限公司 | Data query method and device |
| CN109284351A (en) * | 2018-08-14 | 2019-01-29 | 青海大学 | A kind of data query method based on HBase database |
| CN109885570A (en) * | 2019-01-30 | 2019-06-14 | 南京邮电大学 | A kind of multi-Dimensional Range querying method of secondary index combination hash table |
| CN110287198A (en) * | 2019-07-01 | 2019-09-27 | 四川新网银行股份有限公司 | Finance data indexing means based on HBase database |
-
2020
- 2020-06-23 CN CN202010583096.4A patent/CN111797134A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010048789A1 (en) * | 2008-11-03 | 2010-05-06 | 中国移动通信集团公司 | Index building, querying method, device, and system for distributed column memory database |
| CN101727465A (en) * | 2008-11-03 | 2010-06-09 | 中国移动通信集团公司 | Methods for establishing and inquiring index of distributed column storage database, device and system thereof |
| US20140317093A1 (en) * | 2013-04-22 | 2014-10-23 | Salesforce.Com, Inc. | Facilitating dynamic creation of multi-column index tables and management of customer queries in an on-demand services environment |
| CN104102710A (en) * | 2014-07-15 | 2014-10-15 | 浪潮(北京)电子信息产业有限公司 | Massive data query method |
| WO2017067117A1 (en) * | 2015-10-21 | 2017-04-27 | 华为技术有限公司 | Data query method and device |
| CN109284351A (en) * | 2018-08-14 | 2019-01-29 | 青海大学 | A kind of data query method based on HBase database |
| CN109885570A (en) * | 2019-01-30 | 2019-06-14 | 南京邮电大学 | A kind of multi-Dimensional Range querying method of secondary index combination hash table |
| CN110287198A (en) * | 2019-07-01 | 2019-09-27 | 四川新网银行股份有限公司 | Finance data indexing means based on HBase database |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112364014A (en) * | 2020-10-26 | 2021-02-12 | 北京达佳互联信息技术有限公司 | Data query method, device, server and storage medium |
| CN112364014B (en) * | 2020-10-26 | 2024-01-02 | 北京达佳互联信息技术有限公司 | Data query method, device, server and storage medium |
| CN112463827A (en) * | 2020-11-16 | 2021-03-09 | 北京达佳互联信息技术有限公司 | Query method and device, electronic equipment and storage medium |
| CN112463827B (en) * | 2020-11-16 | 2024-03-12 | 北京达佳互联信息技术有限公司 | Query method, query device, electronic equipment and storage medium |
| CN112905600A (en) * | 2021-03-19 | 2021-06-04 | 腾讯科技（深圳）有限公司 | Data query method and device, storage medium and electronic equipment |
| CN112905600B (en) * | 2021-03-19 | 2023-09-26 | 腾讯科技（深圳）有限公司 | Data query method and device, storage medium and electronic equipment |
| CN112988666A (en) * | 2021-03-22 | 2021-06-18 | 湖南大学 | Distributed log condition query method and system based on cuckoo filter |
| CN112988666B (en) * | 2021-03-22 | 2022-04-22 | 湖南大学 | Distributed log condition query method and system based on cuckoo filter |
| CN112989171A (en) * | 2021-03-26 | 2021-06-18 | 广东电网有限责任公司 | Data query method, device, equipment and medium |
| CN112989171B (en) * | 2021-03-26 | 2024-08-30 | 广东电网有限责任公司 | Data query method, device, equipment and medium |
| CN113723907A (en) * | 2021-08-05 | 2021-11-30 | 中核武汉核电运行技术股份有限公司 | ES quick query method and system based on nuclear power spare part management |
| CN114168589A (en) * | 2021-10-28 | 2022-03-11 | 阿里巴巴（中国）有限公司 | Index construction method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111797134A (en) | Data query method and device of distributed database and storage medium | |
| US11068441B2 (en) | Caseless file lookup in a distributed file system | |
| US8321444B2 (en) | Federated search | |
| EP4158497A1 (en) | Automatic transformation of data by patterns | |
| US10909086B2 (en) | File lookup in a distributed file system | |
| CN108536745B (en) | Shell-based data table extraction method, terminal, equipment and storage medium | |
| CN107329987A (en) | A kind of search system based on mongo databases | |
| CN109361628B (en) | Message assembling method and device, computer equipment and storage medium | |
| CN111984659B (en) | Data updating method, device, computer equipment and storage medium | |
| CN106547646B (en) | Data backup and recovery method and data backup and recovery device | |
| CN112883030A (en) | Data collection method and device, computer equipment and storage medium | |
| CN113220659A (en) | Data migration method, system, electronic device and storage medium | |
| CN109726295A (en) | Brand knowledge map display methods, device, figure server and storage medium | |
| CN111209061A (en) | Method and device for filling in user information, computer equipment and storage medium | |
| CN111858581B (en) | Paging query method and device, storage medium and electronic equipment | |
| CN112783866B (en) | Data reading method, device, computer equipment and storage medium | |
| CN109697234B (en) | Multi-attribute information query method, device, server and medium for entity | |
| CN110737662A (en) | data analysis method, device, server and computer storage medium | |
| CN111008198A (en) | Service data acquisition method and device, storage medium and electronic equipment | |
| CN110471708B (en) | Method and device for acquiring configuration items based on reusable components | |
| US9460139B2 (en) | Distributed storage system with pluggable query processing | |
| CN117009430A (en) | Data management method, device, storage medium and electronic equipment | |
| CN113094367A (en) | Data processing method and device and server | |
| CN110347333A (en) | Improve method, apparatus, computer equipment and the storage medium of clone's mirror image performance | |
| CN113626046B (en) | Container mirror image updating method, device, storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |