CN111797134A - Data query method and device of distributed database and storage medium - Google Patents

Data query method and device of distributed database and storage medium Download PDF

Info

Publication number
CN111797134A
CN111797134A CN202010583096.4A CN202010583096A CN111797134A CN 111797134 A CN111797134 A CN 111797134A CN 202010583096 A CN202010583096 A CN 202010583096A CN 111797134 A CN111797134 A CN 111797134A
Authority
CN
China
Prior art keywords
target
field
data
query
primary key
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010583096.4A
Other languages
Chinese (zh)
Inventor
牛豫林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Xiaomi Pinecone Electronic Co Ltd
Original Assignee
Beijing Xiaomi Pinecone Electronic Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Xiaomi Pinecone Electronic Co Ltd filed Critical Beijing Xiaomi Pinecone Electronic Co Ltd
Priority to CN202010583096.4A priority Critical patent/CN111797134A/en
Publication of CN111797134A publication Critical patent/CN111797134A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2458Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2471Distributed queries
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present disclosure relates to a data query method, device and storage medium for a distributed database, and relates to the technical field of electronic information, wherein the method comprises the following steps: the method comprises the steps of obtaining a query request, wherein the query request comprises at least one query field and query conditions corresponding to each query field, if a target query field exists in the query request, querying a target key value of a primary key field matched with the query conditions corresponding to the target query field according to a pre-established target index, wherein the target query field is an appointed non-primary key field, the target index is established when data are written into a distributed database and comprises an index of a mapping relation between the appointed non-primary key field and the primary key field, the target index is taken out of the distributed database, the primary key field is target data of the target key value, and the target data are used as query results corresponding to the query request. According to the method and the device, the non-primary key field can be quickly inquired in the distributed database through the pre-established target index, and the inquiry efficiency is improved.

Description

Data query method and device of distributed database and storage medium
Technical Field
The present disclosure relates to the field of electronic information technologies, and in particular, to a data query method and apparatus for a distributed database, and a storage medium.
Background
With the development of electronic information technologies such as internet and cloud computing, the Data volume brought correspondingly also shows explosive growth, but the traditional database can not meet the storage requirement of mass Data, so a Distributed database (english: Distributed Data Base, abbreviation: DDB) is usually used for storing mass Data. For example, the Hbase database is a distributed, column-oriented open-source database suitable for unstructured data storage. One row of data is read in the Hbase database by a primary key field (i.e., row key, english: Rowkey) that can uniquely identify one row of data.
In the related technology, the Hbase database only provides query based on the primary key field, the index of the primary key field is single, multi-dimensional query cannot be achieved, and query of non-primary key fields can only be achieved through a Filter or an offline task and the like, so that computing resources are wasted, delay is caused, and query efficiency is low.
Disclosure of Invention
To overcome the problems in the related art, the present disclosure provides a data query method, apparatus, and storage medium for a distributed database.
According to a first aspect of the embodiments of the present disclosure, there is provided a data query method for a distributed database, the method including:
acquiring a query request, wherein the query request comprises at least one query field and a query condition corresponding to each query field;
if the query request has a target query field, querying a target key value of a primary key field matched with a query condition corresponding to the target query field according to a pre-established target index; the target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field;
and taking out the data from the distributed database, wherein the primary key field is the target data of the target key value, and the target data is used as the query result corresponding to the query request.
Optionally, the method further comprises:
writing first data to the distributed database;
if a target establishing field exists in the first data, acquiring the value of the target establishing field in the first data and the key value of a primary key field in the first data, wherein the target establishing field is a designated non-primary key field;
and taking the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field, and storing the mapping in the target index.
Optionally, the distributed database is an Hbase database, and the writing the first data into the distributed database includes:
writing the first data into a pre-written log of the Hbase database;
if the first data has a target establishing field, acquiring a value of the target establishing field in the first data and a key value of a primary key field in the first data, including:
judging whether the target establishing field exists in the first data or not through a Replication mechanism;
and if the first data has a target establishing field, reading the value of the target establishing field in the first data and the key value of the primary key field in the first data from the pre-written log.
Optionally, the using the value of the target creation field in the first data and the key value of the primary key field in the first data as a mapping between the target creation field and the primary key field, and storing the mapping into the target index includes:
and sending the value of the target establishing field in the first data and the key value of the primary key field in the first data to a search server so that the search server takes the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field and stores the mapping in the target index in the search server.
Optionally, the querying, according to a pre-established target index, a target key value of a primary key field matched with a query condition corresponding to the target query field includes:
sending the target query field and the query condition corresponding to the target query field to a search server storing the target index, so that the search server queries a target mapping matched with the query condition corresponding to the target query field in the target index;
and obtaining key values of the primary key fields in the target mapping from the search server, and taking the key values of the primary key fields in the target mapping as the target key values.
According to a second aspect of the embodiments of the present disclosure, there is provided a data query apparatus for a distributed database, the apparatus including:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is configured to acquire a query request, and the query request comprises at least one query field and a query condition corresponding to each query field;
the query module is configured to query a target key value of a primary key field matched with a query condition corresponding to a target query field according to a pre-established target index if the target query field exists in the query request; the target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field;
and the determining module is configured to take out the data from the distributed database, wherein the primary key field is target data of the target key value, and the target data is used as a query result corresponding to the query request.
Optionally, the apparatus further comprises:
a write module configured to write first data to the distributed database;
a second obtaining module, configured to, if a target establishment field exists in the first data, obtain a value of the target establishment field in the first data and a key value of a primary key field in the first data, where the target establishment field is a specified non-primary key field;
and the establishing module is configured to take the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field, and store the mapping in the target index.
Optionally, the writing module is configured to:
writing the first data into a pre-written log of the Hbase database;
the second acquisition module includes:
the judgment sub-module is configured to judge whether the target establishment field exists in the first data through a Replication mechanism;
and the obtaining sub-module is configured to read a value of the target establishment field in the first data and a key value of a primary key field in the first data from the pre-written log if the target establishment field exists in the first data.
Optionally, the establishing module is configured to:
and sending the value of the target establishing field in the first data and the key value of the primary key field in the first data to a search server so that the search server takes the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field and stores the mapping in the target index in the search server.
Optionally, the query module includes:
the sending sub-module is configured to send the target query field and the query condition corresponding to the target query field to a search server storing the target index, so that the search server queries a target mapping matched with the query condition corresponding to the target query field in the target index;
and the query submodule is configured to acquire key values of the primary key fields in the target mapping from the search server and take the key values of the primary key fields in the target mapping as the target key values.
According to a third aspect of the embodiments of the present disclosure, there is provided a data query apparatus for a distributed database, including:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to:
acquiring a query request, wherein the query request comprises at least one query field and a query condition corresponding to each query field;
if the query request has a target query field, querying a target key value of a primary key field matched with a query condition corresponding to the target query field according to a pre-established target index; the target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field;
and taking out the data from the distributed database, wherein the primary key field is the target data of the target key value, and the target data is used as the query result corresponding to the query request.
According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium on which computer program instructions are stored, which program instructions, when executed by a processor, implement the steps of the data query method of the distributed database provided by the first aspect of the present disclosure.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
the method comprises the steps of firstly obtaining a query request comprising at least one query field and query conditions corresponding to each query field, under the condition that a target query field exists in the query request, querying a target key value of a primary key field matched with the query conditions corresponding to the target query field according to a pre-established target index, wherein the target query field is an appointed non-primary key field, the target index is established when data are written into a distributed database and comprises an index of a mapping relation between the appointed non-primary key field and the primary key field, and finally determining target data of which the primary key field is the target key value from the distributed database and using the target data as a query result corresponding to the query request. According to the method and the device, the non-primary key field can be rapidly inquired in the distributed database through the pre-established target index comprising the mapping relation between the non-primary key field and the primary key field, and the inquiry efficiency is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a flow diagram illustrating a method of data querying a distributed database in accordance with an exemplary embodiment;
FIG. 2 is a flow diagram illustrating another method of data querying a distributed database in accordance with an illustrative embodiment;
FIG. 3 is a flow diagram illustrating another method of data querying a distributed database in accordance with an illustrative embodiment;
FIG. 4 is a flow diagram illustrating another method of data querying a distributed database in accordance with an illustrative embodiment;
FIG. 5 is a block diagram illustrating a data querying device of a distributed database in accordance with an exemplary embodiment;
FIG. 6 is a block diagram illustrating another data querying device of a distributed database in accordance with an exemplary embodiment;
FIG. 7 is a block diagram illustrating another data querying device of a distributed database in accordance with an exemplary embodiment;
FIG. 8 is a block diagram illustrating another data querying device of a distributed database in accordance with an exemplary embodiment;
fig. 9 is a block diagram illustrating a data querying device of a distributed database according to an example embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
Before introducing the data query method, apparatus, and storage medium of the distributed database provided by the present disclosure, an application scenario related to various embodiments of the present disclosure is first introduced. The application scenario may include a server and a terminal, and the terminal may be capable of communicating with the server through a wired network or a wireless network. The server is deployed with a distributed database (for example, an Hbase database), and is capable of receiving a query request sent by the terminal and returning a corresponding query result to the terminal. The server may be a local server, or may also be a cloud server or a server cluster, which is not specifically limited in this disclosure. The terminal may be a mobile terminal such as a smart phone, a tablet computer, a smart television, a smart watch, a PDA (Personal Digital Assistant, chinese), a portable computer, or a fixed terminal such as a desktop computer. It should be noted that the execution subject of each embodiment in the present disclosure may be the server described above.
Fig. 1 is a flowchart illustrating a data query method of a distributed database, according to an exemplary embodiment, as shown in fig. 1, the method including the steps of:
in step 101, a query request is obtained, where the query request includes at least one query field and a query condition corresponding to each query field.
For example, when a user needs to search for data in the distributed database, a query request may be sent to the server through the terminal, where the query request may include at least one query field to be queried by the user and a query condition corresponding to each query field. For example, a user needs to search for data in a distributed database, wherein the gender of the data is female and the age of the data is less than 35 years old, and a query request can be sent to a server through a terminal, and the query request comprises two query fields: "sex" and "age", "sex" corresponds to a query condition of "female", and "age" corresponds to a query condition of "less than 35 years". After receiving the query request, the server may analyze the query request to obtain each query field and a query condition corresponding to each query field.
In step 102, if a target query field exists in the query request, a target key value of a primary key field matched with a query condition corresponding to the target query field is queried according to a pre-established target index. The target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field.
For example, it may be determined whether a non-primary key field is present in at least one query field included in the query request. In the distributed database, the primary key field is a row key (Rowkey), and the non-primary key field can be understood as other fields except the row key. If a certain query field in the query request is a primary key field, the query can be directly performed in the distributed database according to the query condition corresponding to the query field, that is, the query is performed according to the primary key field, so as to obtain key values (one or more) of the primary key field matching the query condition corresponding to the query field, which can be recorded as initial key values.
If the query request has a target query field, wherein the target query field is a designated non-primary key field, then the target key values of the primary key fields matched with the query conditions corresponding to the target query field can be queried according to a pre-established target index, and the target key values can be one or more. The target index stores a mapping relationship between a designated non-primary key field and a primary key field, and can be understood as that a large number of mappings are stored in the target index, and each mapping comprises a designated non-primary key field, a value of the designated non-primary key field and a key value corresponding to the primary key field corresponding to the designated non-primary key field. Therefore, the mapping including the target query field in the target index can be queried, then, the mapping matching the query condition corresponding to the target query field is screened out from the mapping including the target query field (that is, the value of the target query field included in the mapping satisfies the corresponding query condition), and the key value corresponding to the primary key field included in the screened mapping is used as the target key value.
It should be noted that the designated non-primary key field may be one or more. For example, the specified non-primary key field may be pre-specified by a user when creating or modifying a table in a distributed database, 100 fields exist in a certain table, wherein the a field is the primary key field, and the B field and the C field are considered by the user to be frequently queried in addition to the primary key field, so that the B field and the C field may be set as the specified non-primary key field. The query behavior of the user can be analyzed statistically, and the field frequently queried by the user is used as the designated non-primary key field, which is not limited in the disclosure. The target index may be understood as, when data is written into the distributed database, if the data to be written includes a designated non-primary key field, the mapping relationship between the designated non-primary key field and the primary key field in the data to be written may be stored in the target index to establish the target index, or may be understood as updating an existing target index, so that the mapping relationship between the designated non-primary key field and the primary key field is stored in the target index.
In step 103, the target data with the primary key field as the target key value is taken out from the distributed database, and the target data is used as the query result corresponding to the query request.
For example, according to the target key value, data with the primary key field as the target key value may be taken out from the distributed database as target data, and then the target data may be used as a query result corresponding to the query request. It should be noted that, in the case that only one query field is included in the query request, and the query field is a target query field (i.e. a specified non-primary key field), the target data whose primary key field is a target key value may be directly used as the query result. If the query request includes a plurality of query fields, and the query fields are all target query fields, then in step 102, a target key value corresponding to each target query field may be determined, and then intersection is performed on the target key values corresponding to each target query field, that is, repeated target key values (which may be one or more) in the target key values corresponding to each target query field are determined, and then target data with the primary key field being the repeated target key values are taken out from the distributed database according to the repeated target key values and serve as a query result. In the case that the query request includes a plurality of query fields, and one of the query fields is a primary key field, and the other query fields are target query fields, the query may be performed in the distributed database according to the query condition corresponding to the primary key field, so as to obtain the key value of the primary key field matching the corresponding query condition, that is, the initial key value. Then, the target key values corresponding to other target query fields are determined through step 102, and then the target key value corresponding to each target query field is intersected with the initial key value, that is, repeated key values (which may be one or more) in the target key value and the initial key value corresponding to each target query field are determined. And then, according to the repeated key values, taking out the data with the primary key fields as the repeated key values from the distributed database, and taking the data as a query result.
In summary, the present disclosure first obtains a query request including at least one query field and a query condition corresponding to each query field, and queries, in a case that a target query field exists in the query request, a target key value of a primary key field matching the query condition corresponding to the target query field according to a pre-established target index, where the target query field is an assigned non-primary key field, the target index is an index that is established when data is written in a distributed database and includes a mapping relationship between the assigned non-primary key field and the primary key field, and finally determines, from the distributed database, target data whose primary key field is the target key value, and uses the target data as a query result corresponding to the query request. According to the method and the device, the non-primary key field can be rapidly inquired in the distributed database through the pre-established target index comprising the mapping relation between the non-primary key field and the primary key field, and the inquiry efficiency is improved.
Fig. 2 is a flowchart illustrating another data query method for a distributed database according to an exemplary embodiment, where the method may further include the following steps, as shown in fig. 2:
in step 104, the first data is written to the distributed database.
In step 105, if the target establishment field exists in the first data, a value of the target establishment field in the first data and a key value of a primary key field in the first data are obtained, and the target establishment field is a designated non-primary key field.
In step 106, the value of the target establishment field in the first data and the key value of the primary key field in the first data are used as a mapping of the target establishment field and the primary key field, and are stored in the target index.
In one implementation, the target index may be established when new data to be written is written to the distributed database. Taking the new data to be written as the first data, for example, the first data is first written into the distributed database, for example, a pre-written Log (Write Ahead Log, abbreviated as WAL) of the distributed database may be written. Meanwhile, whether a target establishing field exists in the first data or not is judged, wherein the target establishing field is a field which belongs to a specified non-primary key field in the first data. If the first data has the target establishing field, acquiring the value of the target establishing field in the first data and the key value of the primary key field in the first data, then taking the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field, and storing the mapping into a target index to establish the target index, which can also be understood as updating the existing target index.
Fig. 3 is a flowchart illustrating another data query method for a distributed database, which may be an Hbase database, as shown in fig. 3, according to an example embodiment.
The implementation of step 104 may be:
writing the first data into a pre-written log of the Hbase database.
Accordingly, step 105 may be implemented by:
in step 1051, it is determined whether a target setup field exists in the first data through a Replication mechanism.
In step 1052, if the target setup field exists in the first data, the value of the target setup field in the first data and the key value of the primary key field in the first data are read from the pre-write log.
For example, for Hbase database, the target index can be established (or the existing target index can be updated) by two mechanisms: a coreprocessor mechanism and a Replication mechanism. Taking a coreprocesser mechanism as an example, before writing first data into a pre-written log of an Hbase database, judging whether a target establishing field exists in the first data or not, if the target establishing field exists in the first data, obtaining a value of the target establishing field in the first data and a key value of a primary key field in the first data, so as to establish a target index, and simultaneously writing the first data into the pre-written log. It can be understood that, in the coreprocesser mechanism, the establishment of the target index and the writing of the first data into the pre-write log are synchronous, both operations must return success to complete the establishment of the target index and the writing of the first data, and if either operation (establishing the target index or writing the first data into the pre-write log) returns failure, neither the update of the target index nor the writing of the first data can be completed.
If the Replication mechanism is used, the first data may be written into the pre-written log of the Hbase database, and then the first data is read from the pre-written log through the Replication mechanism, so as to determine whether the target creation field exists in the first data. If the target establishing field exists in the first data, reading the value of the target establishing field in the first data and the key value of the primary key field in the first data from the pre-written log through a Replication mechanism, and thus establishing the target index. It can be understood that, in the Replication mechanism, the establishment of the target index and the writing of the first data into the pre-written log are asynchronous, and whether the target index is established successfully or not does not affect the writing of the first data, thereby ensuring the reliability of the Hbase database.
In one implementation, the implementation of step 106 may be:
and sending the value of the target establishing field in the first data and the key value of the primary key field in the first data to a search server so that the search server takes the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field and stores the mapping in a target index in the search server.
For example, a specific implementation manner of establishing the target index may be that a server deploying the distributed database first sends a value of the target establishment field in the first data and a key value of the primary key field in the first data to a search server, so that the search server takes the value of the target establishment field in the first data and the key value of the primary key field in the first data as a mapping between the target establishment field and the primary key field, and stores the mapping in the target index stored in the search server. The search server may be understood as any search engine capable of building an inverted index, for example, an open source search engine such as apacesor, elastic search, IndexTank, Katta, Compass, and Summa, which is not limited in this disclosure.
Fig. 4 is a flowchart illustrating another data query method for a distributed database according to an exemplary embodiment, and as shown in fig. 4, the implementation of step 102 may include:
in step 1021, the target query field and the query condition corresponding to the target query field are sent to the search server storing the target index, so that the search server queries the target index for a target map matching the query condition corresponding to the target query field.
In step 1022, the key values of the primary key fields in the target map are obtained from the search server, and the key values of the primary key fields in the target map are used as the target key values.
In an implementation manner, the manner of searching for the target key value according to the target index may be that a server deploying the distributed database first sends the target query field and the query condition corresponding to the target query field to a search server storing the target index. The search server then completes the operation of finding the target key value. Taking the example that the search server is an elastic search (hereinafter abbreviated as ES), first, the ES query target index includes a mapping of a target query field. And then screening out a target mapping matched with the query condition corresponding to the target query field from the mapping containing the target query field, namely, the value of the target query field contained in the target mapping meets the corresponding query condition. Finally, the ES can actively send the key values of the primary key fields in the target mapping to a server for deploying the distributed database, and the server can also actively obtain the key values of the primary key fields in the target mapping from the ES. And after obtaining the key values of the primary key fields in the target mapping, the server takes the key values of the primary key fields in the target mapping as target key values.
In summary, the present disclosure first obtains a query request including at least one query field and a query condition corresponding to each query field, and queries, in a case that a target query field exists in the query request, a target key value of a primary key field matching the query condition corresponding to the target query field according to a pre-established target index, where the target query field is an assigned non-primary key field, the target index is an index that is established when data is written in a distributed database and includes a mapping relationship between the assigned non-primary key field and the primary key field, and finally determines, from the distributed database, target data whose primary key field is the target key value, and uses the target data as a query result corresponding to the query request. According to the method and the device, the non-primary key field can be rapidly inquired in the distributed database through the pre-established target index comprising the mapping relation between the non-primary key field and the primary key field, and the inquiry efficiency is improved.
Fig. 5 is a block diagram illustrating a data query apparatus of a distributed database according to an exemplary embodiment, and as shown in fig. 5, the apparatus 200 includes:
the first obtaining module 201 is configured to obtain a query request, where the query request includes at least one query field and a query condition corresponding to each query field.
The query module 202 is configured to query, according to a pre-established target index, a target key value of a primary key field matching a query condition corresponding to a target query field if the target query field exists in the query request. The target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field.
And the determining module 203 is configured to take out the target data with the primary key field as the target key value from the distributed database, and take the target data as a query result corresponding to the query request.
Fig. 6 is a block diagram illustrating another data query apparatus for a distributed database according to an exemplary embodiment, where the apparatus 200 further includes, as shown in fig. 6:
a write module 204 configured to write the first data to the distributed database.
The second obtaining module 205 is configured to, if the target establishment field exists in the first data, obtain a value of the target establishment field in the first data and a key value of the primary key field in the first data, where the target establishment field is a specified non-primary key field.
The creating module 206 is configured to use the value of the target creating field in the first data and the key value of the primary key field in the first data as a mapping between the target creating field and the primary key field, and store the mapping in the target index.
Fig. 7 is a block diagram illustrating another data querying device for a distributed database according to an example embodiment, where, as shown in fig. 7, the writing module 204 is configured to:
writing the first data into a pre-written log of the Hbase database.
The second obtaining module 205 includes:
the determining sub-module 2051 is configured to determine whether the target establishment field exists in the first data through a Replication mechanism.
The obtaining sub-module 2052 is configured to, if the target establishment field exists in the first data, read a value of the target establishment field in the first data and a key value of a primary key field in the first data from the pre-write log.
In one implementation, the setup module 206 is configured to:
and sending the value of the target establishing field in the first data and the key value of the primary key field in the first data to a search server so that the search server takes the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field and stores the mapping in a target index in the search server.
Fig. 8 is a block diagram illustrating another data query apparatus for a distributed database according to an exemplary embodiment, where, as shown in fig. 8, the query module 202 includes:
the sending sub-module 2021 is configured to send the target query field and the query condition corresponding to the target query field to the search server storing the target index, so that the search server queries the target index for a target map matching the query condition corresponding to the target query field.
The query submodule 2022 is configured to obtain key values of the primary key fields in the target mapping from the search server, and use the key values of the primary key fields in the target mapping as target key values.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
In summary, the present disclosure first obtains a query request including at least one query field and a query condition corresponding to each query field, and queries, in a case that a target query field exists in the query request, a target key value of a primary key field matching the query condition corresponding to the target query field according to a pre-established target index, where the target query field is an assigned non-primary key field, the target index is an index that is established when data is written in a distributed database and includes a mapping relationship between the assigned non-primary key field and the primary key field, and finally determines, from the distributed database, target data whose primary key field is the target key value, and uses the target data as a query result corresponding to the query request. According to the method and the device, the non-primary key field can be rapidly inquired in the distributed database through the pre-established target index comprising the mapping relation between the non-primary key field and the primary key field, and the inquiry efficiency is improved.
The present disclosure also provides a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the data query method of the distributed database provided by the present disclosure.
Fig. 9 is a block diagram illustrating a data query apparatus 1900 for a distributed database according to an example embodiment. For example, the apparatus 1900 may be provided as a server. Referring to fig. 9, the device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by the processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the data query method of the distributed database described above.
The device 1900 may also include a power component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input/output (I/O) interface 1958. The device 1900 may operate based on an operating system, such as Windows Server, stored in memory 1932TM,MacOS XTM,UnixTM,LinuxTM,FreeBSDTMOr the like.
In another exemplary embodiment, a computer program product is also provided, which comprises a computer program executable by a programmable apparatus, the computer program having code portions for performing the above-mentioned data query method of the distributed database when executed by the programmable apparatus.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (10)

1. A data query method for a distributed database, the method comprising:
acquiring a query request, wherein the query request comprises at least one query field and a query condition corresponding to each query field;
if the query request has a target query field, querying a target key value of a primary key field matched with a query condition corresponding to the target query field according to a pre-established target index; the target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field;
and taking out the data from the distributed database, wherein the primary key field is the target data of the target key value, and the target data is used as the query result corresponding to the query request.
2. The method of claim 1, further comprising:
writing first data to the distributed database;
if a target establishing field exists in the first data, acquiring the value of the target establishing field in the first data and the key value of a primary key field in the first data, wherein the target establishing field is a designated non-primary key field;
and taking the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field, and storing the mapping in the target index.
3. The method of claim 2, wherein the distributed database is an Hbase database, and wherein writing the first data to the distributed database comprises:
writing the first data into a pre-written log of the Hbase database;
if the first data has a target establishing field, acquiring a value of the target establishing field in the first data and a key value of a primary key field in the first data, including:
judging whether the target establishing field exists in the first data or not through a Replication mechanism;
and if the first data has a target establishing field, reading the value of the target establishing field in the first data and the key value of the primary key field in the first data from the pre-written log.
4. The method of claim 2, wherein said mapping the value of the target-established field in the first data and the key value of the primary key field in the first data into the target index as a mapping between the target-established field and the primary key field comprises:
and sending the value of the target establishing field in the first data and the key value of the primary key field in the first data to a search server so that the search server takes the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field and stores the mapping in the target index in the search server.
5. The method of claim 1, wherein the querying, according to the pre-established target index, target key values of primary key fields matching query conditions corresponding to the target query fields comprises:
sending the target query field and the query condition corresponding to the target query field to a search server storing the target index, so that the search server queries a target mapping matched with the query condition corresponding to the target query field in the target index;
and obtaining key values of the primary key fields in the target mapping from the search server, and taking the key values of the primary key fields in the target mapping as the target key values.
6. An apparatus for querying data in a distributed database, the apparatus comprising:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is configured to acquire a query request, and the query request comprises at least one query field and a query condition corresponding to each query field;
the query module is configured to query a target key value of a primary key field matched with a query condition corresponding to a target query field according to a pre-established target index if the target query field exists in the query request; the target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field;
and the determining module is configured to take out the data from the distributed database, wherein the primary key field is target data of the target key value, and the target data is used as a query result corresponding to the query request.
7. The apparatus of claim 6, further comprising:
a write module configured to write first data to the distributed database;
a second obtaining module, configured to, if a target establishment field exists in the first data, obtain a value of the target establishment field in the first data and a key value of a primary key field in the first data, where the target establishment field is a specified non-primary key field;
and the establishing module is configured to take the value of the target establishing field in the first data and the key value of the primary key field in the first data as a mapping of the target establishing field and the primary key field, and store the mapping in the target index.
8. The apparatus of claim 7, wherein the write module is configured to:
writing the first data into a pre-written log of the Hbase database;
the second acquisition module includes:
the judgment sub-module is configured to judge whether the target establishment field exists in the first data through a Replication mechanism;
and the obtaining sub-module is configured to read a value of the target establishment field in the first data and a key value of a primary key field in the first data from the pre-written log if the target establishment field exists in the first data.
9. A data query apparatus for a distributed database, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to:
acquiring a query request, wherein the query request comprises at least one query field and a query condition corresponding to each query field;
if the query request has a target query field, querying a target key value of a primary key field matched with a query condition corresponding to the target query field according to a pre-established target index; the target query field is a designated non-primary key field, and the target index is established when data is written into the distributed database and comprises an index of a mapping relation between the designated non-primary key field and the primary key field;
and taking out the data from the distributed database, wherein the primary key field is the target data of the target key value, and the target data is used as the query result corresponding to the query request.
10. A computer-readable storage medium, on which computer program instructions are stored, which program instructions, when executed by a processor, carry out the steps of the method according to any one of claims 1 to 5.
CN202010583096.4A 2020-06-23 2020-06-23 Data query method and device of distributed database and storage medium Pending CN111797134A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010583096.4A CN111797134A (en) 2020-06-23 2020-06-23 Data query method and device of distributed database and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010583096.4A CN111797134A (en) 2020-06-23 2020-06-23 Data query method and device of distributed database and storage medium

Publications (1)

Publication Number Publication Date
CN111797134A true CN111797134A (en) 2020-10-20

Family

ID=72803744

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010583096.4A Pending CN111797134A (en) 2020-06-23 2020-06-23 Data query method and device of distributed database and storage medium

Country Status (1)

Country Link
CN (1) CN111797134A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112364014A (en) * 2020-10-26 2021-02-12 北京达佳互联信息技术有限公司 Data query method, device, server and storage medium
CN112463827A (en) * 2020-11-16 2021-03-09 北京达佳互联信息技术有限公司 Query method and device, electronic equipment and storage medium
CN112905600A (en) * 2021-03-19 2021-06-04 腾讯科技(深圳)有限公司 Data query method and device, storage medium and electronic equipment
CN112988666A (en) * 2021-03-22 2021-06-18 湖南大学 Distributed log condition query method and system based on cuckoo filter
CN112989171A (en) * 2021-03-26 2021-06-18 广东电网有限责任公司 Data query method, device, equipment and medium
CN113723907A (en) * 2021-08-05 2021-11-30 中核武汉核电运行技术股份有限公司 ES quick query method and system based on nuclear power spare part management
CN114168589A (en) * 2021-10-28 2022-03-11 阿里巴巴(中国)有限公司 Index construction method and device

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010048789A1 (en) * 2008-11-03 2010-05-06 中国移动通信集团公司 Index building, querying method, device, and system for distributed column memory database
CN104102710A (en) * 2014-07-15 2014-10-15 浪潮(北京)电子信息产业有限公司 Massive data query method
US20140317093A1 (en) * 2013-04-22 2014-10-23 Salesforce.Com, Inc. Facilitating dynamic creation of multi-column index tables and management of customer queries in an on-demand services environment
WO2017067117A1 (en) * 2015-10-21 2017-04-27 华为技术有限公司 Data query method and device
CN109284351A (en) * 2018-08-14 2019-01-29 青海大学 A kind of data query method based on HBase database
CN109885570A (en) * 2019-01-30 2019-06-14 南京邮电大学 A kind of multi-Dimensional Range querying method of secondary index combination hash table
CN110287198A (en) * 2019-07-01 2019-09-27 四川新网银行股份有限公司 Finance data indexing means based on HBase database

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010048789A1 (en) * 2008-11-03 2010-05-06 中国移动通信集团公司 Index building, querying method, device, and system for distributed column memory database
CN101727465A (en) * 2008-11-03 2010-06-09 中国移动通信集团公司 Methods for establishing and inquiring index of distributed column storage database, device and system thereof
US20140317093A1 (en) * 2013-04-22 2014-10-23 Salesforce.Com, Inc. Facilitating dynamic creation of multi-column index tables and management of customer queries in an on-demand services environment
CN104102710A (en) * 2014-07-15 2014-10-15 浪潮(北京)电子信息产业有限公司 Massive data query method
WO2017067117A1 (en) * 2015-10-21 2017-04-27 华为技术有限公司 Data query method and device
CN109284351A (en) * 2018-08-14 2019-01-29 青海大学 A kind of data query method based on HBase database
CN109885570A (en) * 2019-01-30 2019-06-14 南京邮电大学 A kind of multi-Dimensional Range querying method of secondary index combination hash table
CN110287198A (en) * 2019-07-01 2019-09-27 四川新网银行股份有限公司 Finance data indexing means based on HBase database

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112364014A (en) * 2020-10-26 2021-02-12 北京达佳互联信息技术有限公司 Data query method, device, server and storage medium
CN112364014B (en) * 2020-10-26 2024-01-02 北京达佳互联信息技术有限公司 Data query method, device, server and storage medium
CN112463827A (en) * 2020-11-16 2021-03-09 北京达佳互联信息技术有限公司 Query method and device, electronic equipment and storage medium
CN112463827B (en) * 2020-11-16 2024-03-12 北京达佳互联信息技术有限公司 Query method, query device, electronic equipment and storage medium
CN112905600A (en) * 2021-03-19 2021-06-04 腾讯科技(深圳)有限公司 Data query method and device, storage medium and electronic equipment
CN112905600B (en) * 2021-03-19 2023-09-26 腾讯科技(深圳)有限公司 Data query method and device, storage medium and electronic equipment
CN112988666A (en) * 2021-03-22 2021-06-18 湖南大学 Distributed log condition query method and system based on cuckoo filter
CN112988666B (en) * 2021-03-22 2022-04-22 湖南大学 Distributed log condition query method and system based on cuckoo filter
CN112989171A (en) * 2021-03-26 2021-06-18 广东电网有限责任公司 Data query method, device, equipment and medium
CN112989171B (en) * 2021-03-26 2024-08-30 广东电网有限责任公司 Data query method, device, equipment and medium
CN113723907A (en) * 2021-08-05 2021-11-30 中核武汉核电运行技术股份有限公司 ES quick query method and system based on nuclear power spare part management
CN114168589A (en) * 2021-10-28 2022-03-11 阿里巴巴(中国)有限公司 Index construction method and device

Similar Documents

Publication Publication Date Title
CN111797134A (en) Data query method and device of distributed database and storage medium
US11068441B2 (en) Caseless file lookup in a distributed file system
US8321444B2 (en) Federated search
EP4158497A1 (en) Automatic transformation of data by patterns
US10909086B2 (en) File lookup in a distributed file system
CN108536745B (en) Shell-based data table extraction method, terminal, equipment and storage medium
CN107329987A (en) A kind of search system based on mongo databases
CN109361628B (en) Message assembling method and device, computer equipment and storage medium
CN111984659B (en) Data updating method, device, computer equipment and storage medium
CN106547646B (en) Data backup and recovery method and data backup and recovery device
CN112883030A (en) Data collection method and device, computer equipment and storage medium
CN113220659A (en) Data migration method, system, electronic device and storage medium
CN109726295A (en) Brand knowledge map display methods, device, figure server and storage medium
CN111209061A (en) Method and device for filling in user information, computer equipment and storage medium
CN111858581B (en) Paging query method and device, storage medium and electronic equipment
CN112783866B (en) Data reading method, device, computer equipment and storage medium
CN109697234B (en) Multi-attribute information query method, device, server and medium for entity
CN110737662A (en) data analysis method, device, server and computer storage medium
CN111008198A (en) Service data acquisition method and device, storage medium and electronic equipment
CN110471708B (en) Method and device for acquiring configuration items based on reusable components
US9460139B2 (en) Distributed storage system with pluggable query processing
CN117009430A (en) Data management method, device, storage medium and electronic equipment
CN113094367A (en) Data processing method and device and server
CN110347333A (en) Improve method, apparatus, computer equipment and the storage medium of clone's mirror image performance
CN113626046B (en) Container mirror image updating method, device, storage medium and computer equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination