CN107741937B - Data query method and device - Google Patents

Data query method and device Download PDF

Info

Publication number
CN107741937B
CN107741937B CN201610822514.4A CN201610822514A CN107741937B CN 107741937 B CN107741937 B CN 107741937B CN 201610822514 A CN201610822514 A CN 201610822514A CN 107741937 B CN107741937 B CN 107741937B
Authority
CN
China
Prior art keywords
data
database
query
searching
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610822514.4A
Other languages
Chinese (zh)
Other versions
CN107741937A (en
Inventor
李家昌
何瑞
曾凡
万志颖
阮华
史晓茸
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201610822514.4A priority Critical patent/CN107741937B/en
Publication of CN107741937A publication Critical patent/CN107741937A/en
Application granted granted Critical
Publication of CN107741937B publication Critical patent/CN107741937B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

The invention discloses a data query method and a data query device, wherein the data query method comprises the following steps: receiving a query request sent by a client, wherein the query request comprises a query statement of data to be queried; determining whether the data to be queried is stored in a first database according to the query statement, wherein the first database is a relational database; if so, searching data corresponding to the query statement from the first database, otherwise, searching data corresponding to the query statement from a second database, wherein the second database is a distributed database comprising a plurality of sub-databases; and generating a query result based on the searched data, and sending the query result to the client. The data query method can select a proper data query mode according to the size of the data volume, and is simple and high in query efficiency.

Description

Data query method and device
Technical Field
The invention relates to the technical field of computers, in particular to a data query method and device.
Background
In the internet era with a huge increase in data volume, how to efficiently process data is a problem which is urgently needed to be solved in the technical field of data processing at present. Due to the lack of horizontal scalability of traditional centralized databases (such as relational databases), in the face of massive amounts of data, a relatively common data processing system is a distributed processing system.
At present, a distributed processing system generally uses a uniform flexible distributed data set (RDD) data structure for data processing, linear expansion of storage capacity can be realized by adding storage nodes, and fault tolerance is realized by a backup method, so that compared with a centralized database, the distributed processing system has high scalability, high concurrency and high availability, but because the distributed processing system has the characteristics of delay in executing tasks and discrete distribution of data, when processing a small amount of data, the processing efficiency of the distributed processing system is far lower than that of the centralized database.
Disclosure of Invention
The invention aims to provide a data query method and a data query device, which aim to solve the technical problem that the existing distributed processing system is low in efficiency when small data volume is processed.
In order to solve the above technical problems, embodiments of the present invention provide the following technical solutions:
a method of data query, comprising:
receiving a query request sent by a client, wherein the query request comprises a query statement of data to be queried;
determining whether the data to be queried is stored in a first database according to the query statement, wherein the first database is a relational database;
if so, searching data corresponding to the query statement from the first database;
if not, searching data corresponding to the query statement from a second database, wherein the second database is a distributed database comprising a plurality of sub-databases;
and generating a query result based on the searched data, and sending the query result to the client.
In order to solve the above technical problems, embodiments of the present invention further provide the following technical solutions:
a data querying device, comprising:
the system comprises a receiving module, a query module and a query module, wherein the receiving module is used for receiving a query request sent by a client, and the query request comprises a query statement of data to be queried;
the determining module is used for determining whether the data to be inquired is stored in a first database according to the inquiry statement, wherein the first database is a relational database;
the searching module is used for searching the data corresponding to the query statement from the first database if the query statement is in the first database; if not, searching data corresponding to the query statement from a second database, wherein the second database is a distributed database comprising a plurality of sub-databases;
and the sending module is used for generating a query result based on the searched data and sending the query result to the client.
According to the data query method and device, the query request sent by the client is received, the query request comprises a query statement of data to be queried, whether the data to be queried is stored in the first database is determined according to the query statement, if yes, the data corresponding to the query statement is searched from the first database, if not, the data corresponding to the query statement is searched from the second database, the second database is a distributed database comprising a plurality of sub-databases, then, a query result is generated based on the searched data, and the query result is sent to the client.
Drawings
The technical solution and other advantages of the present invention will become apparent from the following detailed description of specific embodiments of the present invention, which is to be read in connection with the accompanying drawings.
FIG. 1a is a schematic view of a data query system according to an embodiment of the present invention;
FIG. 1b is a schematic flow chart of a data query method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a data query method according to an embodiment of the present invention;
fig. 3a is a schematic structural diagram of a data query apparatus according to an embodiment of the present invention;
FIG. 3b is a schematic structural diagram of a data query apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a server according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention provides a data query method, a data query device and a data query system.
Referring to fig. 1a, the data query system may include any one of the data query devices provided in the embodiments of the present invention, and the data query device may be specifically integrated in a server, and the server may include a web server.
The server can receive a query request sent by a client, the query request comprises a query statement of data to be queried, whether the data to be queried is stored in a first database is determined according to the query statement, the first database is a relational database, if yes, data corresponding to the query statement are searched from the first database, if not, data corresponding to the query statement are searched from a second database, the second database is a distributed database comprising a plurality of sub-databases, then, a query result is generated based on the searched data, and the query result is sent to the client.
The client may include a computer, a mobile phone, or a tablet computer, and may be connected to the server through a network. The query results may be presented in the form of a list, chart, or numerical value. The Query statement may be a Query command written in Structured Query Language (SQL) or other languages. When a client sends a query request to a server, the server determines whether data to be queried is stored in a relational database according to a query statement in the query request, if so, the data to be queried is directly searched from the relational database, if not, the data to be queried is searched from a distributed database, then, the searched data is generated into a query result in a list or chart form, and the query result is sent to the client.
The details will be described below separately. The numbers in the following examples are not intended to limit the order of priority of the examples.
First embodiment
The present embodiment will be described from the perspective of a data query apparatus, which may be integrated in a server.
Referring to fig. 1b, fig. 1b specifically illustrates a data query method according to a first embodiment of the present invention, which may include:
s101, receiving a query request sent by a client, wherein the query request comprises a query statement of data to be queried.
In this embodiment, the query statement may be a query command written through Structured Query Language (SQL) or other languages, and is mainly used for querying data stored in the data table. Each query statement may include a table name, a field, and a key, generally, a "column" in each data table is referred to as a "field", each field contains information of a certain topic, a "row" in the data table is referred to as a "record", each record contains all information in the row, and data at the intersection of the row and the column of the data table is referred to as a "value", the value may include letters, chinese characters, and the like, the key mentioned here is a value, and the data to be queried is a record.
It should be noted that, because the query statement generally has a default fixed format, and the program cannot be read out for query statements that do not conform to the format, it is necessary to detect in advance whether the format of the query statement meets the requirement before performing the relevant query operation according to the query statement.
S102, determining whether the data to be queried is stored in a first database according to the query statement, where the first database is a relational database, if so, performing the following step S103, and if not, performing the following step S104.
In this embodiment, the relational database is a database established on the basis of a relational database model, each data table in the database includes one or more data types represented by columns, and the same data type corresponds to a unique data entity (i.e., value) in each row.
For example, the query statement may include a table name, a target field, and a target keyword corresponding to the target field, where the step S102 may specifically include:
searching a target data table corresponding to the table name from a first database;
searching a keyword set corresponding to the target field from the target data table;
matching the target keyword with the keyword set;
if the matching is successful, determining that the data to be queried is stored in the first database;
and if the matching fails, determining that the data to be queried is not stored in the first database.
In this embodiment, the number of the target fields and the target keywords may be multiple, each target field corresponds to multiple entity data in a certain column in the data table, each entity data is located in a different row, and each row represents one record. Generally, if it is determined whether the relational database stores the designated record, the corresponding entity data are found from the target data table according to the target field, and then the target keyword is matched with the entity data, if the matching is successful, it is indicated that the designated record is stored, and information of a row of the successfully matched entity data is the designated record to be searched, that is, the data to be searched.
In addition, to ensure the security of data, when the user performs data query, the identity of the user may be verified first, and only after the verification is passed, the query of related data may be performed, that is, the query request may further include the identity information of the user, and at this time, before the step S102, the data query method may further include:
determining the authority information of the user according to the identity information;
judging whether the user has the authority to inquire the data to be inquired or not according to the authority information;
and if the user has the authority of inquiring the data to be inquired, executing the operation of determining whether the data to be inquired is stored in the first database according to the inquiry statement.
In this embodiment, the identity information may include information such as a user account and a password, and the authority information mainly refers to authority levels, for example, the authority levels may include no authority, a low-level authority, a medium-level authority, a high-level authority, and the like.
It is easy to understand that, in the actual application process, if the authority information is determined according to the identity information of the user and whether the data to be queried can be queried is determined according to the authority information, the identity information, the authority information, and the authority information need to be associated with the queryable data in advance, so that the relevant lookup operation can be performed according to the association relationship.
S103, searching corresponding data from the first database according to the query statement.
In this embodiment, when it is determined that the data to be queried is stored in the relational database, the data to be queried can be directly searched from the relational database without searching from the distributed database, so that the query efficiency of a small data volume is improved. Specifically, when the target keyword is successfully matched with some entity data in the relational database, information of a row where the entity data is located, that is, the information is the data to be queried, may be directly obtained.
S104, searching corresponding data from a second database according to the query statement, wherein the second database is a distributed database comprising a plurality of sub-databases.
In this embodiment, the distributed database is a logically unified database formed by connecting a plurality of physically distributed database units (i.e., sub-databases) by using a computer network, and is highly efficient in processing mass data.
For example, when the query statement includes a table name, a target field, and a target keyword corresponding to the target field, the step S104 may specifically include:
searching an index table corresponding to the table name from a second database;
searching a pointer set corresponding to the target field from the index table;
and acquiring corresponding data from the plurality of sub-databases according to the searched pointer set and the target keyword.
In this embodiment, the index table may be located in each sub-database, or in a designated sub-database, and different from the data table, information of each row in the index table is a pointer set, information of each row in the data table is a data set (i.e., a record), the data table is located in the sub-databases, and each pointer points to a unique field of one data table, so that, if a designated record needs to be searched from a distributed database, a corresponding index table needs to be found first, a pointer set included in a corresponding column is found from the index table according to a target field, then a column of the data table corresponding to a pointer in the pointer set is found in multiple sub-databases in a parallel manner, and a corresponding row is found according to a target keyword, where information of the row is data to be queried.
It is easy to understand that, in the practical application process, data storage needs to be performed in the relational database and the distributed database in advance, that is, before the step S101, the data query method may further include:
acquiring data to be uploaded;
storing the data to be uploaded in the second database, and acquiring a preset synchronous uploading strategy;
and storing the data to be uploaded in a first database according to the synchronous uploading strategy.
In this embodiment, the synchronous uploading policy may be determined according to actual requirements, for example, may be set by default in the system, or may be set by the user. When the synchronous uploading strategy is set by the user, the user can specify whether to upload the data to the first database while uploading the data.
When the synchronous uploading policy is set by default, the step of "storing the data to be uploaded in the first database according to the synchronous uploading policy" may specifically include:
detecting the data quantity value of the data to be uploaded;
and when the data magnitude value does not exceed a preset threshold value, storing the data to be uploaded in a first database.
In this embodiment, the preset threshold may be determined according to actual application requirements, and is mainly used for defining the size of the data to be uploaded. The data magnitude can be expressed in terms of the size of the bytes occupied by the data, with one Chinese character or Chinese punctuation mark occupying two bytes and one character or English punctuation mark occupying one byte. Because the relational database (i.e. the first database) has higher processing efficiency when processing a small amount of data and has obvious advantages compared with the distributed database (i.e. the second database), the data to be uploaded with a small data amount can be synchronized into the relational database, so that subsequent efficient query operation is facilitated.
And S105, generating a query result based on the searched data, and sending the query result to the client.
In this embodiment, the client may include a computer, a mobile phone, or a tablet computer and other terminals. The expression form of the query result may specifically depend on actual requirements, and may be, for example, expressed in the form of a chart, a list, or a numerical value. The type of chart can be selected in many ways, such as a pie chart, a bar chart, a line chart and the like. After generating the required query result, the server may send the query result to the client via the network, and the client performs landing on a specified data landing path, such as a table, a folder, or a storage disk.
In order to make the query result simple, clear, intuitive and vivid, the query result may be represented in a form of a chart, and at this time, the step of "generating the query result based on the searched data" may specifically include:
acquiring preset chart setting information;
and converting the searched data into a chart according to the chart setting information to obtain a query result.
In this embodiment, the chart setting information may be set by the user, and may be information carried in the query request, or information input by the system after the data is found. The chart setting information may include setting contents such as a title, a scale, a legend, and a body, and is mainly used for defining two parts of a header and a data area of the chart.
As can be seen from the above, in the data query method provided in this embodiment, by receiving a query request sent by a client, where the query request includes a query statement of data to be queried, and determining whether the data to be queried is stored in a first database according to the query statement, where the first database is a relational database, if so, searching data corresponding to the query statement from the first database, and if not, searching data corresponding to the query statement from a second database, where the second database is a distributed database including a plurality of sub-databases, and then generating a query result based on the searched data and sending the query result to the client, a suitable data query manner can be selected according to the size of data volume, which is simple and has high query efficiency.
Second embodiment
The method described in the first embodiment is further illustrated by way of example.
In the present embodiment, the data query apparatus will be described in detail by taking an example in which the data query apparatus is integrated in a server.
As shown in fig. 2, a specific flow of a data query method may be as follows:
s201, the server obtains data to be uploaded and stores the data to be uploaded in a second database, wherein the second database is a distributed database comprising a plurality of sub databases.
For example, the data to be uploaded may be personal information of a student in the class M, such as name, gender, age, home address, contact information, examination scores of each department, etc., and the data may be stored in a plurality of data tables of the sub-databases in a split manner, such as storing the name and the home address, the name and the contact information, and the name and the examination scores of each department in different data tables on the basis of the name, each data table being stored in a different sub-database, and then the distributed database may establish an index table with a table name "table 1" according to the data tables stored in the sub-databases.
S202, the server detects a data quantity value of the data to be uploaded, and when the data quantity value does not exceed a preset threshold value, the data to be uploaded is stored in a first database, wherein the first database is a relational database.
For example, the server may calculate the total number of bytes occupied by the data to be uploaded, and obtain the data quantity value. For example, when the data size value occupies a number of bytes, the preset threshold occupies B bytes, and if a is less than or equal to B, it indicates that the data size value of the data to be uploaded is relatively small, at this time, the server may store all the personal information of the M classes of students in the data table named "table 1" in the first database.
S203, the server receives a query request sent by the client, wherein the query request comprises a query statement of data to be queried and identity information of a user, and the query statement comprises a table name, a target field and a target keyword corresponding to the target field.
For example, the identity information may include an account number and a password of the user, and the query statement may be: the term "SELECT". multidable 1WHERE name is small red, WHERE table1 is a table name, name is a target field, and small red is a keyword in the name field, and the query statement means to query a record of classmate with name of small red from table 1.
S204, the server determines the authority information of the user according to the identity information, and judges whether the user has the authority to query the data to be queried according to the authority information, if so, the following step S205 is executed, and if not, the server can prompt the user not to have the query authority.
For example, if the user account is an account of an M class teacher (non-class master), the authority information is a middle-level authority, and the middle-level authority can search personal information of all students in the M class, that is, can query records of all classmates in the M class, and has query authority. If the user account is an account of a classmate of the M classes, the query right is a low-level right which can only be used for querying the personal information of the classmate, namely the records of all the classmates of the M classes cannot be queried, and the query right is not available.
S205, the server searches a target data table corresponding to the table name from the first database, and searches a keyword set corresponding to the target field from the target data table.
For example, when the user account is an account of a teacher in M shifts, the server may look up a data table with a table name of "table 1" in the first database, and find out all names (i.e., keyword sets) of classmates in M shifts currently stored according to "name", such as xiaohong, xiaoming, and xiaoqin, etc.
S206, the server matches the target keyword with the keyword set, if the matching is successful, the following step S207 is executed, and if the matching is failed, the following step S208 is executed.
For example, if the M classmates currently stored in the first database include small red, the matching is successful, and if the M classmates currently stored in the first database do not include small red, the matching is failed.
S207, the server searches corresponding data from the first database according to the query statement.
For example, the server may directly obtain the information of the line where the "xiaohong" is located in the "table 1" data table, that is, the sex, age, home address, contact address, and achievement obtained.
S208, the server searches corresponding data from the second database according to the query statement.
For example, the server may find the "table 1" index table from the second database and find the set of pointers based on the "name", each pointer pointing to a data table in one of the sub-databases, and then may look up information such as gender, age, home address, contact address, and test score of the xiaohong from the plurality of data tables based on the "xiaohong".
S209, the server converts the searched data into a chart according to preset chart setting information to obtain a query result, and sends the query result to the client.
For example, the user may choose to generate query results in the form of a histogram of the test scores of the subjects in the lower red, and send the query results to the client.
As can be seen from the above, in the data query method provided in this embodiment, the server obtains data to be uploaded, and stores the data to be uploaded in the second database, where the second database is a distributed database including a plurality of sub-databases, detects a data size of the data to be uploaded, and stores the data to be uploaded in the first database when the data size does not exceed a preset threshold, where the first database is a relational database, so as to implement storage of the data in the first database and the second database according to a size of the data size, and then, the server receives a query request sent by the client, where the query request includes a query statement of the data to be queried and identity information of a user, where the query statement includes a table name, a target field, and a target keyword corresponding to the target field, and determines authority information of the user according to the identity information, judging whether the user has the authority to inquire the data to be inquired according to the authority information, if not, prompting the user to have no inquiry authority, if so, obtaining a target data table corresponding to the table name from a first database, searching a keyword set corresponding to the target field from the target data table, then matching the target keyword with the keyword set, and when the matching is successful, searching corresponding data from the first database according to the inquiry statement so as to realize the efficient inquiry operation of small data amount, wherein the operation is simple, when the matching is failed, searching corresponding data from a second database according to the inquiry statement, the second database is a distributed database comprising a plurality of sub-databases, and finally, converting the searched data into a chart according to preset chart setting information to obtain an inquiry result, and the query result is sent to the client, the searched data can be converted into a proper form according to the user requirement and sent to the user, the output type and readability of the data are increased, and the flexibility is high.
Third embodiment
Based on the methods in the first and second embodiments, this embodiment will be further described from the perspective of a processing device of voice information, please refer to fig. 3a, and fig. 3a specifically describes a data query device according to a third embodiment of the present invention, which may include: a receiving module 10, a determining module 20, a searching module 30 and a sending module 40, wherein:
(1) receiving module 10
The receiving module 10 is configured to receive an inquiry request sent by a client, where the inquiry request includes an inquiry statement of data to be inquired.
In this embodiment, the query statement may be a query command written through Structured Query Language (SQL) or other languages, and is mainly used for querying data stored in the data table. Each query statement may include a table name, a field, and a key, generally, a "column" in each data table is referred to as a "field", each field contains information of a certain topic, a "row" in the data table is referred to as a "record", each record contains all information in the row, and data at the intersection of the row and the column of the data table is referred to as a "value", the value may include letters, chinese characters, and the like, the key mentioned here is a value, and the data to be queried is a record.
It should be noted that, because the query statement generally has a default fixed format, and the program cannot be read out for query statements that do not conform to the format, it is necessary to detect in advance whether the format of the query statement meets the requirement before performing the relevant query operation according to the query statement.
(2) Determination module 20
And the determining module 20 is configured to determine whether the data to be queried is stored in a first database according to the query statement, where the first database is a relational database.
In this embodiment, the relational database is a database established on the basis of a relational database model, each data table in the database includes one or more data types represented by columns, and the same data type corresponds to a unique data entity (i.e., value) in each row.
For example, the query statement may include a table name, a target field, and a target keyword corresponding to the target field, and at this time, the determining module 20 may specifically be configured to:
acquiring a target data table corresponding to the table name from a first database;
searching a keyword set corresponding to the target field from the target data table;
matching the target keyword with the keyword set;
if the matching is successful, determining that the data to be queried is stored in the first database;
and if the matching fails, determining that the data to be queried is not stored in the first database.
In this embodiment, the number of the target fields and the target keywords may be multiple, each target field corresponds to multiple entity data in a certain column in the data table, each entity data is located in a different row, and each row represents one record. Generally, if the determining module 20 determines whether the specified record is stored in the relational database, it first finds a plurality of corresponding entity data from the target data table according to the target field, and then performs a matching operation on the target keyword and the plurality of entity data, if the matching is successful, it indicates that the specified record is stored, and information of a row of the successfully matched entity data is the specified record to be searched, that is, the data to be searched.
In addition, in order to ensure the security of data, when the user performs data query, the identity of the user may be verified first, and only after the verification is passed, the query of related data may be performed, that is, the query request may further include the identity information of the user, and the data query apparatus may further include a determining module 60, which may be configured to:
before the determining module 20 determines whether the data to be queried is stored in the first database according to the query statement, determining the authority information of the user according to the identity information;
judging whether the user has the authority to inquire the data to be inquired or not according to the authority information;
and if the user has the authority of inquiring the data to be inquired, executing the operation of determining whether the data to be inquired is stored in the first database according to the inquiry statement.
In this embodiment, the identity information may include information such as a user account and a password, and the authority information mainly refers to authority levels, for example, the authority levels may include no authority, a low-level authority, a medium-level authority, a high-level authority, and the like.
It is easy to understand that, in the actual application process, if the authority information is determined according to the identity information of the user and whether the data to be queried can be queried is determined according to the authority information, the identity information, the authority information, and the authority information need to be associated with the queryable data in advance, so that the relevant lookup operation can be performed according to the association relationship.
(3) Lookup module 30
A searching module 30, configured to search, if yes, data corresponding to the query statement from the first database; if not, searching data corresponding to the query statement from a second database, wherein the second database is a distributed database comprising a plurality of sub-databases.
In this embodiment, the distributed database is a logically unified database formed by connecting a plurality of physically distributed database units (i.e., sub-databases) by using a computer network, and is highly efficient in processing mass data.
When the determining module 20 determines that the data to be queried is stored in the relational database, the searching module 30 may directly search from the relational database without searching from a distributed database, thereby improving the query efficiency with small data volume. Specifically, when the target keyword is successfully matched with some entity data in the relational database, information of a row where the entity data is located, that is, the information is the data to be queried, may be directly obtained.
When the determining module 20 determines that the data to be queried is not stored in the relational database, the searching module 30 may perform the following operations:
searching an index table corresponding to the table name from a second database;
searching a pointer set corresponding to the target field from the index table;
and acquiring corresponding data from the plurality of sub-databases according to the searched pointer set and the target keyword.
In this embodiment, the index table may be located in each sub-database, or in a designated sub-database, and different from the data table, the information of each row in the index table is a pointer set, the information of each row in the data table is a data set (i.e., a record), the data table is located in the sub-databases, and each pointer points to a unique field of one data table, so that, if the lookup module 30 needs to lookup a designated record from the distributed database, it needs to first find a corresponding index table, find a pointer set included in a corresponding column from the index table according to a target field, then find a column of the data table corresponding to a pointer in the pointer set in multiple sub-databases in a parallel manner, and find a corresponding row according to a target keyword, where the information of the row is the data to be queried.
It is easy to understand that, in the practical application process, data storage needs to be performed in the relational database and the distributed database in advance, that is, the data query apparatus may further include an upload module 50, and the upload module 50 may include:
an obtaining sub-module, configured to obtain data to be uploaded before the receiving module 10 receives a query request sent by a client;
the first storage submodule is used for storing the data to be uploaded in the second database and acquiring a preset synchronous uploading strategy;
and the second storage submodule is used for storing the data to be uploaded in the first database according to the synchronous uploading strategy.
In this embodiment, the synchronous uploading policy may be determined according to actual requirements, for example, may be set by default in the system, or may be set by the user. When the synchronous uploading strategy is set by the user, the user can specify whether to upload the data to the first database while uploading the data.
When the synchronous uploading policy is set by default, the second storage submodule may be specifically configured to:
detecting the data quantity value of the data to be uploaded;
and when the data magnitude value does not exceed a preset threshold value, storing the data to be uploaded in a first database.
In this embodiment, the preset threshold may be determined according to actual application requirements, and is mainly used for defining the size of the data to be uploaded. The data magnitude can be expressed in terms of the size of the bytes occupied by the data, with one Chinese character or Chinese punctuation mark occupying two bytes and one character or English punctuation mark occupying one byte. Because the relational database (i.e. the first database) has higher processing efficiency when processing a small amount of data and has obvious advantages compared with the distributed database (i.e. the second database), the data to be uploaded with a small data amount can be synchronized into the relational database, so that subsequent efficient query operation is facilitated.
(4) Sending module 40
And the sending module 40 is configured to generate a query result based on the found data, and send the query result to the client.
In this embodiment, the client may include a computer, a mobile phone, or a tablet computer and other terminals. The expression form of the query result may specifically depend on actual requirements, and may be, for example, expressed in the form of a chart, a list, or a numerical value. The type of chart can be selected in many ways, such as a pie chart, a bar chart, a line chart and the like. After generating the required query result, the sending module 40 may send the query result to the client via the network, and the client falls to a specified data fall path, such as a table, a folder, or a storage disk.
In order to make the query result simple, clear, intuitive and vivid, it may be represented in the form of a chart, and in this case, the sending module 40 may specifically be configured to:
acquiring preset chart setting information;
and converting the searched data into a chart according to the chart setting information to obtain a query result.
In this embodiment, the chart setting information may be set by the user, and may be information carried in the query request, or information input by the system after the data is found. The chart setting information may include setting contents such as a title, a scale, a legend, and a body, and is mainly used for defining two parts of a header and a data area of the chart.
In a specific implementation, the above units may be implemented as independent entities, or may be combined arbitrarily to be implemented as the same or several entities, and the specific implementation of the above units may refer to the foregoing method embodiments, which are not described herein again.
As can be seen from the above, the data query apparatus provided in this embodiment receives the query request sent by the client through the receiving module 10, the query request includes a query statement of the data to be queried, the determining module 20 determines whether the data to be queried is stored in the first database according to the query statement, the first database is a relational database, if yes, the searching module 30 searches the data corresponding to the query sentence from the first database, if no, the searching module 30 searches the data corresponding to the query sentence from the second database, the second database is a distributed database comprising a plurality of sub-databases, after which the sending module 40 generates a query result based on the found data, and the query result is sent to the client, so that a proper data query mode can be selected according to the size of the data volume, the method is simple, and the query efficiency is high.
Fourth embodiment
Correspondingly, the embodiment of the present invention further provides a data query system, including any one of the data query devices provided in the embodiments of the present invention, and the data query device may specifically refer to the third embodiment.
The data query device may be specifically integrated in the server, and for example, may be as follows:
the server is used for receiving a query request sent by the client, wherein the query request comprises a query statement of data to be queried; determining whether the data to be queried is stored in a first database according to the query statement, wherein the first database is a relational database; if so, searching corresponding data from the first database according to the query statement, otherwise, searching corresponding data from a second database according to the query statement, wherein the second database is a distributed database comprising a plurality of sub-databases; and generating a query result based on the searched data, and sending the query result to the client.
In addition, the data query system can further comprise a client, and the client can be used for sending the query request to the server and receiving the query result sent by the server.
The specific implementation of each device can be referred to the previous embodiment, and is not described herein again.
Since the data query system may include any data query device provided in the embodiment of the present invention, the beneficial effects that can be achieved by any data query device provided in the embodiment of the present invention can be achieved, which are detailed in the foregoing embodiments and will not be described herein again.
Fifth embodiment
Correspondingly, an embodiment of the present invention further provides a server, as shown in fig. 4, which shows a schematic structural diagram of the server according to the embodiment of the present invention, specifically:
the server may include components such as a processor 701 of one or more processing cores, memory 702 of one or more computer-readable storage media, Radio Frequency (RF) circuitry 703, a power supply 704, an input unit 705, and a display unit 706. Those skilled in the art will appreciate that the server architecture shown in FIG. 4 is not meant to be limiting, and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components. Wherein:
the processor 701 is a control center of the server, connects various parts of the entire server using various interfaces and lines, and performs various functions of the server and processes data by running or executing software programs and/or modules stored in the memory 702 and calling data stored in the memory 702, thereby performing overall monitoring of the server. Optionally, processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 701.
The memory 702 may be used to store software programs and modules, and the processor 701 executes various functional applications and data processing by operating the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to the use of the server, and the like. Further, the memory 702 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.
The RF circuit 703 may be used for receiving and transmitting signals during the process of transmitting and receiving information, and in particular, for processing the received downlink information of the base station by the one or more processors 701; in addition, data relating to uplink is transmitted to the base station. In general, the RF circuitry 703 includes, but is not limited to, an antenna, at least one Amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, the RF circuitry 703 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to Global System for mobile communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Message Service (SMS), and the like.
The server further includes a power supply 704 (e.g., a battery) for supplying power to the various components, and preferably, the power supply 704 is logically connected to the processor 701 via a power management system, so that functions of managing charging, discharging, and power consumption are implemented via the power management system. The power supply 704 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.
The server may further include an input unit 705, and the input unit 705 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. Specifically, in one particular embodiment, input unit 705 may include a touch-sensitive surface as well as other input devices. The touch-sensitive surface, also referred to as a touch display screen or a touch pad, may collect touch operations by a user (e.g., operations by a user on or near the touch-sensitive surface using a finger, a stylus, or any other suitable object or attachment) thereon or nearby, and drive the corresponding connection device according to a predetermined program. Alternatively, the touch sensitive surface may comprise two parts, a touch detection means and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 701, and can receive and execute commands sent by the processor 701. In addition, touch sensitive surfaces may be implemented using various types of resistive, capacitive, infrared, and surface acoustic waves. The input unit 705 may include other input devices in addition to a touch-sensitive surface. In particular, other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.
The server may also include a display unit 706, which display unit 706 may be used to display information input by or provided to the user, as well as various graphical user interfaces of the server, which may be made up of graphics, text, icons, video, and any combination thereof. The Display unit 706 may include a Display panel, and optionally, the Display panel may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-emitting diode (OLED), or the like. Further, the touch-sensitive surface may cover the display panel, and when a touch operation is detected on or near the touch-sensitive surface, the touch operation is transmitted to the processor 701 to determine the type of the touch event, and then the processor 701 provides a corresponding visual output on the display panel according to the type of the touch event. Although in FIG. 4 the touch-sensitive surface and the display panel are shown as two separate components to implement input and output functions, in some embodiments the touch-sensitive surface may be integrated with the display panel to implement input and output functions.
Although not shown, the server may further include a camera, a bluetooth module, etc., which will not be described herein. Specifically, in this embodiment, the processor 701 in the server loads the executable file corresponding to the process of one or more application programs into the memory 702 according to the following instructions, and the processor 701 runs the application program stored in the memory 702, thereby implementing various functions as follows:
receiving a query request sent by a client, wherein the query request comprises a query statement of data to be queried;
determining whether the data to be queried is stored in a first database according to the query statement, wherein the first database is a relational database;
if yes, searching corresponding data from the first database according to the query statement;
if not, searching corresponding data from a second database according to the query statement, wherein the second database is a distributed database comprising a plurality of sub-databases;
and generating a query result based on the searched data, and sending the query result to the client.
The implementation method of the above operations may specifically refer to the above embodiments, and details are not described herein.
The terminal can achieve the effective effect that can be achieved by any data query device provided by the embodiment of the invention, which is detailed in the previous embodiment and not described herein again.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, and the storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.
The data query method, device and system provided by the embodiment of the present invention are described in detail above, and the principle and the implementation of the present invention are explained in this document by applying specific examples, and the description of the above embodiments is only used to help understanding the method and the core idea of the present invention; meanwhile, for those skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (9)

1. A method for querying data, comprising:
receiving a query request sent by a client, wherein the query request comprises a query statement of data to be queried;
the query statement comprises a table name, a target field and a target keyword corresponding to the target field;
searching a target data table corresponding to the table name from a first database, wherein the first database is a relational database;
searching a keyword set corresponding to the target field from the target data table;
matching the target keyword with the keyword set;
if the matching is successful, determining that the data to be queried is stored in the first database; searching data corresponding to the query statement from the first database;
if the matching fails, determining that the data to be queried is not stored in the first database; searching an index table corresponding to the table name from a second database; the second database is a distributed database comprising a plurality of sub databases; searching a pointer set corresponding to the target field from the index table; acquiring corresponding data from the plurality of sub-databases according to the searched pointer set and the target keyword;
and generating a query result based on the searched data, and sending the query result to the client.
2. The data query method of claim 1, wherein the generating query results based on the located data comprises:
acquiring preset chart setting information;
and converting the searched data into a chart according to the chart setting information to obtain a query result.
3. The data query method of claim 1, wherein before receiving the query request sent by the client, the data query method further comprises:
acquiring data to be uploaded;
storing the data to be uploaded in the second database, and acquiring a preset synchronous uploading strategy;
and storing the data to be uploaded in a first database according to the synchronous uploading strategy.
4. The data query method according to claim 3, wherein the storing the data to be uploaded in a first database according to the synchronous upload policy comprises:
detecting a data quantity value of the data to be uploaded;
and when the data magnitude value does not exceed a preset threshold value, storing the data to be uploaded in a first database.
5. A data query apparatus, comprising:
the system comprises a receiving module, a query module and a query module, wherein the receiving module is used for receiving a query request sent by a client, and the query request comprises a query statement of data to be queried; the query statement comprises a table name, a target field and a target keyword corresponding to the target field;
a determination module to:
searching a target data table corresponding to the table name from a first database; the first database is a relational database;
searching a keyword set corresponding to the target field from the target data table;
matching the target keyword with the keyword set;
if the matching is successful, determining that the data to be queried is stored in the first database;
if the matching fails, determining that the data to be queried is not stored in the first database;
the searching module is used for searching data corresponding to the query statement from the first database if the matching is successful; if the matching fails, searching an index table corresponding to the table name from a second database; the second database is a distributed database comprising a plurality of sub databases; searching a pointer set corresponding to the target field from the index table; acquiring corresponding data from the plurality of sub-databases according to the searched pointer set and the target keyword;
and the sending module is used for generating a query result based on the searched data and sending the query result to the client.
6. The data query device of claim 5, wherein the sending module is specifically configured to:
acquiring preset chart setting information;
and converting the searched data into a chart according to the chart setting information to obtain a query result.
7. The data query device of claim 5, further comprising an upload module, the upload module comprising:
the acquisition sub-module is used for acquiring data to be uploaded before the receiving module receives the query request sent by the client;
the first storage submodule is used for storing the data to be uploaded in the second database and acquiring a preset synchronous uploading strategy;
and the second storage submodule is used for storing the data to be uploaded in the first database according to the synchronous uploading strategy.
8. The data query device of claim 7, wherein the second storage submodule is specifically configured to:
detecting a data quantity value of the data to be uploaded;
and when the data magnitude value does not exceed a preset threshold value, storing the data to be uploaded in a first database.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a program for instructing associated hardware to perform the method of any one of claims 1 to 4.
CN201610822514.4A 2016-09-13 2016-09-13 Data query method and device Active CN107741937B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610822514.4A CN107741937B (en) 2016-09-13 2016-09-13 Data query method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610822514.4A CN107741937B (en) 2016-09-13 2016-09-13 Data query method and device

Publications (2)

Publication Number Publication Date
CN107741937A CN107741937A (en) 2018-02-27
CN107741937B true CN107741937B (en) 2020-04-28

Family

ID=61235113

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610822514.4A Active CN107741937B (en) 2016-09-13 2016-09-13 Data query method and device

Country Status (1)

Country Link
CN (1) CN107741937B (en)

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107679064A (en) * 2017-07-31 2018-02-09 平安科技(深圳)有限公司 Data query method, apparatus and computer-readable recording medium
CN108664613B (en) * 2018-05-11 2022-04-01 中国平安人寿保险股份有限公司 Data query method and device, computer equipment and storage medium
CN109271358A (en) * 2018-11-15 2019-01-25 深圳乐信软件技术有限公司 Data summarization method, querying method, device, equipment and storage medium
CN111400342A (en) * 2019-01-03 2020-07-10 百度在线网络技术(北京)有限公司 Database updating method, device, equipment and storage medium
CN110515953A (en) * 2019-08-29 2019-11-29 百度在线网络技术(北京)有限公司 Querying method, device, equipment and the storage medium of data
CN111190898B (en) * 2019-11-25 2023-07-14 泰康保险集团股份有限公司 Data processing method and device, electronic equipment and storage medium
CN111049887A (en) * 2019-11-29 2020-04-21 天脉聚源(杭州)传媒科技有限公司 Download control method, system and storage medium based on dynamic search strategy
CN111324579A (en) * 2020-03-10 2020-06-23 畅捷通信息技术股份有限公司 ERP file query method, system, terminal and storage medium
CN112231317B (en) * 2020-09-25 2023-05-09 浙江三维通信科技有限公司 Data query method, device, electronic device and storage medium
CN112328704B (en) * 2020-11-03 2024-02-23 成都中科大旗软件股份有限公司 Method, system, computer device and storage medium for implementing multiple data source joint inquiry
CN112506965A (en) * 2020-12-03 2021-03-16 北京天融信网络安全技术有限公司 Database management device and method
CN112732756B (en) * 2020-12-30 2023-08-18 北京奇艺世纪科技有限公司 Data query method, device, equipment and storage medium
CN113239054A (en) * 2021-05-11 2021-08-10 北京百度网讯科技有限公司 Information generation method, related device and computer program product
CN113255007B (en) * 2021-06-25 2021-09-21 蓝象智联(杭州)科技有限公司 Safe hidden three-factor query method
CN113704290A (en) * 2021-09-02 2021-11-26 宏桥高科技集团有限公司 Data query system and method
CN113836405B (en) * 2021-09-09 2024-03-12 深圳Tcl新技术有限公司 Information query method, device and computer readable storage medium
CN114510534B (en) * 2022-01-28 2022-09-06 广东航宇卫星科技有限公司 Data synchronization method, device, equipment and storage medium

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104008191A (en) * 2014-06-12 2014-08-27 北京京东尚科信息技术有限公司 Data inquiring method

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10366056B2 (en) * 2013-11-15 2019-07-30 Salesforce.Com, Inc. Asynchronous search for big objects

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104008191A (en) * 2014-06-12 2014-08-27 北京京东尚科信息技术有限公司 Data inquiring method

Also Published As

Publication number Publication date
CN107741937A (en) 2018-02-27

Similar Documents

Publication Publication Date Title
CN107741937B (en) Data query method and device
US11755835B2 (en) Method and apparatus for displaying candidate word, and graphical user interface
US10455014B2 (en) Method and apparatus for synchronizing content
CN104852885B (en) Method, device and system for verifying verification code
US20150154149A1 (en) Typesetting method, device and system
US10462243B2 (en) Method and device for interaction between terminals
JP2015509625A (en) Method and apparatus for text retrieval on a touch terminal
US10204164B2 (en) Systems and methods for filtering microblogs
CN113420051A (en) Data query method and device, electronic equipment and storage medium
CN106502833B (en) Data backup method and device
CN109241031B (en) Model generation method, model using method, device, system and storage medium
CN110825863A (en) Text pair fusion method and device
WO2015085962A1 (en) Method, device and system for processing account information of web application in joint operation
US20220284060A1 (en) Question Answering Method and Apparatus Based on Knowledge Graph
CN108234275B (en) Method and device for releasing communication information
CN115065609A (en) B-end SaaS system hybrid deployment method and system
CN114840565A (en) Sampling query method, device, electronic equipment and computer readable storage medium
CN111274463B (en) Information display method, device and storage medium based on IM contact person grouping setting
CN111190897B (en) Information processing method, information processing apparatus, storage medium, and server
CN106657281B (en) File sharing method and device
CN104809121B (en) Method and device for controlling display of browser webpage window
CN112764603A (en) Message display method and device and electronic equipment
CN109803173B (en) Video transcoding method and device and storage device
CN111242635A (en) After-sale data maintenance method and system
CN114710501B (en) Service request processing method, device, server and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant