CN113220710A - Data query method and device, electronic equipment and storage medium - Google Patents

Data query method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN113220710A
CN113220710A CN202110514766.1A CN202110514766A CN113220710A CN 113220710 A CN113220710 A CN 113220710A CN 202110514766 A CN202110514766 A CN 202110514766A CN 113220710 A CN113220710 A CN 113220710A
Authority
CN
China
Prior art keywords
database
query
query statement
data
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110514766.1A
Other languages
Chinese (zh)
Inventor
张笑
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202110514766.1A priority Critical patent/CN113220710A/en
Publication of CN113220710A publication Critical patent/CN113220710A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/242Query formulation
    • G06F16/2433Query languages
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution

Abstract

The disclosure provides a data query method, a data query device, electronic equipment and a storage medium, and relates to the technical field of data processing, in particular to the field of intelligent search. The specific implementation scheme is as follows: acquiring initial query information for querying target data; analyzing the initial query information to obtain a database query statement suitable for querying target data, wherein the database query statement is a first query statement or a second query statement; under the condition that the database query statement is a first query statement, querying target data from a first database according to the first query statement; and under the condition that the database query statement is the second query statement, querying the target data from the second database according to the second query statement.

Description

Data query method and device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of data processing technology, and more particularly, to the field of intelligent search.
Background
Databases, also known as data management systems, are collections of data that are processed and stored together in a manner that allows multiple users to share the data and minimize redundancy. One database can be composed of a plurality of data table spaces, and users can perform operations of adding, intercepting, updating, deleting and the like on data in files.
Disclosure of Invention
The present disclosure provides a data query method, apparatus, electronic device, storage medium, and computer program product.
According to an aspect of the present disclosure, there is provided a data query method including: acquiring initial query information for querying target data; analyzing the initial query information to obtain a database query statement suitable for querying the target data, wherein the database query statement is a first query statement or a second query statement; under the condition that the database query statement is the first query statement, querying the target data from a first database according to the first query statement; and under the condition that the database query statement is a second query statement, querying the target data from a second database according to the second query statement.
According to another aspect of the present disclosure, there is provided a data query apparatus including: the first acquisition module is used for acquiring initial query information for querying target data; the analysis module is used for analyzing the initial query information to obtain a database query statement suitable for querying the target data, wherein the database query statement is a first query statement or a second query statement; the first query module is used for querying the target data from a first database according to the first query statement under the condition that the database query statement is the first query statement; and the second query module is used for querying the target data from a second database according to the second query statement under the condition that the database query statement is the second query statement.
According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method as described above.
According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method as described above.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 schematically illustrates an exemplary system architecture to which the data query method and apparatus may be applied, according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a flow diagram of a data query method according to an embodiment of the disclosure;
FIG. 3 schematically illustrates another flow diagram of a data query method according to an embodiment of the disclosure;
FIG. 4 schematically illustrates an example flow diagram of a data query method in accordance with an embodiment of this disclosure;
FIG. 5 schematically illustrates a data maintenance flow diagram for a data maintenance service according to an embodiment of the disclosure;
FIG. 6 schematically illustrates an architecture diagram of a data query system according to an embodiment of the present disclosure;
FIG. 7 schematically shows a block diagram of a data querying device according to an embodiment of the present disclosure; and
FIG. 8 shows a schematic block diagram of an example electronic device that may be used to implement embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
With the development of science and technology and the popularization of internet information, artificial intelligence, big data and the like, more and more data interaction is completed on line. Online data interaction requires a huge pool of good quality resources. In order to satisfy the multi-semantic query of the data resource, a large number of fields are required to be established for the data to store information of each dimension of the data, so each piece of data usually has more than 100 attributes.
The graph database is a database for storing and querying data in a data structure such as a graph, the minimum unit is each piece of data, and the data have an edge maintenance relationship with each other. Graph databases differ from other databases in that relationships dominate graph databases, which maintain overall data resources through maintenance of relationships among data.
An ES (elastic search, an open source highly extended distributed full-text search engine) database maintains a plurality of pieces of data by Inverted indexes (inversed indexes). Wherein, one index is used as a database, the minimum unit is each piece of data, and each piece of data is independent from each other. The inverted index is a table in which each entry includes an attribute value and an address of each record having the attribute value, and the address of a record is determined by the attribute value, rather than the attribute value, and is referred to as an inverted index.
The inventor finds that storing data by using a graph database is inconvenient for maintenance because data can only be searched through graph traversal, and does not support inverted query, and the query performance is obviously reduced when the data volume is large. In addition, using graph databases requires learning specialized graph query languages, resulting in excessive learning and usage costs for developers.
In addition, although the ES storage is used to support inverted query, which is flexible, because the data stored in the ES storage are independent of each other, when some relationships need to be queried, two or even more queries need to be performed, and for an upstream caller of a service, a query request is likely to fail due to an excessively long query time.
Fig. 1 schematically illustrates an exemplary system architecture to which the data query method and apparatus may be applied, according to an embodiment of the present disclosure.
It should be noted that fig. 1 is only an example of a system architecture to which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, and does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios.
As shown in fig. 1, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired and/or wireless communication links, and so forth.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have installed thereon various communication client applications, such as a knowledge reading application, a web browser application, a search application, an instant messaging tool, a mailbox client, and/or social platform software, etc. (by way of example only).
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (for example only) providing support for content browsed by the user using the terminal devices 101, 102, 103. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device. The Server may also be a cloud Server, which is also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service extensibility in a traditional physical host and a VPS service ("Virtual Private Server", or "VPS" for short). The server can also be a server of a distributed system or a server combined with a block chain
It should be noted that the data query method provided by the embodiment of the present disclosure may be generally executed by the terminal device 101, 102, or 103. Correspondingly, the data query device provided by the embodiment of the present disclosure may also be disposed in the terminal device 101, 102, or 103.
Alternatively, the data query method provided by the embodiment of the present disclosure may also be generally executed by the server 105. Accordingly, the data query device provided by the embodiment of the present disclosure may be generally disposed in the server 105. The data query method provided by the embodiment of the present disclosure may also be executed by a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the data query apparatus provided in the embodiments of the present disclosure may also be disposed in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.
For example, when a user needs to query target data, the user may input initial query information for querying the target data in a search box of the terminal device 101, 102, 103. The terminal devices 101, 102, and 103 may obtain the initial query information, then send the obtained initial query information to the server 105, and the server 105 parses the initial query information to obtain a database query statement for querying the target data, so as to query the target data from a corresponding database according to the database query statement. Or the server or server cluster capable of communicating with the terminal devices 101, 102, 103 and/or the server 105 parses the initial query information and queries the target data according to the parsed database query statement.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Fig. 2 schematically shows a flow chart of a data query method according to an embodiment of the present disclosure.
As shown in fig. 2, the method includes operations S210 to S240.
In operation S210, initial query information for querying target data is acquired.
In operation S220, the initial query information is parsed to obtain a database query statement applicable to query the target data. The database query statement is a first query statement or a second query statement.
In operation S230, in case that the database query statement is the first query statement, the target data is queried from the first database according to the first query statement.
In operation S240, in case that the database query statement is the second query statement, the target data is queried from the second database according to the second query statement.
According to an embodiment of the present disclosure, the target data may be data that is stored in a background database and is desired to be obtained by a user, the background database may include the first database and the second database, and the first database and the second database may include any two of various relational databases and non-relational databases. For example, the first and second databases may include any two of MySQL (a small open source relational database management system), SQL Server (a relational database management system), Oracle (a currently popular relational database management system), DB2 (a relational database platform), Redis (a key-value store database), HBase (a distributed, column-oriented open source database), MongoDB (a document-oriented open source database), ES, Neo4j (graph database), and the like.
According to the embodiment of the present disclosure, the background database may not be limited to two kinds of databases, and in practical applications, the background database may also include a combination of more kinds of databases. Meanwhile, in order to ensure the integrity of data storage and the validity of data query, the background database may be, for example, a combination of a relational database and a non-relational database, and may also be, for example, a combination of a graph database and a non-graph database.
According to an embodiment of the present disclosure, the initial query information is, for example, information entered into a search box by a user for querying target data.
According to an embodiment of the present disclosure, a database query statement is, for example, a statement used by a search engine in performing a data query in response to initial query information. The statement may be, for example, a programmed statement, including a structured query statement, a graph query statement, and so on. For example, where the first database and the second database are a combination of a graph database and a non-graph database, the first query statement and the second query statement may be a graph query statement and a structured query statement.
According to the embodiment of the present disclosure, in the case where the background databases are the first database and the second database, the database query statement applicable to the query target data may be determined from both the first query statement and the second query statement. In the case where the background database includes a plurality of databases, a database query statement applicable to query target data may be determined from query statements used to query the plurality of databases. The determined database query statement may be the query statement with the fastest query efficiency in all selectable query modes.
It should be noted that the above embodiments are only exemplary embodiments, but are not limited thereto, and may also include other data query methods known in the art as long as the data query can be implemented.
According to the embodiment of the disclosure, the database query statement suitable for querying the target data can be obtained by analyzing the initial query information, and the database query statement suitable for querying the target data can be utilized to query data from the corresponding database, so that the technical problem that a single-type data storage mode is low in query efficiency when data query is needed can be at least partially solved, and the data query efficiency is further improved.
The data query method shown in fig. 2 is further described below with reference to fig. 3 to 6 in conjunction with specific embodiments.
According to the embodiment of the disclosure, in view of the advantage that the graph database has higher performance when performing the relational query and the advantage that the ES storage supports the inverted index, for example, a data query system can be constructed by combining the graph database and the ES storage. For example, the first database may be a database of databases, and the second database may be a distributed search engine database, such as an ES database, or the like.
By the embodiment of the disclosure, the advantages of the database and the ES database can be integrated, and the data query efficiency is improved.
According to an embodiment of the present disclosure, the initial query information includes textual information. Analyzing the initial query information to obtain a database query statement suitable for querying target data, comprising: and performing word segmentation processing on the text information to obtain at least one keyword for representing the text information. And acquiring a target query statement matched with the at least one keyword. And taking the target query statement as a database query statement.
According to the embodiment of the disclosure, taking the text information as "the era of the author of the calm wind wave" as an example, the at least one keyword obtained by performing word segmentation processing on the text information may be "the calm wind wave", "the author", or "the era". The target query statement may be a query statement matching all three keywords of "fixed wind wave", "author", and "dynasty". For example, the target query statement may be a query statement that contains information of three variables, namely "poetry", "author" and "dynasty".
By the embodiment of the disclosure, the database query statement is determined according to the keywords obtained by segmenting the text information, and a more appropriate query statement can be selected to realize the query of the target data, which is beneficial to the improvement of the query performance.
According to an embodiment of the present disclosure, the at least one keyword may include N keywords, where N is an integer greater than 1. Obtaining the target query statement that matches the at least one keyword may include: and under the condition that the full-matching query statement matched with the N keywords is obtained, taking the full-matching query statement as a target query statement. Under the condition that the query sentences matched with the N keywords are not obtained: selecting i target keywords from the N keywords, and acquiring query sentences matched with the i target keywords as target query sentences, wherein i is more than or equal to 1 and less than N.
According to the embodiment of the disclosure, taking at least one keyword as "fixed wind wave", "author" and "dynasty" as an example, it can be determined that the value of N at this time is 3, and the fully-matched query statement may be the above-mentioned query statement simultaneously containing three variable information of "poetry", "author" and "dynasty". In the event that the three keywords fail to match to the full match query statement, two keywords may be selected from the three keywords to match again. For example, two words "calm wind wave" and "author" may be selected as the target keyword, and the full-match query statement at this time may be a query statement containing two variable information "poetry" and "author". Similarly, in the event that a full match query statement matching any two target keywords cannot be obtained, the number of keywords may continue to be reduced. For example, "calm wind" may be selected as a target keyword, and a query sentence including variable information of "poetry" may be acquired as a target query sentence.
Through the embodiment of the disclosure, in the process of determining the query statement based on the keyword, the target query statement can be determined by reducing the number of the keyword, so that the most appropriate database query statement can be effectively determined, and effective and reasonable guarantee is provided for the query process of the target data.
FIG. 3 schematically shows another flow diagram of a data query method according to an embodiment of the disclosure.
As shown in fig. 3, the method includes operations S310 to S370.
In operation S310, initial query information for querying target data is acquired.
In operation S320, the initial query information is parsed to obtain a database query statement applicable to query the target data. The database query statement is a first query statement or a second query statement.
In operation S330, a format of a database query statement is acquired.
In operation S340, in case the format matches the format of the query syntax of the first database, the database query statement is determined to be the first query statement.
In operation S350, target data is queried from a first database according to a first query statement.
In operation S360, in case the format matches the format of the query syntax of the second database, the database query statement is determined to be the second query statement.
In operation S370, target data is queried from the second database according to the second query statement.
According to the embodiment of the disclosure, when data is queried from different databases, quick query needs to be realized by combining with the adaptive query statement, so that after the database query statement is obtained, the type of the database needing to be queried can be determined by judging the syntactic format of the database query statement. For example, if the database query statement is a structured query statement and the syntax format is JSON (a lightweight data exchange format), it may be determined that the database to be queried is, for example, an ES database. For another example, if the database query statement is a graph query statement and the syntax format is gremlin (a graph query language), it can be determined that the database to be queried is a graph database.
It should be noted that, other technical features in this embodiment have been described in the foregoing embodiment, and are not described herein again.
According to the embodiment of the disclosure, as the initial query information for querying the target data is aimed at, the query mode most suitable for querying the target data can be automatically adapted in the query process, so that the query performance can be effectively improved while the intelligent query is realized.
According to an embodiment of the present disclosure, in a case where the database query statement is a first query statement, querying the target data from the first database according to the first query statement includes: and obtaining a first query result with a first data format by querying the first database according to the first query statement. A target data format for presenting the target data is determined. The first query result is converted into a target query result having the target data format. And displaying the target data based on the target query result.
According to the embodiment of the disclosure, because the query results obtained by querying from different databases have different formats, in order to ensure that the different query results are transparent to the upstream data format, the query results can be converted into results with a uniform target data format. For example, taking the first database as the graph database as an example, the data format of the first query result obtained by the query may be a character string format. After the first query result in the character string format is obtained, the first query result in the character string format may be converted into a result in the target data format, and the target data may be displayed based on the result in the target data format.
By the embodiment of the disclosure, after the query result is obtained, the upstream data format is transparent by unifying the data format, so that the work of developers in upstream design can be effectively reduced, and the query efficiency is improved.
According to an embodiment of the present disclosure, determining a target data format for presenting target data includes: a second data format is determined, the second data format corresponding to a second query result from querying a second database according to a second query statement. And taking the second data format as a target data format.
According to the embodiment of the disclosure, since the ES database can support storage structures such as character strings, key-values, objects, JSON, and the like, the graph database only supports storage structures such as character strings, arrays, and the like, and the graph database does not support the JSON storage structures. Taking the first database as a database and the second database as an ES database as an example, in order to ensure that the query result obtained by querying from the ES database and the database is transparent to the upstream data format, the query result in the string format obtained by querying from the database can be converted into a result in the JSON format. Meanwhile, since the ES database supports the JSON structure, in order to simplify the design process, for example, the format of the JSON structure may be used as a target data format for displaying target data.
Through the embodiment of the disclosure, the design process can be further simplified, and meanwhile, the effective display of the query result is ensured.
FIG. 4 schematically shows an example flow diagram of a data query method according to an embodiment of the disclosure.
As shown in fig. 4, the method includes operations S410 to S490.
In operation S410, initial query information for querying target data is acquired.
In operation S420, a Query parsing module is invoked to Query and parse the initial Query information. Where Query represents a message sent by a search engine or database to find a particular file, web site, record or series of records in the database.
In operation S430, it is determined whether a database Query statement corresponding to Query is acquired. If yes, executing operations S440-470; if not, operation S490 is performed.
In operation S440, the database query statement is analyzed to determine a target database to query.
In operation S450, in case that the database query statement is the first query statement, the first database is queried.
In operation S460, in the case that the database query statement is the second query statement, the second database is queried.
In operation S470, whether the query result is successfully obtained. If yes, perform operation S480; if not, operation S490 is performed.
In operation S480, the data format is unified, and the target data is returned.
In operation S490, an error message is returned.
According to the embodiment of the present disclosure, the first database is set as a database, and the second database is an ES database. Aiming at the problems of the learning cost of a graph database and the slow graph query in a big data scene, the graph database can be used for storing when the data is queried in a graph query mode in a specific scene according to the query scene, and the graph database and the ES database can be packaged together to realize data management service by only using ES storage in other unnecessary situations. Thus, the data management service can determine whether an ES query or a graph query is required by analysis only by inputting initial query information by the user.
According to the embodiment of the disclosure, for example, error information may be returned when the database Query statement corresponding to Query cannot be obtained through the Query analysis module or the Query result cannot be successfully obtained from the database.
According to the embodiment of the disclosure, the error information may include, for example, at least any one of information such as failure of query, failure to find related data, recommendation of similar words, and the like.
By the embodiment of the disclosure, different databases are packaged into the same service, resource recall in more query scenes can be met, the recall rate can be effectively improved, time consumption of multiple queries in single ES storage is reduced, and the problem of database response timeout under the condition of big data query is solved. At the same time, the amount of data in the graph database can be reduced. Moreover, for a developer, the developer only needs to know the ES, so that the learning cost and the development cost are reduced.
According to the embodiment of the present disclosure, before data query is performed, for example, data maintenance may be performed on a data management service, that is, a data maintenance service is provided, a corresponding data processing program is configured for the data management service, so that any data query process in the embodiment of the present disclosure can be followed when data query is needed, and basic data for searching is configured for each database in the data management service. This can improve the success rate of data query using the data management service.
According to an embodiment of the present disclosure, the process of configuring the base data may include, for example: base data is obtained that provides a basis for a query for the target data, where the base data may have a third data format. In the event that it is determined that the underlying data needs to be stored in both the first database and the second database: and converting the basic data into a storage format which is suitable for the first database and storing the basic data in the first database, and converting the basic data into a storage format which is suitable for the second database and storing the basic data in the second database.
According to the embodiment of the disclosure, the data format of the basic data can be different from the storage formats of the first database and the second database, and only the storage format of the corresponding database needs to be adapted when the basic data needs to be stored in the corresponding database.
According to an embodiment of the present disclosure, the data format of the underlying data may also be different from the data format of only one of the first database and the second database. For example, taking the first database as a database and the second database as an ES database as an example, the basic data may be data in an ES format. After receiving the incoming ES formatted data, the data maintenance service may determine if data relationships need to be stored and maintained in both databases at the same time, as needed. Under the condition that basic data need to be stored in the graph database at the same time, the data structure which is suitable for the storage format of the graph database can be obtained by adjusting the data structure, and the storage in the graph database is completed.
Through the embodiment of the disclosure, a data maintenance method is provided, which can provide data support for data management service.
According to an embodiment of the present disclosure, the process of configuring the basic data may further include, for example: and under the condition that the first storage process for storing the basic data into the first database is abnormal, performing rollback (rollback) operation on the first storage process. And under the condition that the second storage process for storing the basic data into the second database is abnormal, performing rollback operation on the second storage process.
Through the above embodiments of the present disclosure, by performing a rollback operation on the second stored procedure, the availability and stability of the database may be maintained.
FIG. 5 schematically illustrates a data maintenance flow diagram for a data maintenance service according to an embodiment of the disclosure.
As shown in fig. 5, the method includes operations S510 to S580.
In operation S510, basic data of an ES format is acquired.
In operation S520, whether the base data requires maintenance of the graph database. If yes, executing operations S530-S550; if not, operations S570-S580 are performed.
In operation S530, a data structure of the base data is adjusted to fit a storage format of the graph database and the edge relationship is maintained.
In operation S540, a graph database is imported.
In operation S550, whether the graph database is successfully imported. If not, operation S560 is performed; if so, operation S580 is performed.
In operation S560, a rollback is performed.
In operation S570, the ES database is imported.
In operation S580, whether the ES database is successfully imported. If not, operation S560 is performed; if yes, the flow is ended.
According to embodiments of the present disclosure, for example, where there is a batch of base data that needs to be imported into a database, all of the base data may be analyzed to determine whether the base data needs to be imported into the database at the same time. When it is determined through analysis that the graph database needs to be imported, the base data can be imported into the graph database as a whole to maintain the relationship between the data, which can be determined by maintaining the edge relationship in the graph database. Meanwhile, the process of importing the basic data into the ES database can be realized in batch, for example.
According to the embodiment of the disclosure, in the case of an error in the process of importing the basic data into the database or the ES database, a rollback operation may be performed on the currently executed import process to restore the correct state of the database before the import process is executed.
Through the above embodiments of the present disclosure, in data maintenance, a developer only needs to know the ES to operate on data. Because the data is transparent to the user, the development cost is reduced, and meanwhile, the rollback guarantee is provided.
FIG. 6 schematically shows an architecture diagram of a data query system according to an embodiment of the present disclosure.
As shown in FIG. 6, data Query system 600 includes, for example, a data management module 610, a data maintenance module 620, a Query parsing module 630, an ES database 640, and a graph database 650.
According to the embodiment of the present disclosure, the data management module 610 is configured to, when receiving a Query request for target data from a user, invoke the Query parsing module 630 to parse initial Query information input by the user, so as to obtain a database Query statement suitable for querying the target data. Then, by analyzing the database query statement, it is determined whether to call the ES database 640 or the database 650 to perform data query, and the queried result is processed in a uniform data format. Thereafter, the data management module 610 may generate a target visualization based on the results of the unified data format and return the target visualization to the user.
According to an embodiment of the present disclosure, the data maintenance module 620 may receive the data in the ES format and determine the database that needs to be maintained by analyzing the data structure, that is, determine whether the data in the ES format needs to be imported into the ES database and the database at the same time. In the case that it is determined that the graph database needs to be imported, the data maintenance module 620 may further change the data structure of the data in the ES format to adapt to the storage format of the graph database, and import the graph database.
Through the embodiment of the disclosure, the data management service and the data maintenance service are combined, so that the reliable, high-efficiency and high-accuracy data query system is realized.
Fig. 7 schematically shows a block diagram of a data querying device according to an embodiment of the present disclosure.
As shown in fig. 7, the data query apparatus 700 includes: a first obtaining module 710, a parsing module 720, a first querying module 730, and a second querying module 740.
The first obtaining module 710 is configured to obtain initial query information for querying the target data.
And an analyzing module 720, configured to analyze the initial query information to obtain a database query statement suitable for querying the target data, where the database query statement is a first query statement or a second query statement.
The first query module 730 is configured to query the target data from the first database according to the first query statement when the database query statement is the first query statement.
The second query module 740 is configured to, if the database query statement is the second query statement, query the target data from the second database according to the second query statement.
According to the embodiment of the disclosure, the database query statement suitable for querying the target data can be obtained by analyzing the initial query information, and the database query statement suitable for querying the target data can be utilized to query data from the corresponding database, so that the technical problem that a single-type data storage mode is low in query efficiency when data query is needed can be at least partially solved, and the technical effect of improving the data query efficiency is further achieved.
According to an embodiment of the present disclosure, the initial query information includes text information. The analysis module comprises: the word segmentation unit, the acquisition unit and the definition unit.
And the word segmentation unit is used for carrying out word segmentation processing on the text information to obtain at least one keyword for representing the text information.
An obtaining unit configured to obtain a target query statement matching at least one keyword.
And the definition unit is used for taking the target query statement as a database query statement.
According to an embodiment of the present disclosure, the at least one keyword includes N keywords, where N is an integer greater than 1. The above-mentioned acquisition unit includes: a first definition subunit and a second definition subunit.
And the first definition subunit is used for taking the full-matching query statement as the target query statement under the condition of acquiring the full-matching query statement matched with the N keywords.
A second defining subunit, configured to, in the case that no query statement matching all of the N keywords is obtained: selecting i target keywords from the N keywords, and acquiring query statements matched with the i target keywords as target query statements, wherein i is less than or equal to 1 and less than N.
According to an embodiment of the present disclosure, the data query apparatus further includes: the device comprises a second acquisition module, a first determination module and a second determination module.
And the second acquisition module is used for acquiring the format of the database query statement.
And the first determining module is used for determining the database query statement as the first query statement under the condition that the format is matched with the format of the query grammar of the first database.
And the second determining module is used for determining the database query statement as the second query statement under the condition that the format is matched with the format of the query grammar of the second database.
According to an embodiment of the present disclosure, the first query module includes: the device comprises a query unit, a determination unit, a conversion unit and a display unit.
And the query unit is used for obtaining a first query result with a first data format by querying the first database according to the first query statement.
And the determining unit is used for determining a target data format for displaying the target data.
And the conversion unit is used for converting the first query result into a target query result with a target data format.
And the display unit is used for displaying the target data based on the target query result.
According to an embodiment of the present disclosure, the determining unit includes: determining the sub-unit and the third defining sub-unit.
And the determining subunit is used for determining a second data format, wherein the second data format corresponds to a second query result obtained by querying the second database according to the second query statement.
And the third definition subunit is used for taking the second data format as the target data format.
According to an embodiment of the present disclosure, the data query apparatus further includes: the device comprises a third acquisition module and a storage module.
And the third acquisition module is used for acquiring basic data for providing a query basis for the target data, wherein the basic data has a third data format.
A storage module, configured to, if it is determined that the base data needs to be stored in both the first database and the second database: and converting the basic data into a storage format which is suitable for the first database and storing the basic data in the first database, and converting the basic data into a storage format which is suitable for the second database and storing the basic data in the second database.
According to an embodiment of the present disclosure, the data query apparatus further includes: a first rollback module and a second rollback module.
The first rollback module is used for executing rollback operation on a first storage process when the first storage process for storing the basic data into the first database is abnormal; and
and the second rollback module is used for executing rollback operation on a second storage process when the second storage process for storing the basic data into the second database is abnormal.
According to an embodiment of the present disclosure, the first database includes a database, and the second database includes a distributed search engine database.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.
According to an embodiment of the present disclosure, a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method as described above.
According to an embodiment of the disclosure, a computer program product comprising a computer program which, when executed by a processor, implements the method as described above.
FIG. 8 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
Computing unit 801 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The calculation unit 801 executes the respective methods and processes described above, such as the data query method. For example, in some embodiments, the data query method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and/or installed onto device 800 via ROM 802 and/or communications unit 809. When loaded into RAM803 and executed by computing unit 801, a computer program may perform one or more of the steps of the data querying method described above. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the data query method in any other suitable manner (e.g., by way of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims (20)

1. A method of data query, comprising:
acquiring initial query information for querying target data;
analyzing the initial query information to obtain a database query statement suitable for querying the target data, wherein the database query statement is a first query statement or a second query statement;
under the condition that the database query statement is the first query statement, querying the target data from a first database according to the first query statement; and
and under the condition that the database query statement is a second query statement, querying the target data from a second database according to the second query statement.
2. The method of claim 1, wherein the initial query information comprises textual information; the analyzing the initial query information to obtain a database query statement suitable for querying the target data includes:
performing word segmentation processing on the text information to obtain at least one keyword for representing the text information;
acquiring a target query statement matched with the at least one keyword; and
and taking the target query statement as the database query statement.
3. The method of claim 2, wherein the at least one keyword comprises N keywords, N being an integer greater than 1; the obtaining of the target query statement matched with the at least one keyword comprises:
under the condition that a full-matching query statement matched with the N keywords is obtained, taking the full-matching query statement as the target query statement;
under the condition that the query sentences matched with the N keywords are not obtained:
selecting i target keywords from the N keywords; and
and acquiring query sentences matched with the i target keywords as the target query sentences, wherein i is more than or equal to 1 and less than N.
4. The method of claim 1, further comprising:
acquiring the format of the database query statement;
determining the database query statement as the first query statement if the format matches a format of a query grammar of the first database; and
determining the database query statement as the second query statement if the format matches a format of the query grammar of the second database.
5. The method of claim 1, wherein, in the case that the database query statement is the first query statement, querying the target data from a first database according to the first query statement comprises:
obtaining a first query result with a first data format by querying the first database according to the first query statement;
determining a target data format for displaying the target data;
converting the first query result into a target query result having the target data format; and
and displaying the target data based on the target query result.
6. The method of claim 5, wherein determining a target data format for presenting the target data comprises:
determining a second data format, wherein the second data format corresponds to a second query result obtained by querying the second database according to the second query statement; and
and taking the second data format as the target data format.
7. The method of claim 1, further comprising:
obtaining basic data for providing a query basis for the target data, wherein the basic data has a third data format;
in the event that it is determined that the base data needs to be stored in both the first database and the second database:
converting the basic data into a storage format suitable for the first database and storing the basic data in the first database; and
and converting the basic data into a storage format which is suitable for the second database and storing the basic data in the second database.
8. The method of claim 7, further comprising:
when a first storage process for storing the basic data into the first database is abnormal, performing rollback operation on the first storage process; and
and under the condition that the second storage process for storing the basic data into the second database is abnormal, executing rollback operation on the second storage process.
9. The method of any of claims 1-8, wherein the first database comprises a database and the second database comprises a distributed search engine database.
10. A data query apparatus, comprising:
the first acquisition module is used for acquiring initial query information for querying target data;
the analysis module is used for analyzing the initial query information to obtain a database query statement suitable for querying the target data, wherein the database query statement is a first query statement or a second query statement;
the first query module is used for querying the target data from a first database according to the first query statement under the condition that the database query statement is the first query statement; and
and the second query module is used for querying the target data from a second database according to the second query statement under the condition that the database query statement is the second query statement.
11. The apparatus of claim 10, wherein the initial query information comprises textual information; the parsing module includes:
the word segmentation unit is used for carrying out word segmentation processing on the text information to obtain at least one keyword for representing the text information;
an obtaining unit configured to obtain a target query statement matching the at least one keyword; and
and the definition unit is used for taking the target query statement as the database query statement.
12. The apparatus of claim 11, wherein the at least one keyword comprises N keywords, N being an integer greater than 1; the acquisition unit includes:
the first definition subunit is used for taking the full-matching query statement as the target query statement under the condition of acquiring the full-matching query statement matched with the N keywords;
a second defining subunit, configured to, when a query statement matching all of the N keywords is not obtained:
selecting i target keywords from the N keywords; and
and acquiring query sentences matched with the i target keywords as the target query sentences, wherein i is more than or equal to 1 and less than N.
13. The apparatus of claim 10, further comprising:
the second acquisition module is used for acquiring the format of the database query statement;
a first determining module, configured to determine the database query statement as the first query statement when the format matches a format of a query syntax of the first database; and
a second determining module, configured to determine the database query statement as the second query statement when the format matches a format of the query syntax of the second database.
14. The apparatus of claim 10, wherein the first query module comprises:
the query unit is used for obtaining a first query result with a first data format by querying the first database according to the first query statement;
the determining unit is used for determining a target data format for displaying the target data;
a conversion unit, configured to convert the first query result into a target query result having the target data format; and
and the display unit is used for displaying the target data based on the target query result.
15. The apparatus of claim 14, wherein the determining unit comprises:
a determining subunit, configured to determine a second data format, where the second data format corresponds to a second query result obtained by querying the second database according to the second query statement; and
a third defining subunit, configured to use the second data format as the target data format.
16. The apparatus of claim 10, further comprising:
a third obtaining module, configured to obtain basic data used for providing a basis for querying the target data, where the basic data has a third data format;
a storage module, configured to, if it is determined that the base data needs to be stored in the first database and the second database at the same time:
converting the basic data into a storage format suitable for the first database and storing the basic data in the first database; and
and converting the basic data into a storage format which is suitable for the second database and storing the basic data in the second database.
17. The apparatus of claim 16, further comprising:
the first rollback module is used for executing rollback operation on a first storage process when the first storage process for storing the basic data into the first database is abnormal; and
and the second rollback module is used for executing rollback operation on a second storage process when the second storage process for storing the basic data into the second database is abnormal.
18. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.
19. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-9.
20. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-9.
CN202110514766.1A 2021-05-11 2021-05-11 Data query method and device, electronic equipment and storage medium Pending CN113220710A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110514766.1A CN113220710A (en) 2021-05-11 2021-05-11 Data query method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110514766.1A CN113220710A (en) 2021-05-11 2021-05-11 Data query method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113220710A true CN113220710A (en) 2021-08-06

Family

ID=77094882

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110514766.1A Pending CN113220710A (en) 2021-05-11 2021-05-11 Data query method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113220710A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114496140A (en) * 2021-12-31 2022-05-13 医渡云(北京)技术有限公司 Data matching method, device, equipment and medium for query conditions
WO2023134614A1 (en) * 2022-01-17 2023-07-20 北京奥星贝斯科技有限公司 Processing of transaction

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109446358A (en) * 2018-08-27 2019-03-08 电子科技大学 A kind of chart database accelerator and method based on ID caching technology
CN110968615A (en) * 2018-09-30 2020-04-07 北京国双科技有限公司 Data query method and device
CN111414407A (en) * 2020-02-13 2020-07-14 中国平安人寿保险股份有限公司 Data query method and device of database, computer equipment and storage medium
CN111475534A (en) * 2020-05-12 2020-07-31 北京爱笔科技有限公司 Data query method and related equipment
CN112416962A (en) * 2020-11-06 2021-02-26 北京偶数科技有限公司 Data query method, device and storage medium
WO2021083239A1 (en) * 2019-10-28 2021-05-06 北京大学 Graph data query method and apparatus, and device and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109446358A (en) * 2018-08-27 2019-03-08 电子科技大学 A kind of chart database accelerator and method based on ID caching technology
CN110968615A (en) * 2018-09-30 2020-04-07 北京国双科技有限公司 Data query method and device
WO2021083239A1 (en) * 2019-10-28 2021-05-06 北京大学 Graph data query method and apparatus, and device and storage medium
CN111414407A (en) * 2020-02-13 2020-07-14 中国平安人寿保险股份有限公司 Data query method and device of database, computer equipment and storage medium
CN111475534A (en) * 2020-05-12 2020-07-31 北京爱笔科技有限公司 Data query method and related equipment
CN112416962A (en) * 2020-11-06 2021-02-26 北京偶数科技有限公司 Data query method, device and storage medium

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114496140A (en) * 2021-12-31 2022-05-13 医渡云(北京)技术有限公司 Data matching method, device, equipment and medium for query conditions
CN114496140B (en) * 2021-12-31 2022-12-30 医渡云(北京)技术有限公司 Data matching method, device, equipment and medium for query conditions
WO2023134614A1 (en) * 2022-01-17 2023-07-20 北京奥星贝斯科技有限公司 Processing of transaction

Similar Documents

Publication Publication Date Title
CN109522341B (en) Method, device and equipment for realizing SQL-based streaming data processing engine
US10191946B2 (en) Answering natural language table queries through semantic table representation
CN111708805A (en) Data query method and device, electronic equipment and storage medium
CN114356971A (en) Data processing method, device and system
CN113220710A (en) Data query method and device, electronic equipment and storage medium
JP2022003509A (en) Entity relation mining method, device, electronic device, computer readable storage medium, and computer program
CN114579104A (en) Data analysis scene generation method, device, equipment and storage medium
CN113568938A (en) Data stream processing method and device, electronic equipment and storage medium
CN113609100B (en) Data storage method, data query device and electronic equipment
CN113836316A (en) Processing method, training method, device, equipment and medium for ternary group data
CN108959294B (en) Method and device for accessing search engine
CN113760948A (en) Data query method and device
CN116955856A (en) Information display method, device, electronic equipment and storage medium
CN114115908A (en) Instruction set generation method, program analysis method, device and storage medium
CN113468529A (en) Data searching method and device
CN112988778A (en) Method and device for processing database query script
CN113971216B (en) Data processing method and device, electronic equipment and memory
CN115563183B (en) Query method, query device and program product
CN115828915B (en) Entity disambiguation method, device, electronic equipment and storage medium
US20220374603A1 (en) Method of determining location information, electronic device, and storage medium
CN117806965A (en) Database testing method and device, electronic equipment and storage medium
CN114428891A (en) Network resource processing method, device, system, equipment and storage medium
CN117633194A (en) Large model prompt data processing method and device, electronic equipment and storage medium
CN116383333A (en) Data storage method, device, equipment and storage medium
CN115146070A (en) Key value generation method, knowledge graph generation method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination