CN116340336A - Data query method and device based on database analysis engine - Google Patents
Data query method and device based on database analysis engine Download PDFInfo
- Publication number
- CN116340336A CN116340336A CN202310313285.3A CN202310313285A CN116340336A CN 116340336 A CN116340336 A CN 116340336A CN 202310313285 A CN202310313285 A CN 202310313285A CN 116340336 A CN116340336 A CN 116340336A
- Authority
- CN
- China
- Prior art keywords
- database
- queried
- data
- target information
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 76
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000003780 insertion Methods 0.000 claims abstract description 65
- 230000037431 insertion Effects 0.000 claims abstract description 65
- 238000004891 communication Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 13
- 238000013500 data storage Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001172 regenerating effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data query method and device based on a database analysis engine, wherein the database analysis engine receives a data query request, and the data query request comprises information to be queried; the database analysis engine converts the information to be queried into a query format of each queried database; the database analysis engine queries the stored hash table according to the query format of each queried database to acquire target information in different queried databases corresponding to the converted information to be queried; and the database analysis engine inserts the target information in different queried databases into the public database according to preset insertion conditions. The method solves the technical problem of low data gathering efficiency of the multiple databases in the prior art.
Description
Technical Field
The present application relates to the field of database analysis, and in particular, to a data query method and apparatus based on a database analysis engine.
Background
The database stores the data together in a certain mode, so that all users can share the data set with the redundancy reduced as much as possible.
The database engine is a core service for storing, processing and protecting data. The database engine can be used for controlling access rights and rapidly processing transactions, thereby meeting the requirements of most application programs in enterprises which need to process a large amount of data.
At present, information in each enterprise may be stored in different databases, and a plurality of databases are independent from each other, and the storage modes of the data are different, for example: oracle, mysql or other types of databases, if data among different databases is to be searched, a corresponding person cannot be found out in time to inquire the data through a data engine corresponding to each database so as to push the data, so that the data cannot be quickly searched and summarized, and huge trouble is brought to the data counting into a unified system.
Disclosure of Invention
An embodiment of the application aims to provide a data query method and device based on a database analysis engine, which are used for solving the technical problem that the efficiency of summarizing data by multiple types of databases in the prior art is low.
In a first aspect, a data query method based on a database parsing engine is provided, the method may include:
the method comprises the steps that a database analysis engine receives a data query request, wherein the data query request comprises information to be queried;
the database analysis engine converts the information to be queried into a query format of each queried database;
the database analysis engine queries the stored hash table according to the query format of each queried database to obtain target information in different queried databases corresponding to the converted information to be queried; the hash table comprises identification IDs of all the queried databases and data of corresponding databases;
and the database analysis engine inserts the target information in different queried databases into a public database according to preset insertion conditions.
Further, the hash table construction includes:
the database analysis engine acquires the identification ID of each queried database and the data in the corresponding database;
the database analysis engine converts the identification ID of each queried database and the data in the corresponding database into a hash value;
and the database analysis engine constructs a hash table by using the identification ID of each queried database and the hash value of the data in the corresponding database.
Further, the step of querying, by the database parsing engine, the stored hash table according to the query format of each queried database, and obtaining target information corresponding to the converted information to be queried, includes:
and the database analysis engine queries the stored hash table according to the identification ID of each queried database to obtain the target information in the corresponding queried database.
Further, the common database includes a plurality of data tables, each queried database including a plurality of data tables;
before the target information is inserted into the public database according to the preset insertion condition, the method further comprises the following steps:
the database analysis engine displays an insertion position indication interface to a user, wherein the insertion position indication interface comprises two display areas, the first display area is used for displaying data field information of a plurality of data tables of the public database, and the second display area is used for displaying data field information of a plurality of data tables of different queried databases;
receiving a first trigger operation of a user on a position to be inserted of any data table in the public database and a second trigger operation of any data table in the corresponding queried database aiming at target information in each queried database;
and inserting the target information of the queried database corresponding to the second triggering operation into the position to be inserted corresponding to the first triggering operation.
Further, after the target information is inserted into the public database according to a preset insertion condition, the method further includes:
obtaining an insertion result of inserting the target information into the position to be inserted;
if the insertion result is that the insertion is successful, the database analysis engine sends prompt information of the target information at the corresponding position of the public database to the user;
if the insertion result is that the insertion fails, the database analysis engine sends prompt information for reselecting the position to be inserted to the user.
Further, the public database comprises a plurality of stored data tables and standby data tables; inserting the target information into the public database according to a preset insertion condition, wherein the method comprises the following steps:
detecting the similarity W between the data fields in the target information and the data fields in all the stored data tables in the public database;
if W is more than or equal to a preset similarity threshold W0, inserting the target information into the corresponding stored data table based on the number of the stored data tables meeting the condition that W is more than or equal to W0 and the data storage capacity of each stored data table;
if W is less than W0, the target information is inserted into a standby data table in the public database.
Further, if W.gtoreq.W0, inserting the target information into the corresponding stored data table based on the number of stored data tables meeting the W.gtoreq.W0 condition and the data storage capacity of each stored data table, including:
if the number of the stored data tables meeting the condition that W is more than or equal to W0 is 1 and the storage capacity of the stored data tables reaches the preset maximum capacity, inserting the target information into the standby data tables;
if the number of the stored data tables meeting the condition that W is more than or equal to W0 is more than 1 and the storage capacity of each stored data table reaches the preset maximum capacity, inserting the target information into the standby data table;
if the number of the stored data tables meeting the condition that W is more than or equal to W0 is more than 1, and the stored data tables with the storage capacity not reaching the preset maximum capacity exist, inserting the target information into the stored data tables with the large priority according to the priority of the stored data tables.
In a second aspect, there is provided a data query apparatus based on a database parsing engine, the apparatus comprising:
the receiving unit is used for receiving a data query request, wherein the data query request comprises information to be queried;
the conversion unit is used for converting the information to be queried into query formats of all the queried databases;
the query unit is used for querying the stored hash table according to the query format of each queried database;
the acquisition unit is used for inquiring the stored hash table according to the inquiry format of each inquired database and acquiring target information in different inquired databases corresponding to the converted information to be inquired; the hash table comprises identification IDs of all the queried databases and data of corresponding databases;
and the inserting unit inserts the target information in different queried databases into the public database according to preset inserting conditions.
In a third aspect, an electronic device is provided, the electronic device comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory are in communication with each other via the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any one of the above first aspects when executing a program stored on a memory.
In a fourth aspect, a computer-readable storage medium is provided, in which a computer program is stored which, when being executed by a processor, carries out the method steps of any of the first aspects.
The application provides a data query method based on a database analysis engine, wherein the database analysis engine receives a data query request, and the data query request comprises information to be queried; the database analysis engine converts the information to be queried into a query format of each queried database; the database analysis engine queries the stored hash table according to the query format of each queried database to acquire target information in different queried databases corresponding to the converted information to be queried; the hash table comprises identification IDs of all queried databases and data of corresponding databases; the database analysis engine inserts target information in different queried databases into the public database according to preset insertion conditions; the method can be adapted among various databases, and reduces the development of a large number of repeated codes; and the data summarizing efficiency can be improved while each data is accurately searched according to the user demand.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a data query method based on a database parsing engine according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a data query device based on a database parsing engine according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present application without making any inventive effort, are intended to be within the scope of the present application.
The data query method of the database analysis engine provided by the application needs to establish the connection between the public database and any database to be queried so as to realize the quick connection between the public database and any database to be queried, and further realize the efficient data summarization.
The step of connecting the public database and any queried database can comprise the following steps:
first, the bottom layer in the framework hierarchy sets up different types of queried databases. The queried database may be: dream, oracle, mysql, postgreSql, tiDB, etc.
Secondly, information of different types of queried databases and public databases is configured, and relevant configuration information is stored in the public database of the relational database. Wherein the information of the different types of queried databases and the public database comprises: basic information and dynamic information.
The basic information may be: an account number of a database, a database password, a database connection address, and the like.
The dynamic information may be: the connection time of the database, the connection pool size of the database, the maximum number of connections, the minimum number of connections, and dynamic database information that may be appended.
And obtaining the account number and the password of the database to be queried.
And finally, providing an abstract public method interface through a service layer of the internal framework according to the account number and the password, and realizing a concrete example method in a plurality of concrete classes through the public method interface so as to realize connection with a plurality of queried databases.
The method has good expandability, when the data of the new queried database is needed to be accessed, the information of the new queried database is only needed to be configured and added into the public database, and the database analysis engine is automatically updated and connected with the new queried database.
The preferred embodiments of the present application will be described below with reference to the accompanying drawings of the specification, it being understood that the preferred embodiments described herein are for illustration and explanation only, and are not intended to limit the present application, and embodiments and features of embodiments of the present application may be combined with each other without conflict.
Fig. 1 is a flowchart of a data query method based on a database parsing engine according to an embodiment of the present application. As shown in fig. 1, the method may be performed by a database parsing engine, and the method may include:
step S110, a data query request is received.
In particular implementations, the database parsing engine may receive a data query request sent by a user, which may include information to be queried.
And step S120, converting the information to be queried into a query format of each queried database.
Specifically, because the query grammars of different queried databases are different, the database parsing engine needs to convert the information to be queried into the query formats of the different queried databases. At this time, the database analysis engine automatically connects the generated query information in various query formats with the corresponding queried database.
Specifically, the internal code writes multiple implementation imp class implementation service interfaces.
When a plurality of imp realization service interfaces exist, the database analysis engine internally sets sqlparse analysis based on different databases, and the codes execute the rewriting query information to obtain query formats corresponding to the query information of a plurality of queried databases of different types.
It can be seen that the code analysis tool based on different databases is provided in the application, and the query format of each queried database can be automatically generated based on the information to be queried of the query request. The cost of manually entering the query format is reduced to further increase the efficiency of querying data.
And step 130, inquiring the stored hash table according to the inquiry format of each inquired database, and acquiring the target information in different inquired databases corresponding to the converted information to be inquired.
The hash table may include, among other things, the identification ID of each queried database and the data of the corresponding database.
The hash table can be specifically constructed according to the following steps:
first, constructing a unique identification ID of a queried database and data of a corresponding database, including:
and acquiring the identification parameters of each queried database and the data parameters of the corresponding database. And writing a plurality of realization imp type realization service interfaces through the internal codes of the database analysis engine, and regenerating the identification parameters of the queried database into the identification IDs of the database, namely generating the identification IDs of the queried database as unique values. Therefore, the method can avoid the repeated value of the identification ID of the queried database so as to influence the accuracy of data query.
Secondly, obtaining the identification ID of each queried database and the data in the corresponding database to construct a hash table, which can comprise:
and converting the identification ID of each queried database and the data in the corresponding database into hash values.
And constructing a hash table by using the hash value of the identification ID of each queried database and the hash value of the data in the corresponding database. Specifically, the database analysis engine queries all the queried databases in the external interfaces through the specific data importing example method and loads all the queried databases in the external interfaces into the data importing example method to form a hash table.
The database parsing engine internally uses factory class code to write a specific instance method. All imp classes in the level interface are queried through the in-frame loop. All imp classes are loaded inside the specific instance method to compose one hashMap.
And then, according to the identification ID of each queried database, querying the stored hash table to obtain the target information in the corresponding queried database, and at the moment, sending prompt information of successful acquisition of the target information to the user by the database analysis engine. Specifically, unique Value identification ID of a queried database corresponding to each imp class is used as a Key, and a specific method in the imp class is used as Value to acquire target information.
If the target information corresponding to the information to be queried is not queried, a prompt that the target information does not exist is sent to the user, and prompt information for inputting the information to be queried again is sent.
It can be seen that the present approach can implement a specific instance method inside an imp class obtained by one ID. The database analysis engine receives the identification ID of the queried database, and can acquire the target information of the corresponding database in the hash table through loading the identification ID into a storage set (the storage set is acquired by a specific example method) in the whole program where the application is running. The method can acquire the specific methods inside different imp types by acquiring the identification IDs of different queried databases and loading the specific instantiation methods into the global variables, namely, invoking the data in different queried databases to acquire the target information.
And step 140, inserting the target information in different queried databases into the public database according to preset insertion conditions.
The common database includes a plurality of stored data tables and a spare data table. Each queried database includes a plurality of stored data tables.
In particular implementations, the manner in which the target information in the different queried databases is inserted into the common database may include:
in one mode, after obtaining the target information in different queried databases, the database parsing engine may display an insertion position indication interface to a user, where the insertion position indication interface may include two display areas: a first display area for displaying data field information of a plurality of data tables of a common database and a second display area for displaying data field information of a plurality of data tables of different queried databases.
Receiving a first trigger operation of a user on a position to be inserted of any data table in the public database and a second trigger operation of any data table in the corresponding queried database aiming at target information in each queried database;
and then, inserting the target information of the queried database corresponding to the second triggering operation into the position to be inserted corresponding to the first triggering operation.
According to the preset insertion condition of the first mode, after the target information is inserted into the public database, the method further comprises:
obtaining an insertion result of inserting target information into a position to be inserted;
if the insertion result is that the insertion is successful, the database analysis engine sends prompt information of the target information at the corresponding position of the public database to the user;
if the insertion result is that the insertion fails, the database analysis engine sends prompt information for reselecting the position to be inserted to the user.
If the position to be inserted (the stored table or the standby data table) is empty, the insertion result is that the insertion is successful, and the database analysis engine sends prompt information of the target information at the corresponding position of the public database to the user.
When the location to be inserted (the stored table or the spare data table) is not empty, the storage capacity of the location to be inserted and the data size of the target information may also be detected.
If the storage capacity of the position to be inserted is larger than the data size of the target information, the insertion result is that the insertion is successful, and the database analysis engine sends prompt information of the target information at the corresponding position of the public database to the user.
If the storage capacity of the insertion position is not larger than the data size of the target information, the insertion result is that the insertion fails, and the database analysis engine sends prompt information for reselecting the position to be inserted to the user.
Specifically, target information is received according to a to-be-inserted position of to-be-queried information in a public database input by a user by a database analysis engine, and a code analysis tool based on different databases is provided.
The database analysis engine internally sets sqlparse analysis based on different databases, generates an inserted sentence of the public database, and executes the inserted sentence into a corresponding position of the public database.
Therefore, the method and the device can collect the target information in the databases of different types based on the target information inserting position triggered by the page receiving user, automatically and conveniently insert data with the data table of the public database through the visual page triggering inserting data storage position, and can improve the efficiency of data collection in different queried databases while improving the efficiency of data query.
In a second mode, the database analysis engine obtains preset insertion conditions in advance, and for inserting target information in different queried databases into a public database according to the preset insertion conditions, the method includes:
and detecting the similarity W of the data fields in the target information and the data fields in each stored data table in the public database.
If W is greater than or equal to a preset similarity threshold W0, inserting target information into the corresponding stored data tables based on the number of stored data tables meeting the condition that W is greater than or equal to W0 and the data storage capacity of each stored data table.
If the number of the stored data tables meeting the condition that W is more than or equal to W0 is 1 and the storage capacity of the stored data tables reaches the preset maximum capacity, inserting target information into the standby data tables.
If the number of the stored data tables meeting the condition that W is more than or equal to W0 is more than 1 and the storage capacity of each stored data table reaches the preset maximum capacity, inserting target information into the standby data table.
If the number of the stored data tables meeting the condition that W is more than or equal to W0 is more than 1, and the stored data tables with the storage capacity not reaching the preset maximum capacity exist, inserting the target information into the stored data tables with the large priority according to the priority of the stored data tables.
If W is less than W0, the target information is inserted into a standby data table in the public database.
The sqlparse analysis based on different databases is arranged in the program, the insertion data format of the public database can be automatically generated based on the information to be queried of different query requests, and the target information acquired by the information to be queried is inserted into the corresponding position.
Therefore, according to the method and the device for automatically selecting the insertion position of the target information, the data fields of the tables in the public database are matched with the data fields of the target information to obtain the similarity of the fields, and the insertion position of the target information is selected according to the storage capacity of the corresponding table, so that the data can be stored by simultaneously considering according to the similarity of the data and the storage space capacity, and the accuracy and the safety of data storage are further ensured while the data summarizing efficiency is improved.
The application provides a data query method based on a database analysis engine, wherein the database analysis engine receives a data query request, and the data query request comprises information to be queried; the database analysis engine converts the information to be queried into a query format of each queried database; the database analysis engine queries the stored hash table according to the query format of each queried database to acquire target information in different queried databases corresponding to the converted information to be queried; the hash table comprises identification IDs of all queried databases and data of corresponding databases; the database analysis engine inserts target information in different queried databases into the public database according to preset insertion conditions; the method can be adapted among various databases, and reduces the development of a large number of repeated codes; and the data summarizing efficiency can be improved while each data is accurately searched according to the user demand.
Corresponding to the method, the embodiment of the application also provides a data query device based on a database analysis engine, as shown in fig. 2, which is a schematic structural diagram of the data query device based on the database analysis engine, where the data query device based on the database analysis engine includes:
the receiving unit 210 is configured to receive a data query request, where the data query request includes information to be queried.
The conversion unit 220 is configured to convert the information to be queried into a query format of each queried database.
A query unit 230, configured to query the stored hash table according to the query format of each queried database.
An obtaining unit 240, configured to query the stored hash table according to a query format of each queried database, and obtain target information in different queried databases corresponding to the converted information to be queried; wherein the hash table comprises the identification ID of each queried database and the data of the corresponding database.
And the inserting unit 250 is used for inserting the target information in different queried databases into the public database according to preset inserting conditions.
The functions of each functional unit of the data query device based on the database analysis engine provided in the foregoing embodiments of the present application may be implemented through the foregoing method steps, so specific working processes and beneficial effects of each unit in the data query device based on the database analysis engine provided in the embodiments of the present application are not repeated herein.
The embodiment of the present application further provides an electronic device, as shown in fig. 3, which is a schematic structural diagram of an electronic device, and the electronic device includes a processor 310, a communication interface 320, a memory 330, and a communication bus 340, where the processor 310, the communication interface 320, and the memory 330 complete communication with each other through the communication bus 340.
A memory 330 for storing a computer program;
the processor 310 is configured to execute the program stored in the memory 330, and implement the following steps:
and receiving a data query request, wherein the data query request comprises information to be queried.
And converting the information to be queried into a query format of each queried database.
Inquiring the stored hash table according to the inquiry format of each inquired database, and acquiring target information in different inquired databases corresponding to the converted information to be inquired; wherein the hash table comprises the identification ID of each queried database and the data of the corresponding database.
And inserting the target information in different queried databases into the public database according to preset insertion conditions.
In an alternative embodiment, the hash table construction includes:
and acquiring the identification ID of each queried database and the data in the corresponding database.
And converting the identification ID of each queried database and the data in the corresponding database into hash values.
And constructing a hash table by using the identification IDs of the queried databases and the hash values of the data in the corresponding databases.
In an optional implementation manner, according to a query format of each queried database, querying a stored hash table, and obtaining target information corresponding to the converted information to be queried, where the step includes:
and querying the stored hash table according to the identification ID of each queried database to obtain the target information in the corresponding queried database.
In an alternative embodiment, the common database includes a plurality of data tables, and each queried database includes a plurality of data tables.
The manner in which the target information in the different queried databases is inserted into the common database may include:
in one mode, after obtaining the target information in the different queried databases, the database analysis engine may display an insertion position indication interface to the user, where the insertion position indication interface may include two display areas, a first display area for displaying data field information of multiple data tables of the common database and a second display area for displaying data field information of multiple data tables of the different queried databases. The multiple data tables of the public database are multiple stored data tables and standby data tables, and the multiple data tables of each queried database are multiple stored data tables.
And receiving a first trigger operation of a user on a position to be inserted of any data table in the public database and a second trigger operation of any data table in the corresponding queried database aiming at target information in each queried database.
And then, inserting the target information of the queried database corresponding to the second triggering operation into the position to be inserted corresponding to the first triggering operation.
According to the preset insertion condition of the first mode, after the target information is inserted into the public database, the method further comprises:
obtaining the insertion result of the target information to the position to be inserted.
If the insertion result is that the insertion is successful, the database analysis engine sends prompt information of the target information at the corresponding position of the public database to the user.
If the insertion result is that the insertion fails, the database analysis engine sends prompt information for reselecting the position to be inserted to the user.
If the position to be inserted (the stored table or the standby data table) is empty, the insertion result is that the insertion is successful, and the database analysis engine sends prompt information of the target information at the corresponding position of the public database to the user.
When the location to be inserted (the stored table or the spare data table) is not empty, the storage capacity of the location to be inserted and the data size of the target information may also be detected.
If the storage capacity of the position to be inserted is larger than the data size of the target information, the insertion result is that the insertion is successful, and the database analysis engine sends prompt information of the target information at the corresponding position of the public database to the user.
If the storage capacity of the insertion position is not larger than the data size of the target information, the insertion result is that the insertion fails, and the database analysis engine sends prompt information for reselecting the position to be inserted to the user.
In a second mode, the database analysis engine obtains preset insertion conditions in advance, and for inserting target information in different queried databases into a public database according to the preset insertion conditions, the method includes:
and detecting the similarity W of the data fields in the target information and the data fields in each stored data table in the public database.
If W is greater than or equal to a preset similarity threshold W0, inserting target information into the corresponding stored data tables based on the number of stored data tables meeting the condition that W is greater than or equal to W0 and the data storage capacity of each stored data table.
If the number of the stored data tables meeting the condition that W is more than or equal to W0 is 1 and the storage capacity of the stored data tables reaches the preset maximum capacity, inserting target information into the standby data tables.
If the number of the stored data tables meeting the condition that W is more than or equal to W0 is more than 1 and the storage capacity of each stored data table reaches the preset maximum capacity, inserting target information into the standby data table.
If the number of the stored data tables meeting the condition that W is more than or equal to W0 is more than 1, and the stored data tables with the storage capacity not reaching the preset maximum capacity exist, inserting the target information into the stored data tables with the large priority according to the priority of the stored data tables.
If W is less than W0, the target information is inserted into a standby data table in the public database.
The communication bus mentioned above may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, or the like. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the electronic device and other devices.
The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processing, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
Since the implementation manner and the beneficial effects of the solution to the problem of each device of the electronic apparatus in the foregoing embodiment may be implemented by referring to each step in the embodiment shown in fig. 1, the specific working process and the beneficial effects of the electronic apparatus provided in the embodiment of the present application are not repeated herein.
In yet another embodiment provided herein, a computer readable storage medium is provided, in which instructions are stored, which when executed on a computer, cause the computer to perform a data query method based on a database parsing engine as described in any one of the above embodiments.
In yet another embodiment provided herein, there is also provided a computer program product containing instructions that, when run on a computer, cause the computer to perform a database parsing engine-based data query method as described in any of the above embodiments.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, embodiments of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted to embrace the preferred embodiments and all such variations and modifications as fall within the scope of the embodiments herein.
It will be apparent to those skilled in the art that various modifications and variations can be made in the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if such modifications and variations of the embodiments in the present application fall within the scope of the claims and the equivalents thereof in the embodiments of the present application, such modifications and variations are also intended to be included in the embodiments of the present application.
Claims (10)
1. A data query method based on a database parsing engine, the method comprising:
the method comprises the steps that a database analysis engine receives a data query request, wherein the data query request comprises information to be queried;
the database analysis engine converts the information to be queried into a query format of each queried database;
the database analysis engine queries the stored hash table according to the query format of each queried database to obtain target information in different queried databases corresponding to the converted information to be queried; the hash table comprises identification IDs of all the queried databases and data of corresponding databases;
and the database analysis engine inserts the target information in different queried databases into a public database according to preset insertion conditions.
2. The method of claim 1, wherein the hash table construction comprises:
the database analysis engine acquires the identification ID of each queried database and the data in the corresponding database;
the database analysis engine converts the identification ID of each queried database and the data in the corresponding database into a hash value;
and the database analysis engine constructs a hash table by using the identification ID of each queried database and the hash value of the data in the corresponding database.
3. The method as claimed in claim 2, wherein the step of the database parsing engine querying the stored hash table according to the query format of each queried database to obtain the target information corresponding to the converted information to be queried, comprises:
and the database analysis engine queries the stored hash table according to the identification ID of each queried database to obtain the target information in the corresponding queried database.
4. The method of claim 1, wherein the common database comprises a plurality of data tables, each queried database comprising a plurality of data tables;
before the target information is inserted into the public database according to the preset insertion condition, the method further comprises the following steps:
the database analysis engine displays an insertion position indication interface to a user, wherein the insertion position indication interface comprises two display areas, the first display area is used for displaying data field information of a plurality of data tables of the public database, and the second display area is used for displaying data field information of a plurality of data tables of different queried databases;
receiving a first trigger operation of a user on a position to be inserted of any data table in the public database and a second trigger operation of any data table in the corresponding queried database aiming at target information in each queried database;
and inserting the target information of the queried database corresponding to the second triggering operation into the position to be inserted corresponding to the first triggering operation.
5. The method of claim 4, wherein after inserting the target information into the common database according to a preset insertion condition, the method further comprises:
obtaining an insertion result of inserting the target information into the position to be inserted;
if the insertion result is that the insertion is successful, the database analysis engine sends prompt information of the target information at the corresponding position of the public database to the user;
if the insertion result is that the insertion fails, the database analysis engine sends prompt information for reselecting the position to be inserted to the user.
6. The method of claim 1, wherein the common database comprises a plurality of stored data tables and spare data tables; inserting the target information into the public database according to a preset insertion condition, wherein the method comprises the following steps:
detecting the similarity W between the data fields in the target information and the data fields in all the stored data tables in the public database;
if W is more than or equal to a preset similarity threshold W0, inserting the target information into the corresponding stored data table based on the number of the stored data tables meeting the condition that W is more than or equal to W0 and the data storage capacity of each stored data table;
if W is less than W0, the target information is inserted into a standby data table in the public database.
7. The method of claim 6, wherein if W.gtoreq.W0, inserting the target information into each stored data table based on the number of stored data tables meeting the W.gtoreq.W0 condition and the data storage capacity of each stored data table, comprises:
if the number of the stored data tables meeting the condition that W is more than or equal to W0 is 1 and the storage capacity of the stored data tables reaches the preset maximum capacity, inserting the target information into the standby data tables;
if the number of the stored data tables meeting the condition that W is more than or equal to W0 is more than 1 and the storage capacity of each stored data table reaches the preset maximum capacity, inserting the target information into the standby data table;
if the number of the stored data tables meeting the condition that W is more than or equal to W0 is more than 1, and the stored data tables with the storage capacity not reaching the preset maximum capacity exist, inserting the target information into the stored data tables with the large priority according to the priority of the stored data tables.
8. A database parsing engine based data query apparatus, the apparatus comprising:
the receiving unit is used for receiving a data query request, wherein the data query request comprises information to be queried;
the conversion unit is used for converting the information to be queried into a query format of each queried database by the database analysis engine;
the query unit is used for querying the stored hash table according to the query format of each queried database;
the acquisition unit is used for acquiring target information corresponding to the converted information to be queried; the hash table comprises identification IDs of all the queried databases and data of corresponding databases;
an inserting unit, configured to insert the target information into a public database according to a preset inserting condition; the preset insertion condition is a condition for inserting the target information into the appointed position of the public database.
9. An electronic device, characterized in that the electronic device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are in communication with each other through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any of claims 1-7 when executing a program stored on a memory.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310313285.3A CN116340336A (en) | 2023-03-28 | 2023-03-28 | Data query method and device based on database analysis engine |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310313285.3A CN116340336A (en) | 2023-03-28 | 2023-03-28 | Data query method and device based on database analysis engine |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116340336A true CN116340336A (en) | 2023-06-27 |
Family
ID=86885343
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310313285.3A Pending CN116340336A (en) | 2023-03-28 | 2023-03-28 | Data query method and device based on database analysis engine |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116340336A (en) |
-
2023
- 2023-03-28 CN CN202310313285.3A patent/CN116340336A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107463635B (en) | Method for inquiring picture data and distributed NewSQL database system | |
| JP7021228B2 (en) | Blockchain-based data storage and query methods and devices | |
| CN108519967B (en) | Chart visualization method and device, terminal and storage medium | |
| CN107798038B (en) | Data response method and data response equipment | |
| CN108647357B (en) | Data query method and device | |
| AU2017200895B2 (en) | Software integration testing with unstructured database | |
| CN110704476A (en) | Data processing method, device, equipment and storage medium | |
| WO2022057357A1 (en) | Data query method and apparatus, and database system | |
| CN112182036A (en) | Data sending and writing method and device, electronic equipment and readable storage medium | |
| US20170278005A1 (en) | Technologies for auto discover and connect to a rest interface | |
| CN108140022B (en) | Data query method and database system | |
| CN115408193A (en) | Method and device for processing error codes among multiple modules | |
| CN112860802B (en) | Database operation statement processing method and device and electronic equipment | |
| CN106919607B (en) | Data access method, device and system | |
| CN114297204A (en) | Data storage and retrieval method and device for heterogeneous data source | |
| CN102855297B (en) | A kind of method of control data transmission and connector | |
| CN116340336A (en) | Data query method and device based on database analysis engine | |
| US10838947B2 (en) | Consistency check for foreign key definition | |
| US11157506B2 (en) | Multiform persistence abstraction | |
| CN114238391A (en) | Data paging query method and device, electronic equipment and storage medium | |
| CN112632115A (en) | BI-based data query method and system | |
| CN108228604B (en) | Model construction method based on memory object, information query method and device | |
| CN112256489A (en) | Data acquisition method and device of cloud development platform and data storage architecture | |
| CN110716961A (en) | Data processing method and device | |
| KR102054303B1 (en) | Apparatus and method for processing log data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |