CN114547095A - Data rapid query method and device, electronic equipment and storage medium - Google Patents
Data rapid query method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114547095A CN114547095A CN202210125784.5A CN202210125784A CN114547095A CN 114547095 A CN114547095 A CN 114547095A CN 202210125784 A CN202210125784 A CN 202210125784A CN 114547095 A CN114547095 A CN 114547095A
- Authority
- CN
- China
- Prior art keywords
- query
- data
- keyword
- obtaining
- hot word
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/243—Natural language query formulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a method, a device, electronic equipment and a storage medium for fast data query, wherein the method comprises the following steps: acquiring query information, and analyzing the query information to obtain query keywords; judging whether the query keyword is in a pre-constructed hot word index or not; when the query keyword is in the hot word index, obtaining target data from a cache database; when the query keyword is not in the hot word index, obtaining an associated keyword according to the query keyword and a preset keyword association table, obtaining initial return data from a dynamic database according to the associated keyword, and optimizing the initial return data to obtain target data; and obtaining a query result according to the target data. The method can improve the efficiency of parallel processing and query of the big data, realize real-time query of the business data, improve the operation efficiency of enterprises, reduce the operation and maintenance cost of the enterprises, more comprehensively and accurately provide the business query result and improve the experience of users.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and an apparatus for fast querying data, an electronic device, and a storage medium.
Background
In recent years, with the expansion of business scope of enterprises, the amount of business data stored in databases is rapidly increasing. When a user queries service data through a service terminal, a data processing center usually needs to spend a large amount of time and system resources to search a result required by the terminal from a database; meanwhile, under the condition that the number of concurrent users is large, a connection channel of a background can be occupied quickly, so that more and more users cannot respond to queries quickly, and user experience is affected.
At present, methods for processing large data query mainly include database partitioning, table partitioning, virtual memory addition and the like, but these methods can only reduce the query time to a certain extent, and as the data volume continuously increases, the performance of the database system and the data organization management capability are weakened, even the system is crashed.
Therefore, it is urgently needed to provide a method for quickly querying data, which can realize quick, efficient and accurate query of business data, improve operation and maintenance efficiency of enterprises, and improve experience of users.
Disclosure of Invention
In view of the above, there is a need to provide a method, an apparatus, an electronic device, and a computer-readable storage medium for fast querying data, so as to solve the problems of slow response speed, low efficiency, and inaccurate query result when querying business data in the context of large data volume.
In order to solve the above problem, the present invention provides a method for fast querying data, which includes:
acquiring query information, and analyzing the query information to obtain query keywords;
judging whether the query keyword is in a pre-constructed hot word index or not;
when the query keyword is in the hot word index, obtaining target data from a cache database; when the query keyword is not in the hotword index, obtaining an associated keyword according to the query keyword and a preset keyword association table, obtaining initial return data from a dynamic database according to the associated keyword, and optimizing the initial return data to obtain target data;
and obtaining a query result according to the target data.
Further, acquiring query information, and analyzing a query keyword of the query information, including:
analyzing the query information according to a preset data format configuration table to obtain a standard query statement;
and obtaining a query keyword according to the standard query statement.
Further, the construction of the hotword index comprises:
acquiring historical query data in a first preset time period;
obtaining query hot words with query frequency larger than a first threshold value according to the historical query data;
and obtaining the hot word index according to the query hot word.
Further, the method further comprises:
obtaining a query log according to the query keyword;
and updating the hot word index and the cache database according to the query log.
Further, updating the hotword index and the cache database according to the query log, including:
acquiring a query log in a second preset time period, sorting query keywords in the query log according to query frequency, extracting the query keywords of which the query frequency is greater than a second threshold value, and obtaining updated query hot words;
updating the hot word index according to the updated query hot word to obtain an updated hot word index;
sorting the query keywords in the query log within a third preset time period according to query time, and extracting the query keywords of which the query time is later than a preset time threshold value to obtain recent query words;
and respectively acquiring the updated query hot words and the return data corresponding to the recent query words from the dynamic database, and storing the return data into a cache database.
Further, the keyword association table presetting method comprises the following steps:
acquiring at least one of a near meaning word, a synonym, an homophone and an English translation word of the keyword to obtain a relevant word of the keyword;
and establishing a mapping relation between the keywords and the associated words to obtain the keyword association table.
Further, optimizing the initial return data to obtain target data, including:
and carrying out duplicate removal and sequencing on the initial return data to obtain target data.
The invention also provides a data quick query device, comprising:
the acquisition module is used for acquiring query information and analyzing the query information to obtain query keywords;
the judging module is used for judging whether the query keyword is in a pre-constructed hot word index or not;
the data calling module is used for obtaining target data from a cache database when the query keyword is in the hot word index; when the query keyword is not in the hotword index, obtaining an associated keyword according to the query keyword and a preset keyword association table, obtaining initial return data from a dynamic database according to the associated keyword, and optimizing the initial return data to obtain target data;
and the response module is used for obtaining a query result according to the target data.
The invention also provides an electronic device, which comprises a processor and a memory, wherein the memory is stored with a computer program, and when the computer program is executed by the processor, the method for quickly querying data is realized according to any technical scheme.
The invention also provides a computer readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for quickly querying data according to any one of the above technical solutions is implemented.
Compared with the prior art, the invention has the beneficial effects that: firstly, acquiring query information, and analyzing to obtain query keywords; secondly, judging whether the query keyword is in the hot word index; and finally, returning the target data from the database to obtain a query result. The method can improve the efficiency of parallel processing and query of the big data, can realize the real-time query of the business data under the background of large data volume, improves the overall business operation efficiency and reduces the operation and maintenance cost of enterprises; meanwhile, the service query requirements can be more comprehensively and accurately met, the query result is easy to look up, and the experience of the user is greatly improved.
Drawings
Fig. 1 is a schematic flow chart illustrating an embodiment of a method for fast querying data according to the present invention;
fig. 2 is a schematic structural diagram of an embodiment of a data fast query apparatus according to the present invention;
fig. 3 is a block diagram of an embodiment of an electronic device provided in the present invention.
Detailed Description
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate preferred embodiments of the invention and together with the description, serve to explain the principles of the invention and not to limit the scope of the invention.
The invention provides a data quick query method, a data quick query system, electronic equipment and a computer readable storage medium, which are respectively described in detail below.
The embodiment of the invention provides a method for quickly querying data, a flow diagram of which is shown in fig. 1, and the method specifically comprises the following steps:
step S101, acquiring query information, and analyzing the query information to obtain query keywords;
step S102, judging whether the query keyword is in a pre-constructed hot word index or not;
step S103, when the query keyword is in the hot word index, obtaining target data from a cache database; when the query keyword is not in the hotword index, obtaining an associated keyword according to the query keyword and a preset keyword association table, obtaining initial return data from a dynamic database according to the associated keyword, and optimizing the initial return data to obtain target data;
and step S104, obtaining a query result according to the target data.
Compared with the prior art, the data quick query method provided by the embodiment comprises the steps of firstly, obtaining query information, and analyzing to obtain query keywords; secondly, judging whether the query keyword is in the hot word index; and finally, returning the target data from the database to obtain a query result. The method can improve the efficiency of parallel processing and query of the big data, can realize the real-time query of the business data under the background of large data volume, improves the overall business operation efficiency and reduces the operation and maintenance cost of enterprises; meanwhile, the service query requirements can be more comprehensively and accurately met, the query result is easy to look up, and the experience of the user is greatly improved.
As a preferred embodiment, in step S101, acquiring query information, and analyzing a query keyword of the query information, includes:
analyzing the query information according to a preset data format configuration table to obtain a standard query statement;
and obtaining query keywords according to the standard query statement.
As a specific embodiment, the obtaining of the query keyword according to the standard query statement includes: and performing word segmentation processing on the text information to obtain at least one keyword representing the text information.
As a preferred embodiment, in step S102, the constructing of the hotword index includes:
acquiring historical query data in a first preset time period;
obtaining query hot words with query frequency larger than a first threshold value according to the historical query data;
and obtaining the hot word index according to the query hot word.
By establishing the hot word index, the hot keywords frequently queried by the user can be summarized, and the query result of the hot keywords in a period of time is stored in the cache database for quick reading, so that the operating efficiency of the system is greatly improved, and the query waiting time of the user is reduced.
As a preferred embodiment, the method of the above technical solution further includes:
obtaining a query log according to the query keyword;
and updating the hot word index and the cache database according to the query log.
As a preferred embodiment, updating the hotword index and the cache database according to the query log includes:
acquiring a query log in a second preset time period, sorting query keywords in the query log according to query frequency, extracting the query keywords of which the query frequency is greater than a second threshold value, and obtaining updated query hot words;
updating the hot word index according to the updated query hot word to obtain an updated hot word index;
sorting the query keywords in the query log within a third preset time period according to query time, and extracting the query keywords of which the query time is later than a preset time threshold value to obtain recent query words;
and respectively acquiring the updated query hot words and the return data corresponding to the recent query words from the dynamic database, and storing the return data into a cache database.
As a specific embodiment, the updated query hot word and the return data corresponding to the recent query word are respectively obtained from the dynamic database, and the return data are stored in the cache database to cover the previous cache content. The results of the hot point query keywords and the latest query words are temporarily stored in the cache database through the query frequency and the query time of the query keywords, so that the query speed is increased; the query result is updated within the set time, so that the timeliness and the accuracy of the query result are ensured.
As a preferred embodiment, in step S103, the method for presetting the keyword association table includes:
acquiring at least one of a near meaning word, a synonym, an homophone and an English translation word of the keyword to obtain a relevant word of the keyword;
and establishing a mapping relation between the keywords and the associated words to obtain the keyword association table.
By carrying out multi-dimensional search on the keywords, the accuracy of the query result can be improved.
As a specific embodiment, when the dynamic database does not have the returned data of the query keyword, and the associated keyword of the query keyword does not have the returned data, the keyword is marked as a special keyword in the query log, and the query result returned to the query terminal does not exist. And downloading the return data corresponding to the special keywords from the cloud in a preset time period, and supplementing the return data into the dynamic database. In this way, the integrity of the database is guaranteed.
As a preferred embodiment, optimizing the initial return data to obtain target data includes:
and carrying out duplicate removal and sequencing on the initial return data to obtain target data.
As a specific embodiment, multiple same return data in the initial return data are deleted, and it is ensured that there is one and only one same return data. By carrying out deduplication on the returned data, redundant data in the returned data can be eliminated, and the use experience of a user is improved.
As a specific embodiment, obtaining a query result according to the target data includes: and packaging the target data according to a data format configuration table to obtain a query result, and returning the query result to the query terminal.
An embodiment of the present invention provides a data fast query apparatus, a block diagram of which is shown in fig. 2, where the data fast query system 200 includes:
an obtaining module 201, configured to obtain query information, and analyze the query information to obtain a query keyword;
a judging module 202, configured to judge whether the query keyword is in a pre-constructed hotword index;
the data retrieval module 203 is configured to obtain target data from a cache database when the query keyword is in the hotword index; when the query keyword is not in the hotword index, obtaining an associated keyword according to the query keyword and a preset keyword association table, obtaining initial return data from a dynamic database according to the associated keyword, and optimizing the initial return data to obtain target data;
and the response module 204 is configured to obtain a query result according to the target data.
As shown in fig. 3, the present invention further provides an electronic device 300, which may be a mobile terminal, a desktop computer, a notebook, a palmtop computer, a server, or other computing devices. The electronic device comprises a processor 301, a memory 302 and a display 303.
The storage 302 may be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device, in some embodiments. The memory 302 may also be an external storage device of the computer device in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. provided on the computer device. Further, the memory 302 may also include both internal storage units of the computer device and external storage devices. The memory 302 is used for storing application software installed on the computer device and various data, such as program codes for installing the computer device. The memory 302 may also be used to temporarily store data that has been output or is to be output. In an embodiment, the memory 302 stores a data fast query method program 304, and the data fast query method program 304 can be executed by the processor 301 to implement a data fast query method according to embodiments of the present invention.
The processor 301 may be, in some embodiments, a Central Processing Unit (CPU), microprocessor or other data Processing chip for executing program codes stored in the memory 302 or Processing data, such as executing a data fast query program.
The display 303 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch panel, or the like in some embodiments. The display 303 is used for displaying information at the computer device and for displaying a visualized user interface. The components 301 and 303 of the computer device communicate with each other via a system bus.
The embodiment also provides a computer-readable storage medium, on which a program of a data fast query method is stored, and when the processor executes the program, the data fast query method as described above is implemented.
According to the computer-readable storage medium and the computing device provided by the above embodiments of the present invention, the content specifically described for implementing the above-mentioned data fast query method according to the present invention can be referred to, and the beneficial effects similar to those of the above-mentioned data fast query method are achieved, and are not described herein again.
The invention discloses a method, a device, electronic equipment and a computer readable storage medium for quickly querying data, wherein query information is obtained and analyzed to obtain query keywords; secondly, judging whether the query keyword is in the hot word index; and finally, returning the target data from the database to obtain a query result.
The method can improve the efficiency of parallel processing and query of the big data, can realize the real-time query of the business data under the background of large data volume, improves the overall business operation efficiency and reduces the operation and maintenance cost of enterprises; meanwhile, the service query requirements can be more comprehensively and accurately met, the query result is easy to look up, and the experience of the user is greatly improved.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention.
Claims (10)
1. A method for quickly querying data is characterized by comprising the following steps:
acquiring query information, and analyzing the query information to obtain query keywords;
judging whether the query keyword is in a pre-constructed hot word index or not;
when the query keyword is in the hot word index, obtaining target data from a cache database; when the query keyword is not in the hotword index, obtaining an associated keyword according to the query keyword and a preset keyword association table, obtaining initial return data from a dynamic database according to the associated keyword, and optimizing the initial return data to obtain target data;
and obtaining a query result according to the target data.
2. The method for fast querying data according to claim 1, wherein obtaining query information and analyzing query keywords of the query information comprises:
analyzing the query information according to a preset data format configuration table to obtain a standard query statement;
and obtaining a query keyword according to the standard query statement.
3. The method for fast querying data according to claim 1, wherein the constructing of the hotword index comprises:
acquiring historical query data in a first preset time period;
obtaining query hot words with query frequency larger than a first threshold value according to the historical query data;
and obtaining the hot word index according to the query hot word.
4. The method for fast querying data according to claim 1, further comprising:
obtaining a query log according to the query keyword;
and updating the hot word index and the cache database according to the query log.
5. The method for fast querying data according to claim 4, wherein the updating the hotword index and the cache database according to the query log comprises:
acquiring a query log in a second preset time period, sorting query keywords in the query log according to query frequency, extracting the query keywords of which the query frequency is greater than a second threshold value, and obtaining updated query hot words;
updating the hot word index according to the updated query hot word to obtain an updated hot word index;
sorting the query keywords in the query log within a third preset time period according to query time, and extracting the query keywords of which the query time is later than a preset time threshold value to obtain recent query words;
and respectively acquiring the updated query hot words and the return data corresponding to the recent query words from the dynamic database, and storing the return data into a cache database.
6. The method for rapidly querying data according to claim 1, wherein the method for presetting the keyword association table comprises the following steps:
acquiring at least one of a near meaning word, a synonym, an homophone and an English translation word of the keyword to obtain a relevant word of the keyword;
and establishing a mapping relation between the keywords and the associated words to obtain the keyword association table.
7. The method for fast querying data according to claim 1, wherein optimizing the initial return data to obtain target data comprises:
and carrying out duplicate removal and sequencing on the initial return data to obtain target data.
8. A data fast query device is characterized by comprising:
the acquisition module is used for acquiring query information and analyzing the query information to obtain query keywords;
the judging module is used for judging whether the query keyword is in a pre-constructed hot word index or not;
the data calling module is used for obtaining target data from a cache database when the query keyword is in the hot word index; when the query keyword is not in the hotword index, obtaining an associated keyword according to the query keyword and a preset keyword association table, obtaining initial return data from a dynamic database according to the associated keyword, and optimizing the initial return data to obtain target data;
and the response module is used for obtaining a query result according to the target data.
9. An electronic device comprising a processor and a memory, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements a data fast query method according to any one of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, implements a method for fast data query according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210125784.5A CN114547095A (en) | 2022-02-10 | 2022-02-10 | Data rapid query method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210125784.5A CN114547095A (en) | 2022-02-10 | 2022-02-10 | Data rapid query method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114547095A true CN114547095A (en) | 2022-05-27 |
Family
ID=81673233
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210125784.5A Pending CN114547095A (en) | 2022-02-10 | 2022-02-10 | Data rapid query method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114547095A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115774740A (en) * | 2022-11-21 | 2023-03-10 | 本原数据(北京)信息技术有限公司 | Database snapshot query method, system and storage medium |
| CN115794892A (en) * | 2023-01-09 | 2023-03-14 | 北京创新乐知网络技术有限公司 | Search method, device, equipment and medium based on hierarchical cache |
| CN116010472A (en) * | 2023-03-28 | 2023-04-25 | 山东浪潮科学研究院有限公司 | Data query method, device, equipment and storage medium |
-
2022
- 2022-02-10 CN CN202210125784.5A patent/CN114547095A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115774740A (en) * | 2022-11-21 | 2023-03-10 | 本原数据(北京)信息技术有限公司 | Database snapshot query method, system and storage medium |
| CN115774740B (en) * | 2022-11-21 | 2023-09-12 | 本原数据(北京)信息技术有限公司 | Database snapshot query method, system and storage medium |
| CN115794892A (en) * | 2023-01-09 | 2023-03-14 | 北京创新乐知网络技术有限公司 | Search method, device, equipment and medium based on hierarchical cache |
| CN116010472A (en) * | 2023-03-28 | 2023-04-25 | 山东浪潮科学研究院有限公司 | Data query method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107679144B (en) | News sentence clustering method and device based on semantic similarity and storage medium | |
| CN114547095A (en) | Data rapid query method and device, electronic equipment and storage medium | |
| CN108932294B (en) | Resume data processing method, device, equipment and storage medium based on index | |
| US8903798B2 (en) | Real-time annotation and enrichment of captured video | |
| US9626440B2 (en) | Tenantization of search result ranking | |
| CN109074383B (en) | Document search with visualization within the context of a document | |
| CN109800218B (en) | Distributed storage system, storage node device and data deduplication method | |
| US20130191414A1 (en) | Method and apparatus for performing a data search on multiple user devices | |
| CN112637305B (en) | Data storage and query method, device, equipment and medium based on cache | |
| CN110532347B (en) | Log data processing method, device, equipment and storage medium | |
| US20160063107A1 (en) | Data retrieval via a telecommunication network | |
| US8805848B2 (en) | Systems, methods and computer program products for fast and scalable proximal search for search queries | |
| CN113407785B (en) | Data processing method and system based on distributed storage system | |
| CN110990406B (en) | Fuzzy query method, device, equipment and medium | |
| US8548989B2 (en) | Querying documents using search terms | |
| CN111400323A (en) | Data retrieval method, system, device and storage medium | |
| CN108763524B (en) | Electronic device, chatting data processing method, and computer-readable storage medium | |
| CN111858581B (en) | Paging query method and device, storage medium and electronic equipment | |
| CN115080684B (en) | Network disk document indexing method and device, network disk and storage medium | |
| CN103809915A (en) | Read-write method and device of magnetic disk files | |
| CN114064729A (en) | Data retrieval method, device, equipment and storage medium | |
| CN113722296A (en) | Agricultural information processing method and device, electronic equipment and storage medium | |
| EP2990960A1 (en) | Data retrieval via a telecommunication network | |
| CN111782958A (en) | Recommendation word determining method and device, electronic device and storage medium | |
| CN111221817B (en) | Service information data storage method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |