CN113971238A - Data retrieval method and device, electronic equipment and storage medium - Google Patents
Data retrieval method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113971238A CN113971238A CN202111299034.1A CN202111299034A CN113971238A CN 113971238 A CN113971238 A CN 113971238A CN 202111299034 A CN202111299034 A CN 202111299034A CN 113971238 A CN113971238 A CN 113971238A
- Authority
- CN
- China
- Prior art keywords
- retrieval
- page data
- data
- page
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
- G06F16/9574—Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention provides a data retrieval method and a related device, wherein the method comprises the following steps: when receiving retrieval information sent by front-end equipment, generating a retrieval ID by using the retrieval information; searching page data with a preset page number from an Elasticissearch search engine by using the searching information; the page data comprises a preset number of retrieval data; establishing a corresponding relation between the page data and the retrieval ID, storing the corresponding relation to cache equipment, and sending the first page data and the retrieval ID in the page data to front-end equipment; when receiving a retrieval ID to be queried and a target page number sent by front-end equipment, querying corresponding target page data in cache equipment according to the retrieval ID to be queried and the target page number, and sending the target page data to the front-end equipment; and the target page number is calculated according to the preset number. The invention can acquire the page data of a plurality of pages from the search engine and carry out identification and caching, is convenient for the front end to acquire the page data from the cache through the identification, and further improves the retrieval efficiency.
Description
Technical Field
The present invention relates to the field of network technologies, and in particular, to a data retrieval method and apparatus, an electronic device, and a storage medium.
Background
The Elasticsearch is a distributed full-text search engine. In the related art, when a user performs page turning or repeated retrieval, the front-end device needs to repeatedly send retrieval information to the Elasticsearch engine located at the rear end, so that the engine repeatedly retrieves data and returns the data to the front-end device, however, these additional data retrieval operations easily reduce the processing efficiency of the Elasticsearch engine and increase the response time, thereby increasing the time for the user to wait for retrieving data and reducing the user experience of the Elasticsearch engine.
Disclosure of Invention
The invention aims to provide a data retrieval method, a data retrieval device, electronic equipment and a storage medium, which can acquire page data of a plurality of pages from a search engine at one time and carry out identification and caching, so that a front end can conveniently acquire the page data from the caching through identification, and the efficiency of front end information retrieval is further improved.
In order to solve the above technical problem, the present invention provides a data retrieval method, including:
when receiving retrieval information sent by front-end equipment, generating a retrieval ID by using the retrieval information;
searching page data with a preset page number from an Elasticissearch engine by using the search information; the page data comprises a preset number of retrieval data;
establishing a corresponding relation between the page data and the retrieval ID, storing the corresponding relation to a cache device, and sending the first page data and the retrieval ID in the page data to the front-end device;
when receiving a retrieval ID to be queried and a target page number sent by the front-end equipment, querying corresponding target page data in the cache equipment according to the retrieval ID to be queried and the target page number, and sending the target page data to the front-end equipment; and the target page number is obtained by calculation according to the preset number.
Optionally, after generating a search ID by using the search information, the method further includes:
and matching in the cache device according to the retrieval ID, and sending the first page data in the page data corresponding to the matched target retrieval ID to the front-end device.
Optionally, after sending the first page data of the page data and the search ID to the front-end device, the method further includes:
and transmitting the rest page data in the page data to the front-end equipment in an asynchronous transmission mode so that the front-end equipment caches the rest page data locally and reads the rest page data locally.
Optionally, the retrieving, by using the retrieval information, page data of a preset number of pages to an Elasticsearch engine includes:
utilizing the retrieval information to simultaneously retrieve the page data from a main cluster and a standby cluster of the Elasticissearch engine, and respectively recording first response time and second response time of the main cluster and the standby cluster returning the page data;
if the first response time and the second response time are both smaller than preset time, utilizing page data returned by the main cluster to execute the steps of establishing a corresponding relation between the page data and the retrieval ID and storing the page data and the retrieval ID in a cache device;
and if the first response time or the second response time is not less than the preset time, using the earliest returned page data to execute the steps of establishing the corresponding relation between the page data and the retrieval ID and storing the page data and the retrieval ID in a cache device.
Optionally, the method further comprises:
the backup cluster periodically synchronizes data to the primary cluster.
Optionally, the cache device is a Redis database.
Optionally, the retrieving, by using the retrieval information, page data of a preset number of pages to an Elasticsearch engine includes:
retrieving the first page data from the Elasticissearch search engine in a normal query mode by using the retrieval information;
and retrieving the rest page data in the page data one by one from the Elasticissearch search engine in an asynchronous query mode by utilizing the retrieval information.
The present invention also provides a data retrieval apparatus comprising:
the retrieval ID generation module is used for generating a retrieval ID by utilizing the retrieval information when receiving the retrieval information sent by the front-end equipment;
the first retrieval module is used for retrieving page data with a preset page number from an Elasticissearch search engine by utilizing the retrieval information; the page data comprises a preset number of retrieval data;
the cache module is used for establishing a corresponding relation between the page data and the retrieval ID, storing the corresponding relation to cache equipment, and sending the first page data in the page data and the retrieval ID to the front-end equipment;
the second retrieval module is used for querying corresponding target page data in the cache device according to the retrieval ID to be queried and the target page number when receiving the retrieval ID to be queried and the target page number sent by the front-end device, and sending the target page data to the front-end device; and the target page number is obtained by calculation according to the preset number.
The present invention also provides an electronic device comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data retrieval method as described above when executing the computer program.
The invention also provides a storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of the data retrieval method as described above.
The invention provides a data retrieval method, which comprises the following steps: when receiving retrieval information sent by front-end equipment, generating a retrieval ID by using the retrieval information; searching page data with a preset page number from an Elasticissearch engine by using the search information; the page data comprises a preset number of retrieval data; establishing a corresponding relation between the page data and the retrieval ID, storing the corresponding relation to a cache device, and sending the first page data and the retrieval ID in the page data to the front-end device; when receiving a retrieval ID to be queried and a target page number sent by the front-end equipment, querying corresponding target page data in the cache equipment according to the retrieval ID to be queried and the target page number, and sending the target page data to the front-end equipment; and the target page number is obtained by calculation according to the preset number.
Therefore, when the retrieval information sent by the front-end equipment is received, firstly, a retrieval ID is generated by using the retrieval information, and page data with a preset page number is retrieved to an Elasticissearch engine according to the retrieval information; after the page data is obtained, the retrieval ID and the page data are established in a corresponding relation and cached, in other words, when the retrieval information is received, a batch of page data is obtained from the Elasticissearch engine, and the batch of page data is identified and cached, so that when the front-end equipment performs page turning or repeated retrieval, the front-end equipment can directly obtain the target page data from the cache equipment through the identification information of the retrieval ID, the frequency of sending the retrieval information to the Elasticissearch engine by the front-end equipment can be effectively reduced, the burden of the Elasticissearch engine on repeated inquiry of the front-end equipment can be reduced, the speed of the front-end equipment responding by the rear end can be increased, the waiting time of a user can be reduced, and the use experience of the Elasticissearch engine used by the user can be effectively improved. The invention also provides a data retrieval device, electronic equipment and a storage medium, which have the beneficial effects.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a data retrieval method according to an embodiment of the present invention;
fig. 2 is a block diagram of a data retrieval apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The Elasticsearch is a distributed full-text search engine. In the related art, when a user performs page turning or repeated retrieval, the front-end device needs to repeatedly send retrieval information to the Elasticsearch engine located at the rear end, so that the engine repeatedly retrieves data and returns the data to the front-end device, however, these additional data retrieval operations easily reduce the processing efficiency of the Elasticsearch engine and increase the response time, thereby increasing the time for the user to wait for retrieving data and reducing the user experience of the Elasticsearch engine. In view of this, the present invention provides a data retrieval method, which can acquire page data of multiple pages from a search engine at one time, and perform identification and caching, so that a front end can conveniently acquire the page data from the cache through the identification, thereby improving the efficiency of front-end information retrieval. Referring to fig. 1, fig. 1 is a flowchart of a data retrieval method according to an embodiment of the present invention, where the method includes:
s101, when receiving retrieval information sent by front-end equipment, generating a retrieval ID by using the retrieval information;
in the embodiment of the invention, the retrieval information is information required by the Elasticissearch retrieval engine for retrieval. The embodiment of the present invention does not limit the specific content of the search information, and may include, for example, a search keyword, a query condition, a selection item related to the search, a search time range, and the like, and may be selected according to the actual application requirements.
Further, the embodiment of the invention can acquire a large amount of retrieval data from the Elasticissearch search engine at one time and cache the retrieval data in a batch acquisition mode so as to reduce the times of retrieving the data from the search engine. In order to uniformly manage the retrieved data, the data needs to be labeled. In the embodiment of the present invention, the search information is used to label the search data, considering that the data are obtained by searching the search information, that is, the data carry the characteristic information of the search information. Specifically, a search ID (searchid) is generated using the search information, and the search data is labeled with the search ID. The embodiment of the present invention does not limit how to generate the search ID, for example, generating a unique digital ID for the search information; of course, the character strings corresponding to the search information may also be spliced, the spliced character strings are converted into MD5 values, and then the MD5 value is set as the search ID, where MD5(Message-Digest Algorithm) is an information Digest Algorithm, and the Algorithm may generate a hash value (hash value) for ensuring that the information transmission is complete and consistent. In the embodiment of the present invention, in consideration of the MD5 value being easy to calculate and restore, the search ID may be generated by generating the MD5 value as described above.
S102, retrieving page data with a preset page number from an Elasticissearch engine by using retrieval information; the page data includes a predetermined number of pieces of search data.
In the embodiment of the present invention, when receiving the retrieval information sent by the front-end device, the method obtains a plurality of pages of page data from the Elasticsearch engine, where each page of page data includes a preset number of pieces of retrieval data. The embodiment of the invention does not limit the specific numerical values of the preset pages and the preset strips, and can be set according to the actual application requirements. Furthermore, the embodiment of the invention does not limit whether different modes are needed to be used for distinguishing and acquiring the page data, and when a single acquisition mode is adopted, the efficiency of returning the page data to the front-end equipment can be ensured, the single mode can be adopted for acquiring; when the efficiency of returning the page data to the front-end device needs to be improved, different page data can be acquired in different modes. In the embodiment of the present invention, it is considered that the first page data in the page data is page data that needs to be displayed urgently by the front end device and needs to be returned as soon as possible, and the remaining page data is cache data and does not need to be returned to the front end, or can be returned to the front end at a lower speed, that is, the first page data and the remaining page data in the page data have different requirements on the return speed, and it is difficult to ensure the acquisition efficiency by using a single acquisition mode, so different page data can be acquired by using different acquisition modes. Specifically, because the first page data needs to be returned to the front-end device as soon as possible, the first page data can be acquired from the Elasticsearch engine by adopting a normal query mode (for example, a synchronous query mode); and the rest page data can be returned at a lower speed or does not need to be returned, so that the rest page data can be retrieved to an Elasticissearch search engine in a slower asynchronous query mode.
In one possible case, the retrieving the page data of the preset number of pages from the Elasticsearch engine by using the retrieval information may include:
step 11: and utilizing the retrieval information to retrieve the first page data from the Elasticissearch search engine in a normal query mode.
In order to improve the speed of returning data to the front-end equipment, the first page data can be immediately returned to the front-end equipment when the first page data is obtained, and the retrieval of the rest page data is not required to be finished.
Step 12: and retrieving the rest page data in the page data one by one from an Elasticissearch search engine in an asynchronous query mode by utilizing the retrieval information.
It should be noted that the embodiment of the present invention does not limit how to retrieve data from the Elasticsearch engine by using a normal query manner and an asynchronous query manner, and reference may be made to the related technology of Elasticsearch. It should be noted that the number of asynchronous queries is the same as the number of pages of the remaining page data, for example, 9 remaining pages, 9 asynchronous queries are initiated to the Elasticsearch search engine.
S103, establishing a corresponding relation between the page data and the retrieval ID, storing the corresponding relation to the cache equipment, and sending the first page data and the retrieval ID in the page data to the front-end equipment.
In the embodiment of the invention, in order to avoid repeatedly retrieving data from the elastic search engine, the retrieved data obtained by batch retrieval can be stored in the cache device, and the retrieval ID of the batch is sent to the front-end device, so that the front-end device can quickly acquire data from the cache device by using the retrieval ID when page turning or repeated retrieval is carried out, and further response delay can be reduced and response speed can be increased. It should be noted that the embodiments of the present invention are not limited to specific cache devices, and refer to the related art of databases. In a possible case, in consideration of the simple structure and fast query speed of the Redis database, the Redis database is a key-value storage system, and therefore can be used as a cache device.
It should be noted that the embodiment of the present invention does not limit how many groups of search IDs and page data can be stored in the cache device, and can be set according to the actual application requirements. Of course, the caching device may set an expiration mechanism for the retrieval ID and the page data to ensure that the page data has a higher timeliness. The embodiment of the present invention does not limit the specific expiration mechanism, and for example, refer to the related art of the Redis expiration policy.
Further, in consideration that the front-end device can further improve response efficiency by reading the page data from the local cache, after the first page data is sent to the front-end device, the remaining page data can also be sent to the front-end device for caching, so that the front-end device can read the remaining page data through the local cache. Of course, the remaining page data may be sent to the front-end device by slow transmission, for example, by asynchronous transmission. It should be noted that, the embodiment of the present invention does not limit the specific manner of asynchronous transmission, and reference may be made to the related art of asynchronous transmission. The embodiment of the invention also does not limit how the front-end equipment caches the page data, and can refer to the related technology of the browser.
In one possible case, after sending the first page data of the page data and the search ID to the front-end device, the method may further include:
step 21: and transmitting the rest page data in the page data to the front-end equipment in an asynchronous transmission mode so that the front-end equipment caches the rest page data locally and reads the rest page data locally.
Of course, besides returning the remaining page data, the back end may also return the search ID to the front end device at the same time, so that the front end device can confirm that the received page data belongs to the batch corresponding to the search ID, and can perform caching according to the search ID.
S104, when receiving the retrieval ID to be queried and the target page number sent by the front-end equipment, querying corresponding target page data in the cache equipment according to the retrieval ID to be queried and the target page number, and sending the target page data to the front-end equipment; and the target page number is calculated according to the preset number.
In the embodiment of the invention, because the back end caches the page data acquired according to the batch and marks the retrieval ID, the front-end equipment can directly acquire the required target page data from the back end through the retrieval ID. Specifically, the front-end device may obtain the target page data from the back-end by sending two parameters, i.e., the search ID to be queried and the target page number, to the back-end, where the target page number is calculated by a preset number of search data included in each page data. Specifically, since the page data is usually obtained from the Elasticsearch engine in sequence, the page data has a sequence, and further, the retrieval data among the pages also has a sequence, that is, a sequence number can be set for the retrieval data. For example, for ten pages of page data, each page of page data has 10 pieces of search data, so serial numbers 0 to 9 can be set for the search data of the first page of page data, serial numbers 10 to 19 can be set for the search data of the second page of page data, and so on. When the front-end equipment acquires the page data, the label of the first retrieval data in the target page data can be calculated by utilizing the label of the first retrieval data in the currently output page data and the preset number, and then the label of the first retrieval data in the target page data is used as the target page number. For example, in the above example, if the currently output page of the front-end device is the first page data, and the page to be acquired is the second page data, the front-end device may calculate the serial number 10 of the first search data of the second page data by using the serial number 0 and the preset number 10 of the first search data of the first page data, and then acquire the second page data from the back-end according to the serial number 10. Of course, the front-end device may also send the preset number to the back-end, so that the back-end performs the check calculation, specifically, the front-end device may send parameters from and pageSize, where from is the target page number, and pageSize is the preset number of the search data included in each page data.
Further, besides returning the target page data, the back end can also return the retrieval ID to the front end device at the same time, so that the front end device can confirm that the target page data belongs to the batch corresponding to the retrieval ID. Of course, if the target page data searched by the front-end device is not stored in the cache device, null information may also be returned to the front-end device, and when the front-end device fails to receive the retrieval ID, the retrieval information will request the Elasticsearch engine for the retrieval data in a normal manner.
Based on the embodiment, when receiving retrieval information sent by the front-end device, the invention firstly generates a retrieval ID by using the retrieval information, and retrieves page data with a preset page number to an Elasticissearch engine according to the retrieval information; after the page data is obtained, the retrieval ID and the page data are established in a corresponding relation and cached, in other words, when the retrieval information is received, a batch of page data is obtained from the Elasticissearch engine, and the batch of page data is identified and cached, so that when the front-end equipment performs page turning or repeated retrieval, the front-end equipment can directly obtain the target page data from the cache equipment through the identification information of the retrieval ID, the frequency of sending the retrieval information to the Elasticissearch engine by the front-end equipment can be effectively reduced, the burden of the Elasticissearch engine on repeated inquiry of the front-end equipment can be reduced, the speed of the front-end equipment responding by the rear end can be increased, the waiting time of a user can be reduced, and the use experience of the Elasticissearch engine used by the user can be effectively improved.
Based on the above embodiment, it is considered that the cache device already stores the page data of the history batch, so when the retrieval information sent by the front-end device is converted into the retrieval ID, the page data can also be directly searched for from the cache device according to the retrieval ID. In one possible case, after generating the search ID using the search information, the method may further include:
s201, matching is carried out in the cache device according to the retrieval ID, and the first page data in the page data corresponding to the matched target retrieval ID is sent to the front-end device.
Therefore, the efficiency of returning data to the front-end equipment can be effectively improved. Of course, the flexibility of the front-end device for inquiring information can also be increased, for example, when the front-end device turns pages, the front-end device can inquire target page data to the back end according to the retrieval ID to be inquired; when the retrieval is carried out again, the retrieval information is directly sent to the back end, and the back end generates the corresponding retrieval ID and carries out matching, so that the processing efficiency of the front end can be improved.
Based on the embodiment, considering that the working condition of the Elasticissearch search engine is easily influenced by the request quantity and the network condition, and further the condition that the data is not returned by the search engine is easily caused when the data is searched for the search engine, in the embodiment of the invention, a main cluster and a standby cluster are deployed for the Elasticissearch engine, the search redundancy is set for the search engine, and the problem of query failure caused by the failure of the main cluster is avoided. In one possible case, the retrieving the page data of the preset number of pages from the Elasticsearch engine by using the retrieval information may include:
s301, retrieving information is utilized to retrieve page data from a main cluster and a standby cluster of the Elasticissearch engine at the same time, and first response time and second response time of the page data returned by the main cluster and the standby cluster are recorded respectively.
S302, if the first response time and the second response time are both smaller than the preset time, the page data returned by the main cluster is utilized to execute the steps of establishing the corresponding relation between the page data and the retrieval ID and storing the page data and the retrieval ID in the cache device.
If the first response time and the second response time are both smaller than the preset time, it is indicated that both the two clusters can respond to the data in time, and no abnormal condition exists, and at this time, the page data returned by the main cluster can be utilized to perform subsequent operations. It should be noted that, the embodiment of the present invention is not limited to the specific value of the preset time, and the preset time may be set according to the actual application requirement, for example, may be 1 second.
And S303, if the first response time or the second response time is not less than the preset time, establishing a corresponding relation between the page data and the search ID and storing the page data and the search ID in the cache device by using the earliest returned page data.
If the first response time or the second response time is not less than the preset time, it indicates that the main cluster or the standby cluster is abnormal, and at this time, the earliest returned page data can be used for performing subsequent operations.
Further, to ensure that the data in the standby cluster remains updated, the standby cluster may be controlled to periodically synchronize data with the primary cluster. It should be noted that, the embodiment of the present invention does not limit the specific value of the synchronization period, and may be set according to the actual application requirement, for example, half an hour; the embodiment of the present invention also does not limit the specific way of synchronizing data, and reference may be made to the related art of elastic search.
In one possible case, the method may further include:
step 31: the standby cluster periodically synchronizes data to the primary cluster.
Based on the embodiment, the method and the device can deploy the main cluster and the standby cluster for the Elasticissearch search engine, set retrieval redundancy for the search engine, and avoid the problem of query failure caused by failure of the main cluster or the standby cluster.
The following describes a data retrieval device, an electronic device, and a storage medium according to embodiments of the present invention, and the data retrieval device, the electronic device, and the storage medium described below and the data retrieval method described above may be referred to correspondingly.
Referring to fig. 2, fig. 2 is a block diagram of a data retrieving device according to an embodiment of the present invention, the data retrieving device includes:
a retrieval ID generation module 201, configured to generate a retrieval ID by using the retrieval information when receiving the retrieval information sent by the front-end device;
a first retrieving module 202, configured to retrieve, by using the retrieval information, page data with a preset number of pages to an Elasticsearch engine; the page data comprises a preset number of retrieval data;
the cache module 203 is configured to establish a corresponding relationship between the page data and the search ID, store the corresponding relationship to a cache device, and send the first page data and the search ID in the page data to the front-end device;
the second retrieval module 204 is configured to, when receiving a retrieval ID to be queried and a target page number sent by the front-end device, query, according to the retrieval ID to be queried and the target page number, corresponding target page data in the cache device, and send the target page data to the front-end device; and the target page number is calculated according to the preset number.
Optionally, the apparatus may further include:
and the matching module is used for matching in the cache device according to the retrieval ID and sending the first page data in the page data corresponding to the matched target retrieval ID to the front-end device.
Optionally, the apparatus may further include:
and the asynchronous transmission module is used for transmitting the remaining page data in the page data to the front-end equipment in an asynchronous transmission mode so that the front-end equipment caches the remaining page data locally and reads the remaining page data locally.
Optionally, the first retrieving module 202 includes:
the first retrieval submodule is used for simultaneously retrieving page data from a main cluster and a standby cluster of the Elasticissearch search engine by utilizing retrieval information and respectively recording first response time and second response time of the page data returned by the main cluster and the standby cluster;
the first processing submodule is used for establishing a corresponding relation between the page data and the retrieval ID and storing the page data and the retrieval ID in the cache device by utilizing the page data returned by the main cluster if the first response time and the second response time are both smaller than the preset time;
and the second processing submodule is used for establishing a corresponding relation between the page data and the retrieval ID and storing the page data and the retrieval ID in the cache device by using the earliest returned page data if the first response time or the second response time is not less than the preset time.
Optionally, the apparatus may further include:
and the synchronization module is used for periodically synchronizing data to the main cluster by the standby cluster.
Optionally, the cache module 203 is further configured to establish a corresponding relationship between the page data and the search ID, and store the page data and the search ID in a Redis database.
Optionally, the first retrieving module 202 includes:
the second retrieval submodule is used for retrieving the first page data from the Elasticissearch search engine in a normal query mode by utilizing the retrieval information;
and the third retrieval submodule is used for retrieving the rest page data in the page data from the Elasticissearch search engine one by one in an asynchronous query mode by utilizing the retrieval information.
An embodiment of the present invention further provides an electronic device, including:
a memory for storing a computer program;
a processor for implementing the steps of the data retrieval method as described above when executing the computer program.
Since the embodiment of the electronic device portion corresponds to the embodiment of the data retrieval method portion, please refer to the description of the embodiment of the data retrieval method portion for the embodiment of the electronic device portion, and details are not repeated here.
The embodiment of the present invention further provides a storage medium, where a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the data retrieval method according to any of the above embodiments are implemented.
Since the embodiment of the storage medium portion and the embodiment of the data retrieval method portion correspond to each other, please refer to the description of the embodiment of the data retrieval method portion for the embodiment of the storage medium portion, which is not repeated here.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The data retrieval method, the data retrieval device, the electronic device and the storage medium provided by the invention are described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (10)
1. A method of data retrieval, comprising:
when receiving retrieval information sent by front-end equipment, generating a retrieval ID by using the retrieval information;
searching page data with a preset page number from an Elasticissearch engine by using the search information; the page data comprises a preset number of retrieval data;
establishing a corresponding relation between the page data and the retrieval ID, storing the corresponding relation to a cache device, and sending the first page data and the retrieval ID in the page data to the front-end device;
when receiving a retrieval ID to be queried and a target page number sent by the front-end equipment, querying corresponding target page data in the cache equipment according to the retrieval ID to be queried and the target page number, and sending the target page data to the front-end equipment; and the target page number is obtained by calculation according to the preset number.
2. The data retrieval method according to claim 1, further comprising, after generating a retrieval ID using the retrieval information:
and matching in the cache device according to the retrieval ID, and sending the first page data in the page data corresponding to the matched target retrieval ID to the front-end device.
3. The data retrieval method according to claim 1, further comprising, after transmitting the first page data of the page data and the retrieval ID to the front-end device:
and transmitting the rest page data in the page data to the front-end equipment in an asynchronous transmission mode so that the front-end equipment caches the rest page data locally and reads the rest page data locally.
4. The data retrieval method of claim 1, wherein the retrieving page data of a preset number of pages to an Elasticsearch engine by using the retrieval information comprises:
utilizing the retrieval information to simultaneously retrieve the page data from a main cluster and a standby cluster of the Elasticissearch engine, and respectively recording first response time and second response time of the main cluster and the standby cluster returning the page data;
if the first response time and the second response time are both smaller than preset time, utilizing page data returned by the main cluster to execute the steps of establishing a corresponding relation between the page data and the retrieval ID and storing the page data and the retrieval ID in a cache device;
and if the first response time or the second response time is not less than the preset time, using the earliest returned page data to execute the steps of establishing the corresponding relation between the page data and the retrieval ID and storing the page data and the retrieval ID in a cache device.
5. The data retrieval method of claim 4, further comprising:
the backup cluster periodically synchronizes data to the primary cluster.
6. The data retrieval method of claim 1, wherein the caching device is a Redis database.
7. The data retrieval method according to any one of claims 1 to 6, wherein the retrieving page data of a preset number of pages to an Elasticissearch search engine by using the retrieval information includes:
retrieving the first page data from the Elasticissearch search engine in a normal query mode by using the retrieval information;
and retrieving the rest page data in the page data one by one from the Elasticissearch search engine in an asynchronous query mode by utilizing the retrieval information.
8. A data retrieval device, comprising:
the retrieval ID generation module is used for generating a retrieval ID by utilizing the retrieval information when receiving the retrieval information sent by the front-end equipment;
the first retrieval module is used for retrieving page data with a preset page number from an Elasticissearch search engine by utilizing the retrieval information; the page data comprises a preset number of retrieval data;
the cache module is used for establishing a corresponding relation between the page data and the retrieval ID, storing the corresponding relation to cache equipment, and sending the first page data in the page data and the retrieval ID to the front-end equipment;
the second retrieval module is used for querying corresponding target page data in the cache device according to the retrieval ID to be queried and the target page number when receiving the retrieval ID to be queried and the target page number sent by the front-end device, and sending the target page data to the front-end device; and the target page number is obtained by calculation according to the preset number.
9. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data retrieval method as claimed in any one of claims 1 to 7 when executing the computer program.
10. A storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the data retrieval method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111299034.1A CN113971238A (en) | 2021-11-04 | 2021-11-04 | Data retrieval method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111299034.1A CN113971238A (en) | 2021-11-04 | 2021-11-04 | Data retrieval method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113971238A true CN113971238A (en) | 2022-01-25 |
Family
ID=79589708
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111299034.1A Pending CN113971238A (en) | 2021-11-04 | 2021-11-04 | Data retrieval method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113971238A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115203159A (en) * | 2022-07-25 | 2022-10-18 | 北京字跳网络技术有限公司 | Data storage method and device, computer equipment and storage medium |
-
2021
- 2021-11-04 CN CN202111299034.1A patent/CN113971238A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115203159A (en) * | 2022-07-25 | 2022-10-18 | 北京字跳网络技术有限公司 | Data storage method and device, computer equipment and storage medium |
| CN115203159B (en) * | 2022-07-25 | 2024-06-04 | 北京字跳网络技术有限公司 | Data storage method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102542052B (en) | Priority hash index | |
| CN101390080B (en) | Serving cached query results based on a query portion | |
| CN105224546B (en) | Data storage and query method and equipment | |
| US7840547B1 (en) | Methods and systems for efficient query rewriting | |
| US11567681B2 (en) | Method and system for synchronizing requests related to key-value storage having different portions | |
| US11294920B2 (en) | Method and apparatus for accessing time series data in memory | |
| CN105740472A (en) | Distributed real-time full-text search method and system | |
| WO2007068600B1 (en) | Generating backup sets to a specific point in time | |
| CN108694188B (en) | Index data updating method and related device | |
| CN109766318B (en) | File reading method and device | |
| US11748357B2 (en) | Method and system for searching a key-value storage | |
| CN103353901A (en) | Orderly table data management method and system based on Hadoop distributed file system (HDFS) | |
| US20170270149A1 (en) | Database systems with re-ordered replicas and methods of accessing and backing up databases | |
| CN109254981B (en) | Data management method and device of distributed cache system | |
| CN105447166A (en) | Keyword based information search method and system | |
| CN108038253B (en) | Log query processing method and device | |
| CN111046041A (en) | Data processing method and device, storage medium and processor | |
| CN113971238A (en) | Data retrieval method and device, electronic equipment and storage medium | |
| CN114281819A (en) | Data query method, device, equipment and storage medium | |
| CN114398520A (en) | Data retrieval method, system, device, electronic equipment and storage medium | |
| CN105574010B (en) | Data query method and device | |
| CN110798222B (en) | Data compression method and device | |
| CN110413724B (en) | Data retrieval method and device | |
| CN110765125B (en) | Method and device for storing data | |
| CN103646034A (en) | Web search engine system and search method based content credibility |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |