CN109325047B - Interactive elastic search deep paging query method and device - Google Patents
Interactive elastic search deep paging query method and device Download PDFInfo
- Publication number
- CN109325047B CN109325047B CN201811395531.XA CN201811395531A CN109325047B CN 109325047 B CN109325047 B CN 109325047B CN 201811395531 A CN201811395531 A CN 201811395531A CN 109325047 B CN109325047 B CN 109325047B
- Authority
- CN
- China
- Prior art keywords
- query
- field
- user
- paging
- elasticsearch
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides an interactive ElasticSearch deep paging query method, which is characterized by comprising the following steps: receiving a query request sent by a client; judging whether the query request is a first query request or not and is smaller than the paging threshold, if not, performing an interactive query mode, calculating data of exhaustively classified fields except the query field by aggregation, and sending the data to an ElasticSearch server according to a refined field combination query condition selected by a user; and encapsulating the result data returned by the ElasticSearch server and returning the result data to the client. The invention can dynamically recommend and utilize an interactive query mechanism in real time, can dynamically recommend the query conditions in real time to interact with the user, provides selection for the user, and ensures that the query is more friendly and efficient.
Description
Technical Field
The invention relates to the field of data processing, in particular to an interactive ElasticSearch deep paging query method and electronic equipment.
Background
Queries are one of the most common operations of search servers. Wherein, the ElasticSearch is a search server based on Lucene. It provides a distributed multi-user capability full-text search engine. The Elasticisearch is developed by Java, is released as an open source code under Apache licensing terms, and is a currently popular enterprise-level search engine; the ElasticSearch is document oriented, which means that it can store the entire object or document (document). However, it not only stores, but indexes (indexes) the content of each document so that it can be searched. In ElasticSearch you can index, search, sort, filter documents (rather than rows and columns of data).
However, because the cost and expense of result sorting in a distributed system multiply with the depth of paging (deep paging), the elatic search as a high-performance memory database is the most concerned technical problem of the elatic search database in how to compute and analyze further query operations, especially deep paging queries, according to the previous query results.
Disclosure of Invention
Aiming at the technical problems, particularly when the result of database query is subjected to targeted calculation, a corresponding query mechanism is lacked, so that the interactive query is formed by adding interactive operation on the basis of the original query, and the user can be better helped to acquire the most needed information from the database through accurate and complete retrieval conditions through the interactive query. Specifically, according to the query result, aggregation calculation analysis is recommended to the user, all types of the fields can be aggregated according to the exhaustively classified fields except the non-query condition, the aggregated fields are converted into descriptions which can be understood by the user and returned to the page, the user selects the classification which needs to be refined, the combined query condition is sent to the Web server for data query, the number of matched documents is reduced by further refining the query by increasing the query refining condition, and therefore the problem that the cost is too high in the ElasticSearch deep paging query is effectively solved.
In order to achieve the purpose, the invention provides a method and electronic equipment based on interactive ElasticSearch deep paging query. Specifically, the following technical scheme is adopted:
in a first aspect, the present invention provides a method for interactive ElasticSearch deep paging query, comprising the following steps:
step 1: receiving a query request sent by a client;
step 2: judging whether the query request is a first query request, if so, turning to the step 4, otherwise, turning to the step 3;
and step 3: judging the relation between the paging parameters of the query request and a paging threshold, if the relation is smaller than the paging threshold, turning to the step 4, and otherwise, turning to the step 5;
and 4, step 4: sending the query request to an ElasticSearch, and turning to step 6;
and 5: aggregating and calculating data of exhaustively classified fields except the query field, and sending the data to an ElasticSearch server according to a refined field combination query condition selected by a user;
step 6: encapsulating result data returned by the ElasticSearch server;
and 7: and returning the result data to the client.
In the above technical solution, further, the step 5 further includes the following steps:
step 5.1: aggregating data of fields except the calculation inquiry field;
step 5.2: prompting a user whether to refine the query;
step 5.3, when the user needs to refine the query, pushing and displaying the refined field of the aggregation calculation result to the client;
step 5.4: selecting a corresponding detailed field by a user;
step 5.5: according to all specific detailed classifications corresponding to the field returned by the detailed field, aggregating all types of the field, converting the types of the field into descriptions which can be understood by the user, and returning the descriptions to the page;
step 5.6: selecting a corresponding detailed classification by a user;
step 5.7: and according to the classification which needs to be refined and selected by the user, combining the query conditions and sending the combined query conditions to an ElasticSearch server.
In the above technical solution, further, the fields other than the query field are exhaustively classified fields.
In the technical solution, the polymerization calculation in step 5.1 further includes the following steps:
step 5.1.1: analyzing the relation of fields except the query field;
step 5.1.2: and aggregating the data of the fields with the association relationship.
In the above technical solution, the client is a browser.
In the above technical solution, the client is a client with a graphical interface.
In the technical solution, further, the paging parameter is calculated as follows:
the paging parameter is the position of the document at the starting point of the query + the number of the document pieces of the query.
In the above technical solution, further, the paging threshold is 10000.
In another aspect, the present invention further provides an electronic device for interactive ElasticSearch deep paging query, including:
a processor and a memory, the memory having a medium with program code stored therein, the electronic device being capable of performing the above method when the processor reads the program code stored in the medium.
Compared with the prior art, the technical scheme of the embodiment of the invention at least has the following beneficial technical effects:
(1) the method optimizes the ElasticSearch deep paging query problem, weakens the deep paging bottleneck, and optimizes the query so as to solve the problem that ElasticSearch performs paging query through from size combination. By dynamically providing detailed screening conditions for a user to select through interactive prompt, the query result is maintained within the set maximum value range as much as possible, the problem of error report caused by the fact that the query exceeds the set maximum value is avoided, the system function is optimized, and the query is optimized.
(2) An interactive query mechanism is used, on the premise of obtaining a query result by querying with a basic query condition, aggregation calculation analysis is carried out according to fields which can be classified exhaustively except the non-basic query condition, all types of the fields subjected to aggregation analysis are aggregated and converted into descriptions which can be understood by a user, the descriptions are converted into a detailed classification list, the detailed classification list is flexibly and dynamically recommended to the user in a page returning mode in real time, the user selects the classification which needs to be detailed according to the result of analysis recommendation, and the combined query condition is sent to a Web server for data query.
(3) And increasing query refinement conditions, reducing query pressure, further refining the query, reducing the number of matched documents, and maintaining the query result in a set maximum value range.
The invention improves the overall performance, and leads the user to inquire all the required data in an interactive mode more dynamically and flexibly under the condition of larger data volume. The information is accurately queried according to fields beyond the underlying query condition that can be exhaustively classified. Not only weakens the bottleneck brought by deep paging, but also ensures that the query is more friendly and efficient.
Drawings
FIG. 1 is a schematic diagram illustrating the structure and flow of a preferred embodiment of the interactive ElasticSearch deep paging query method according to the present invention;
FIG. 2 is a diagram of a specific application scenario of the interactive ElasticSearch deep paging query method provided by the present invention;
fig. 3 is a structure of a preferred embodiment of the electronic device for interactive ElasticSearch deep paging query according to the present invention.
The present invention is described in further detail below. The following examples are merely illustrative of the present invention and do not represent or limit the scope of the claims, which are defined by the claims.
Detailed Description
The technical scheme of the invention is further explained by the specific implementation mode in combination with the attached drawings.
In order to better illustrate the present invention and facilitate understanding of the technical solutions thereof, exemplary but non-limiting embodiments of the present invention are as follows, and it should be specifically described herein that the embodiments listed in the description of the present invention are only exemplary embodiments given for convenience of description, and should not be understood as the only correct embodiments of the present invention, and should not be interpreted as limiting the scope of the present invention.
Referring to fig. 1, a preferred embodiment of the interactive ElasticSearch deep paging query method of the present invention is based on an ElasticSearch server. When paging inquiry is carried out, the method comprises the following steps:
s101, receiving a query request sent by a client;
s102, judging whether the query request is a first query request, if so, turning to S104; if not, judging whether the paging parameter is smaller than a paging threshold which can be 10000 or not, wherein the paging parameter is obtained by calculating the sum of a starting point value from and a size, wherein the from represents the number of documents to be inquired, and the size represents the number of documents to be inquired; if the threshold value is smaller than the threshold value, turning to S104; if the threshold value is larger than the threshold value, turning to S105;
s104: sending a query request to an ElasticSearch, and turning to S112;
s105: aggregating data of exhaustively classifiable fields other than the computed query field;
s106: prompting a user whether to refine the query;
s107, when the user needs to refine the query, pushing and displaying the refined field of the aggregation calculation result to a client, wherein the client can be a browser or any client with a graphical interface;
s108: selecting a corresponding detailed field by a user;
s109: according to all specific detailed classifications corresponding to the field returned by the detailed field, aggregating all types of the field, converting the types of the field into descriptions which can be understood by the user, and returning the descriptions to the page;
s110: selecting a corresponding detailed classification by a user;
s111: according to the classification which needs to be refined and selected by the user, combining the query conditions and sending the query conditions to an ElasticSearch server;
s112, encapsulating result data returned by the ElasticSearch server;
and S113, returning the result data to the client, wherein the client can be a browser or any client with a graphical interface.
Steps S105 to S111 implement an interactive query mechanism between the user and the ElasticSearch, may dynamically recommend a query condition in real time to interact with the user, and allows the user to provide a selection, so that the query is more friendly and efficient. Meanwhile, the refined query conditions except the basic query conditions are calculated through aggregation to be selected by the user, so that the user can flexibly and accurately obtain effective data in a large amount of data according to recommendation. The bottleneck brought by deep paging is weakened, the information obtained by the user is more accurate, and a large amount of time cost and labor cost are saved.
As a more preferable embodiment, the aggregation calculation in step S105 further includes the steps of:
(1) analyzing the relation of fields except the query field;
(2) and aggregating the data of the fields with the association relationship.
For ease of understanding, a simple example is described below, such as "age" and "year and month of birth" fields in a database, where the two fields both belong to exhaustively classifiable fields and the data contents of the two fields have a mathematical operation relationship (i.e., age can be deduced through year and month of birth, and vice versa), and then they belong to an associative relationship, and aggregating the data of the two fields into one field can reduce the refinement field pushed to the user's choice. The complexity of selecting the refined field by the user is effectively reduced. And if a certain unit and the type of the work category are related to the gender (some work categories are only suitable for male employees), the work categories also have a certain correlation relationship, and the complexity of detailed selection of the user can be reduced by adopting an aggregation calculation method. It should be reminded that the above simple examples are only for helping understanding the technical solution of the present invention, and should not be taken as limiting the scope of protection. Those skilled in the art will appreciate that the description of the embodiments can be extended to more specific embodiments, and the embodiments are all within the scope of the present invention.
Referring to fig. 2, a specific application implementation scenario of the present invention is that a client (a browser or any client with a graphical interface) sends an external query request to a Web server, an ElasticSearch is responsible for providing corresponding data to the Web server for query, and the client and the Web server acquire and view more data from the ElasticSearch through an interactive query. In addition, the client is also responsible for displaying data for the user, particularly displaying an exhaustive classification field and a refined query list for the user to select the required refined classification; determining by the Web server the relationship of the paging parameters to the paging threshold, exhaustively classifying fields, aggregating field types and converting them into a user understandable description form, and sending a request to an ElasticSearch to query the data.
As a second aspect of the present invention, there is also provided an electronic apparatus, see fig. 3, including: a processor 1120 and a memory 1140, the memory 1140 having a medium (computer-readable storage medium) storing program code, the electronic device being enabled to perform the above-mentioned method steps with the aid of the input unit 1130 and the output unit 1150 when the processor reads the program code stored in the medium.
For method implementation, the various embodiments presented herein may be implemented in a computer-readable medium using, for example, computer software, hardware, or any combination thereof. For a hardware implementation, various embodiments of the present invention may be implemented by using at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a processor, a microprocessor, an electronic unit designed to perform the functions described herein, and in some cases, various embodiments of the present invention may be implemented in the processor 1120 shown in the figure. For a software implementation, the various embodiments presented herein may be implemented with separate software modules that allow for performing at least one function or operation. The software codes may be implemented by software applications (or programs) written in any suitable programming language, which may be stored in memory 1140 and executed by processor 1120.
The applicant declares that the present invention describes the detailed structural features of the present invention through the above embodiments, but the present invention is not limited to the above detailed structural features, that is, it does not mean that the present invention must be implemented depending on the above detailed structural features. It should be understood by those skilled in the art that any modifications of the present invention, equivalent substitutions of selected components of the present invention, additions of auxiliary components, selection of specific modes, etc., are within the scope and disclosure of the present invention.
The preferred embodiments of the present invention have been described in detail, however, the present invention is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present invention within the technical idea of the present invention, and these simple modifications are within the protective scope of the present invention.
It should be noted that the various technical features described in the above embodiments can be combined in any suitable manner without contradiction, and the invention is not described in any way for the possible combinations in order to avoid unnecessary repetition.
In addition, any combination of the various embodiments of the present invention is also possible, and the same should be considered as the disclosure of the present invention as long as it does not depart from the spirit of the present invention.
Claims (7)
1. An interactive ElasticSearch deep paging query method is characterized by comprising the following steps:
step 1: receiving a query request sent by a client;
step 2: judging whether the query request is a first query request, if so, turning to the step 4, otherwise, turning to the step 3;
and step 3: judging the relation between the paging parameters of the query request and a paging threshold, if the relation is smaller than the paging threshold, turning to the step 4, and otherwise, turning to the step 5; the paging parameter is calculated as follows: the paging parameter is the position of the document at the starting point of the query plus the number of the document pieces of the query;
and 4, step 4: sending the query request to an ElasticSearch, and turning to step 6;
and 5: aggregating and calculating data of exhaustively classified fields except the query field, and sending the data to an ElasticSearch server according to a refined field combination query condition selected by a user;
step 6: encapsulating result data returned by the ElasticSearch server;
and 7: returning the result data to the client;
the step 5 further comprises the following steps:
step 5.1: aggregating data of fields except the calculation inquiry field;
step 5.2: prompting a user whether to refine the query;
step 5.3, when the user needs to refine the query, pushing and displaying the refined field of the aggregation calculation result to the client;
step 5.4: selecting a corresponding detailed field by a user;
step 5.5: according to all specific detailed classifications corresponding to the field returned by the detailed field, aggregating all types of the field, converting the types of the field into descriptions which can be understood by the user, and returning the descriptions to the page;
step 5.6: selecting a corresponding detailed classification by a user;
step 5.7: and according to the classification which needs to be refined and selected by the user, combining the query conditions and sending the combined query conditions to an ElasticSearch server.
2. The method of claim 1, wherein the fields other than the query field are exhaustively classifiable fields.
3. The method of claim 1, wherein the calculation of the aggregation in step 5.1 further comprises the steps of:
step 5.1.1: analyzing the relation of fields except the query field;
step 5.1.2: and aggregating the data of the fields with the association relationship.
4. The method of claim 3, wherein the client is a browser.
5. The method of claim 3, wherein the client is a client having a graphical interface.
6. The method of claim 4, wherein the paging threshold is 10000.
7. An electronic device for interactive ElasticSearch deep paging query, comprising:
a processor and a memory, the memory having a medium with program code stored therein, the electronic device being capable of performing the method of any of claims 1-6 when the processor reads the program code stored in the medium.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811395531.XA CN109325047B (en) | 2018-11-22 | 2018-11-22 | Interactive elastic search deep paging query method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811395531.XA CN109325047B (en) | 2018-11-22 | 2018-11-22 | Interactive elastic search deep paging query method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109325047A CN109325047A (en) | 2019-02-12 |
| CN109325047B true CN109325047B (en) | 2021-04-16 |
Family
ID=65258169
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811395531.XA Active CN109325047B (en) | 2018-11-22 | 2018-11-22 | Interactive elastic search deep paging query method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109325047B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110083627B (en) * | 2019-04-28 | 2021-08-24 | 江苏满运软件科技有限公司 | Data processing method, system, computer device and storage medium |
| CN110232044B (en) * | 2019-06-17 | 2023-03-28 | 浪潮通用软件有限公司 | System and method for realizing big data summarizing and scheduling service |
| CN111680063B (en) * | 2020-05-25 | 2023-08-18 | 泰康保险集团股份有限公司 | Method and device for paging query data by elastic search |
| CN112364280A (en) * | 2020-10-27 | 2021-02-12 | 深圳开源互联网安全技术有限公司 | Dynamic adaptation method, system, device and storage medium for paging query |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8090698B2 (en) * | 2004-05-07 | 2012-01-03 | Ebay Inc. | Method and system to facilitate a search of an information resource |
| CN101842787A (en) * | 2007-09-14 | 2010-09-22 | 谷歌公司 | Suggesting alterntive queries in query results |
| US20120179705A1 (en) * | 2011-01-11 | 2012-07-12 | Microsoft Corporation | Query reformulation in association with a search box |
| CN102419778B (en) * | 2012-01-09 | 2013-03-20 | 中国科学院软件研究所 | Information searching method for discovering and clustering sub-topics of query statement |
| CN106528797A (en) * | 2016-11-10 | 2017-03-22 | 上海轻维软件有限公司 | DSL query method based on Elasticsearch |
| CN108509524A (en) * | 2018-03-12 | 2018-09-07 | 上海哔哩哔哩科技有限公司 | Method, server and the system of data processing of data processing |
-
2018
- 2018-11-22 CN CN201811395531.XA patent/CN109325047B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109325047A (en) | 2019-02-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109325047B (en) | Interactive elastic search deep paging query method and device | |
| Venetis et al. | On the selection of tags for tag clouds | |
| US20090240680A1 (en) | Techniques to perform relative ranking for search results | |
| US8078604B2 (en) | Identifying executable scenarios in response to search queries | |
| CN110069610A (en) | Search method, device, equipment and storage medium based on Solr | |
| CN102567461B (en) | Matching funnel for large document index | |
| CN104933100A (en) | Keyword recommendation method and device | |
| Im et al. | Linked tag: image annotation using semantic relationships between image tags | |
| US10796101B2 (en) | Optimizations of search engines for merging search results | |
| CN112257419A (en) | Intelligent retrieval method and device for calculating patent document similarity based on word frequency and semantics, electronic equipment and storage medium thereof | |
| EP3016003A1 (en) | Search method, apparatus and device and non-volatile computer storage medium | |
| CN101661490B (en) | Search engine, client thereof and method for searching page | |
| CN105989066A (en) | Information processing method and device | |
| CN115630144A (en) | Document searching method and device and related equipment | |
| US9760641B1 (en) | Site quality score | |
| US20200341990A1 (en) | Systems and methods for ranking electronic documents based on query token densities | |
| Urbano et al. | Minimal test collections for low-cost evaluation of audio music similarity and retrieval systems | |
| CN117744652A (en) | Domain feature word mining method and device based on large language model | |
| CN113360517A (en) | Data processing method and device, electronic equipment and storage medium | |
| US20080313166A1 (en) | Research progression summary | |
| CN110222156B (en) | Method and device for discovering entity, electronic equipment and computer readable medium | |
| CN105159899B (en) | Searching method and device | |
| CN111782916A (en) | Method and device for generating service information report | |
| US11556550B2 (en) | Methods and systems for no fail searching | |
| TWI674550B (en) | Innovative product development auxiliary system for additional function and method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |