CN113127596A

CN113127596A - Full-text retrieval method, system, electronic equipment and storage medium

Info

Publication number: CN113127596A
Application number: CN202110500852.7A
Authority: CN
Inventors: 杜芳
Original assignee: Beijing Minglue Zhaohui Technology Co Ltd
Current assignee: Beijing Minglue Zhaohui Technology Co Ltd
Priority date: 2021-05-08
Filing date: 2021-05-08
Publication date: 2021-07-16

Abstract

The application discloses a full-text retrieval method, a system, an electronic device and a storage medium, wherein the full-text retrieval method comprises the following steps: index establishment: establishing a full-text index, and writing field information participating in retrieval into the full-text index; and a retrieval parameter construction step: constructing retrieval parameters, and abstracting keywords of the field information to generate a DSL interface class; and query index acquisition: calling a DSL abstract method to obtain a DSL inquiry index, and combining the DSL inquiry index to obtain a complete DSL inquiry index; and (3) retrieval step: retrieving by the full DSL query index to recall relevant documents. According to the invention, the search engine divides the information of different dimensions related to the document and needing to participate in retrieval into different fields to be used as indexes, so that the document which is strongly related to the user intention can be recalled quickly and accurately, the expression capability of the unstructured data retrieval intention is enhanced, and the accuracy of the recalled data is improved.

Description

Full-text retrieval method, system, electronic equipment and storage medium

Technical Field

The present application relates to the field of retrieval technologies, and in particular, to a full-text retrieval method, a full-text retrieval system, an electronic device, and a storage medium.

Background

In most of the prior art, only one entry of a search keyword is usually used for searching for a certain keyword, and a common method is to match each field participating in the search in an index with the keyword, and a certain document is regarded as a hit as long as one field is successfully matched, and the matching degrees of all the fields participating in the search are integrated to perform relevancy ranking. For example, the user's intention is to search a book with the author Zhang III and the publication time 2021-01-01, and if "Zhang III 2021-01-01" is input in a conventional search manner, the document ranking titled "Zhang III 2021-01-01 overnight fullby" is likely to be in the front, while the book with the author Zhang III and the publication time 2021-01-01 is likely to be in the back, so that it is impossible to quickly and efficiently pinpoint and obtain information highly relevant to the client's search intention by the prior art.

Disclosure of Invention

The embodiment of the application provides a full-text retrieval method, a full-text retrieval system, electronic equipment and a storage medium, and at least solves the problems that information related to strong retrieval intention of a client cannot be quickly, efficiently and accurately positioned in the full-text retrieval process.

The invention provides a full-text retrieval method, which comprises the following steps:

index establishment: establishing a full-text index, and writing field information participating in retrieval into the full-text index;

and a retrieval parameter construction step: constructing retrieval parameters, and abstracting keywords of the field information to generate a DSL interface class;

and query index acquisition: calling a DSL abstract method to obtain a DSL inquiry index, and combining the DSL inquiry index to obtain a complete DSL inquiry index;

and (3) retrieval step: retrieving by the full DSL query index to recall relevant documents.

In the full-text retrieval method, the index establishing step includes, after the full-text index is established, extracting the field information to be retrieved from each document participating in the retrieval, and writing each field information into the full-text index.

In the above full-text retrieval method, the step of constructing the retrieval parameter includes constructing the retrieval parameter, abstracting the DSL interface class generated by the keyword corresponding to a single piece of the field information, and then implementing the DSL interface class according to different pieces of the field information.

In the full-text retrieval method, the query index obtaining step includes:

selecting: selecting the DSL interface class suitable for each kv pair according to the key of the retrieval parameter;

a generation step: and after calling the genDSL to generate the DSL query index of the kv pair, combining the relations between all the DSL query index references and the kv pair to generate the complete DSL query index.

The present invention also provides a full-text search system, wherein the full-text search system is suitable for the full-text search method, and the full-text search system comprises:

an index establishing unit: establishing a full-text index, and writing field information participating in retrieval into the full-text index;

a search parameter construction unit: constructing retrieval parameters, and abstracting keywords of the field information to generate a DSL interface class;

the query index acquisition unit: calling a DSL abstract method to obtain a DSL inquiry index, and combining the DSL inquiry index to obtain a complete DSL inquiry index;

a retrieval unit: retrieving by the full DSL query index to recall relevant documents.

In the full-text retrieval system, after the full-text index is established by the index establishing unit, the field information to be retrieved in each document participating in retrieval is extracted, and each field information is written into the full-text index.

In the full-text retrieval system, the retrieval parameter is constructed by the retrieval parameter constructing unit, and the DSL interface class generated by the keyword corresponding to a single piece of field information is abstracted, and then the DSL interface class is realized according to different pieces of field information.

In the full-text retrieval system, the query index obtaining unit includes:

a selecting module: selecting the DSL interface class suitable for each kv pair according to the key of the retrieval parameter;

a generation module: and after calling the genDSL to generate the DSL query index of the kv pair, combining the relations between all the DSL query index references and the kv pair to generate the complete DSL query index.

The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the full-text retrieval methods described above when executing the computer program.

The present invention also provides an electronic device readable storage medium having stored thereon computer program instructions, which, when executed by the processor, implement any of the full-text retrieval methods described above.

Compared with the related technology, the method and the device have the advantages that the search engine divides the information of different dimensions related to the document and needing to participate in retrieval into different fields for indexing, and different keywords are assigned to each field participating in retrieval, so that the document strongly related to the user intention can be recalled quickly and accurately, the expression capability of the unstructured data retrieval intention is enhanced, and the accuracy of the recalled data and the query optimization capability are improved.

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a flow chart of a full text retrieval method according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a full text search system according to the present invention;

fig. 3 is a block diagram of an electronic device according to an embodiment of the present application.

Wherein the reference numerals are:

an index establishing unit: 51;

a search parameter construction unit: 52;

the query index acquisition unit: 53;

a retrieval unit: 54, a first electrode;

a selecting module: 531;

a generation module: 532;

80 parts of a bus;

a processor: 81;

a memory: 82;

a communication interface: 83.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments provided in the present application without any inventive step are within the scope of protection of the present application.

It is obvious that the drawings in the following description are only examples or embodiments of the present application, and that it is also possible for a person skilled in the art to apply the present application to other similar contexts on the basis of these drawings without inventive effort. Moreover, it should be appreciated that such a development effort might be complex and tedious, but would nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill having the benefit of this disclosure, and thus should not be construed as a limitation of this disclosure.

Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of ordinary skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments without conflict.

Unless defined otherwise, technical or scientific terms referred to herein shall have the ordinary meaning as understood by those of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar words throughout this application are not to be construed as limiting in number, and may refer to the singular or the plural. The present application is directed to the use of the terms "including," "comprising," "having," and any variations thereof, which are intended to cover non-exclusive inclusions; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or elements, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Reference to "connected," "coupled," and the like in this application is not intended to be limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as referred to herein means two or more. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. Reference herein to the terms "first," "second," "third," and the like, are merely to distinguish similar objects and do not denote a particular ordering for the objects.

A Search Engine (Search Engine) is a system that collects information from the internet by using a specific computer program according to a certain policy, organizes and processes the information, provides a Search service for a user, and displays information related to user Search to the user. The search engine includes a full text index, a directory index, a meta search engine, a vertical search engine, a collective search engine, a portal search engine, a free link list, and the like. A search engine is composed of four parts of a searcher, an indexer, a retriever and a user interface. The search engine tracks links of web pages through a specific rule software, and crawls from one link to another link like a spider crawling on a spider web, so that the spider is also called a robot. The crawling of the search engine spider is input with certain rules that need to follow some commands or the content of the document. The search engine crawls to the webpage through the spider tracking link and stores the crawled data into an original page database. The page data is identical to the HTML from the user's browser. When a search engine spider grabs a page, the spider also performs certain repeated content detection, and once a large amount of copied, collected or copied content exists on a website with low weight, the spider is likely not to crawl any more. And the search engine carries out preprocessing of various steps on the page captured by the spider. After the user inputs the keywords in the search box, the ranking program calls the index database data, calculates the ranking and displays the ranking to the user, and the ranking process is directly interactive with the user. However, since the search engine has a huge data size, it can achieve small updates every day, but the ranking rules of the search engine are generally updated in different steps according to the day, week and month. The function of the searcher is to roam the internet, discover and gather information. The function of the indexer is to understand the information searched by the searcher, extract index terms therefrom, to represent documents and to generate an index table of the document repository. The function of the retriever is to quickly detect documents in the index base according to the query of a user, evaluate the relevance of the documents and the query, sort the results to be output and realize a certain user relevance feedback mechanism. The user interface functions to input user queries, display query results, and provide a user relevance feedback mechanism. The search engine is a necessary function provided for 'convenience of using websites by users' in website construction, and is also an effective tool for researching behaviors of users of websites. The efficient on-site retrieval can enable users to quickly and accurately find target information, so that the product/service sale is promoted more effectively, and the deep analysis of the search behavior of website visitors has important value for further making more effective network marketing strategies. From the environment of network marketing, the development of the environment of search engine marketing plays a very important role in promoting the network marketing; from the perspective of effective marketing, many companies have utilized search engine marketing to enable network marketing; in view of the components of the complete e-commerce concept, network marketing is the most important component and is an important link for delivering information to end customers.

The invention provides a full-text retrieval method, a full-text retrieval system, electronic equipment and a storage medium, wherein a search engine is utilized to divide information of different dimensions related to a document and needing to participate in retrieval into different fields for indexing, the specific fields can be flexibly customized according to business requirements, a user specifies keywords aiming at each field needing to be retrieved, and the document meeting the intention of the user is accurately recalled.

The present invention will be described with reference to specific examples.

Example one

The embodiment provides a full-text retrieval method. Referring to fig. 1, fig. 1 is a flowchart of a full-text search method according to an embodiment of the present application, and as shown in the figure, the full-text search method includes the following steps:

index creation step S1: establishing a full-text index, and writing field information participating in retrieval into the full-text index;

search parameter construction step S2: constructing retrieval parameters, and abstracting keywords of the field information to generate a DSL interface class;

query index acquisition step S3: calling a DSL abstract method to obtain a DSL inquiry index, and combining the DSL inquiry index to obtain a complete DSL inquiry index;

search step S4: retrieving by the full DSL query index to recall relevant documents.

In an embodiment, the index establishing step S1 includes, after the full-text index is established, extracting the field information to be retrieved in each document participating in the retrieval, and writing each field information into the full-text index.

In specific implementation, after the full-text index is established, a batch of documents are given, for each document, the technology such as nlp is used for extracting all field contents needing to participate in retrieval, and all field information of each document is written into the index. A document in the full-text index corresponds to all contents of a document, and the field name, the field type and the like in mapping can be defined according to different application scenes.

In an embodiment, the step S2 of constructing the search parameter includes constructing the search parameter, abstracting the DSL interface class generated by the keyword corresponding to a single piece of the field information, and then implementing the DSL interface class according to different pieces of the field information.

In specific implementation, the retrieval parameters are constructed to form a structure of Map < String, String >, key is a field name, value is a field value, a single field is abstracted to generate an interface class of the DSL for a keyword, the interface class is written as field keyword DSL, an abstraction method for generating the DSL is defined, and after the field is written as DSL genDsl (String), the interface class of the DSL is realized according to different field information.

In an embodiment, the query index obtaining step S3 includes:

a selecting step S31: selecting the DSL interface class suitable for each kv pair according to the key of the retrieval parameter;

generation step S32: and after calling the genDSL to generate the DSL query index of the kv pair, combining the relations between all the DSL query index references and the kv pair to generate the complete DSL query index.

The detailed implementation takes an elastic search as an example of a search engine, and a retrieval example diagram is given.

Name of field	Type of field	Description of the invention
			_id		es built-in field and is used to represent document id
title	text、keyword	Title of document, participating in retrieval
			content	text	Full-text content of document, participating in retrieval
author	keyword	Author, participating in retrieval
			tag	keyword	Label (R)
docVersion	keyword	Version(s)

Example two

Referring to fig. 2, fig. 2 is a schematic structural diagram of a full-text search system according to the present invention. As shown in fig. 2, the full-text search system of the present invention is suitable for the above full-text search method, and includes:

the index creating unit 51: establishing a full-text index, and writing field information participating in retrieval into the full-text index;

search parameter construction unit 52: constructing retrieval parameters, and abstracting keywords of the field information to generate a DSL interface class;

the query index obtaining unit 53: calling a DSL abstract method to obtain a DSL inquiry index, and combining the DSL inquiry index to obtain a complete DSL inquiry index;

the retrieval unit 54: retrieving by the full DSL query index to recall relevant documents.

In this embodiment, after the full-text index is established by the index establishing unit 51, the field information to be retrieved in each document participating in the retrieval is extracted, and each field information is written into the full-text index.

In this embodiment, the retrieval parameter is constructed by the retrieval parameter constructing unit 52, and the DSL interface class generated by the keyword corresponding to a single piece of field information is abstracted, and then the DSL interface class is implemented according to different pieces of field information.

In this embodiment, the query index obtaining unit 53 includes:

a selecting module 531: selecting the DSL interface class suitable for each kv pair according to the key of the retrieval parameter;

the generation module 532: and after calling the genDSL to generate the DSL query index of the kv pair, combining the relations between all the DSL query index references and the kv pair to generate the complete DSL query index.

EXAMPLE III

Referring to fig. 3, this embodiment discloses an embodiment of an electronic device. The electronic device may include a processor 81 and a memory 82 storing computer program instructions.

Specifically, the processor 81 may include a Central Processing Unit (CPU), or A Specific Integrated Circuit (ASIC), or may be configured to implement one or more Integrated circuits of the embodiments of the present Application.

Memory 82 may include, among other things, mass storage for data or instructions. By way of example, and not limitation, memory 82 may include a Hard Disk Drive (Hard Disk Drive, abbreviated to HDD), a floppy Disk Drive, a Solid State Drive (SSD), flash memory, an optical Disk, a magneto-optical Disk, tape, or a Universal Serial Bus (USB) Drive or a combination of two or more of these. Memory 82 may include removable or non-removable (or fixed) media, where appropriate. The memory 82 may be internal or external to the full-text retrieval device, where appropriate. In a particular embodiment, the memory 82 is a Non-Volatile (Non-Volatile) memory. In particular embodiments, Memory 82 includes Read-Only Memory (ROM) and Random Access Memory (RAM). The ROM may be mask-programmed ROM, Programmable ROM (PROM), Erasable PROM (FPROM), Electrically Erasable PROM (EFPROM), Electrically rewritable ROM (EAROM), or FLASH Memory (FLASH), or a combination of two or more of these, where appropriate. The RAM may be a Static Random-Access Memory (SRAM) or a Dynamic Random-Access Memory (DRAM), where the DRAM may be a Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), an Extended data output Dynamic Random-Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), and the like.

The memory 82 may be used to store or cache various data files for processing and/or communication use, as well as possible computer program instructions executed by the processor 81.

The processor 81 implements any of the full-text retrieval methods in the above-described embodiments by reading and executing computer program instructions stored in the memory 82.

In some of these embodiments, the electronic device may also include a communication interface 83 and a bus 80. As shown in fig. 3, the processor 81, the memory 82, and the communication interface 83 are connected via the bus 80 to complete communication therebetween.

The communication interface 83 is used for implementing communication between modules, devices, units and/or equipment in the embodiment of the present application. The communication port 83 may also be implemented with other components such as: the data communication is carried out among external equipment, image/full text retrieval equipment, a database, external storage, an image/full text retrieval workstation and the like.

The bus 80 includes hardware, software, or both to couple the components of the electronic device to one another. Bus 80 includes, but is not limited to, at least one of the following: data Bus (DataBus), Address Bus (Address Bus), Control Bus (Control Bus), Expansion Bus (Expansion Bus), and Local Bus (Local Bus). By way of example, and not limitation, Bus 80 may include an Accelerated Graphics Port (AGP) or other Graphics Bus, an Enhanced Industry Standard Architecture (EISA) Bus, a Front-Side Bus (FSB), a Hyper Transport (HT) Interconnect, an ISA (ISA) Bus, an InfiniBand (InfiniBand) Interconnect, a Low Pin Count (LPC) Bus, a memory Bus, a microchannel Architecture (MCA) Bus, a PCI (Peripheral Component Interconnect) Bus, a PCI-Express (PCI-X) Bus, a Serial Advanced Technology Attachment (SATA) Bus, a Video Electronics Bus (audio Electronics Association), abbreviated VLB) bus or other suitable bus or a combination of two or more of these. Bus 80 may include one or more buses, where appropriate. Although specific buses are described and shown in the embodiments of the application, any suitable buses or interconnects are contemplated by the application.

The electronic device may be connected to a full text retrieval system to implement the method described in connection with fig. 1.

The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

In conclusion, the invention divides the information of different dimensions related to the document and needing to participate in retrieval into different fields for indexing through the search engine, and appoints different keywords for each field participating in retrieval, thereby quickly and accurately recalling the document strongly related to the user intention, enhancing the expression capability of the unstructured data retrieval intention and improving the accuracy of the recalled data.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent application shall be subject to the protection scope of the appended claims.

Claims

1. A full-text search method, comprising:

2. The full-text retrieval method according to claim 1, wherein the index creating step includes, after creating the full-text index, extracting the field information to be retrieved in each document participating in the retrieval, and writing each of the field information into the full-text index.

3. The full-text retrieval method according to claim 1, wherein the retrieval parameter constructing step includes constructing the retrieval parameter, abstracting the DSL interface class generated by the keyword corresponding to a single field information, and then implementing the DSL interface class according to different field information.

4. The full-text retrieval method according to claim 1, wherein the query index obtaining step includes:

5. A full-text search system adapted to the full-text search method according to any one of claims 1 to 4, the full-text search system comprising:

6. The full-text retrieval system according to claim 5, wherein after the full-text index is created by the index creating unit, the field information to be retrieved in each document participating in retrieval is extracted, and each field information is written in the full-text index.

7. The full-text retrieval system of claim 6, wherein the retrieval parameter is constructed by the retrieval parameter constructing unit, and after the DSL interface class generated by the keyword corresponding to a single piece of the field information is abstracted, the DSL interface class is implemented according to different pieces of the field information.

8. The full-text retrieval system according to claim 7, wherein the query index obtaining unit includes:

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the full text retrieval method of any one of claims 1 to 4 when executing the computer program.

10. An electronic device readable storage medium having stored thereon computer program instructions which, when executed by the processor, implement a full text retrieval method according to any one of claims 1 to 4.