CN113378022A - In-station search platform, search method and related device - Google Patents
In-station search platform, search method and related device Download PDFInfo
- Publication number
- CN113378022A CN113378022A CN202010163274.8A CN202010163274A CN113378022A CN 113378022 A CN113378022 A CN 113378022A CN 202010163274 A CN202010163274 A CN 202010163274A CN 113378022 A CN113378022 A CN 113378022A
- Authority
- CN
- China
- Prior art keywords
- data
- source
- index
- target
- search
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims description 38
- 230000001360 synchronised effect Effects 0.000 claims description 24
- 230000008569 process Effects 0.000 claims description 13
- 238000011161 development Methods 0.000 abstract description 11
- 238000012423 maintenance Methods 0.000 abstract description 10
- 238000007726 management method Methods 0.000 description 35
- 238000012545 processing Methods 0.000 description 14
- 238000004891 communication Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 11
- 230000018109 developmental process Effects 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 5
- 239000000463 material Substances 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application discloses an in-station search platform which comprises a management console, an index management module and a search engine. The management console is used for managing configuration information, the configuration information comprises data source configuration and index configuration, and the data source configuration reflects target data sources corresponding to different search services; the index management module is used for constructing target index data according to index configuration and synchronizing source data in a target data source into the target index data; and the search engine is used for returning a search result corresponding to the object to be queried according to the target index data when receiving a query request carrying the object to be queried. Because the platform is configured with the target data sources corresponding to different search services in advance, when certain search service needs to be realized, the search service can be realized only by synchronizing the data in the target data source corresponding to the search service into the target index data, the platform does not need to be repeatedly developed to meet various search service requirements, and the development cost and the operation and maintenance cost are reduced.
Description
Technical Field
The present application relates to the field of internet, and in particular, to an in-site search platform, a search method, and a related apparatus.
Background
With the development and rapid popularization of internet technology, more and more websites are provided, the number of webpages on the internet per day is increased in the order of tens of millions, and the required materials are required to be searched for in massive data resources, which is not different from a large sea fishing needle. In order to meet the search requirements of people, search engines are produced. The search engine is a system that collects information from the internet by using a specific computer program according to a certain policy, provides a retrieval service for a user after organizing and processing the information, and displays information related to user retrieval to the user. At present, most search engines are all network search, information is noisy, and even the information is flooded by a large number of advertisements. The in-site search engine can perform in-site global search for websites concerned by the user, and the relevance of information is greatly improved.
At present, most internet enterprises develop search platforms of different search services based on open source search engines, and as search requirements for diversity of business lines increase, the search platforms are generally required to be developed aiming at new search services, so that a large amount of repeated development is caused, and the development cost and the operation and maintenance cost are high.
Disclosure of Invention
In order to solve the technical problems, the application provides an in-station search platform, a search method and a related device, so that the platform does not need to be repeatedly developed to meet various search service requirements, and the development cost and the operation and maintenance cost are reduced.
The embodiment of the application discloses the following technical scheme:
in a first aspect, an embodiment of the present application provides an in-site search platform, where the platform includes a management console, an index management module, and a search engine:
the management console is used for managing configuration information, the configuration information comprises data source configuration and index configuration, and the data source configuration reflects target data sources corresponding to different search services;
the index management module is used for constructing target index data according to the index configuration and synchronizing source data in the target data source to the target index data;
and the search engine is used for returning a search result corresponding to the object to be queried according to the target index data determined by the management console when receiving a query request carrying the object to be queried.
Optionally, the index management module is further configured to update the target index data according to the reconstructed index data;
and the search engine is used for returning the search result corresponding to the object to be inquired according to the updated target index data.
Optionally, the reconstruction index data is constructed offline.
Optionally, the index management module is specifically configured to:
loading the reconstruction index data;
if a synchronization process for synchronizing source data in the target data source to the target index data exists, determining whether a difference value between a first offset of the target index data and a second offset of the reconstructed index data meets a preset condition;
and if so, updating the target index data by using the reconstructed index data.
Optionally, the index configuration includes a full index or an incremental index.
Optionally, if the source data and the target index data are in a table form, the mode for synchronizing the source data in the target data source to the target index data includes one or more of the following combinations:
the source data comprises a source main table, and one source main table is synchronized to one target index table;
the source data comprises a source main table and a source auxiliary table, one source main table corresponds to one source auxiliary table, and the source data is synchronized into one target index table in a wide table form;
the source data comprises a source main table and source auxiliary tables, one source main table corresponds to N source auxiliary tables, and the source data is synchronized into one target index table in a parent file mode;
the source data comprises a source main table and source auxiliary tables, N source main tables correspond to one source auxiliary table, and the source data is synchronized into one target index table in a manner of multi-row index record of the wide table.
In a second aspect, an embodiment of the present application provides an intra-site search method, where the method includes:
acquiring a query request, wherein the query request comprises an object to be queried;
obtaining a search result corresponding to the object to be queried according to the target index data; the target index data is constructed according to index configuration in configuration information, the configuration information further comprises data source configuration, and the data source configuration reflects target data sources corresponding to different search services; source data in the target data source is synchronized into the target index data;
and returning the search result.
Optionally, the returning the search result includes:
and sorting the search results according to the user information included in the query request, and returning the sorted search results.
In a third aspect, an embodiment of the present application provides an intra-site search apparatus, where the apparatus includes a first obtaining unit, a second obtaining unit, and a returning unit:
the first obtaining unit is used for obtaining a query request, and the query request comprises an object to be queried;
the second obtaining unit is used for obtaining a search result corresponding to the object to be queried according to the target index data; the target index data is constructed according to index configuration in configuration information, the configuration information further comprises data source configuration, and the data source configuration reflects target data sources corresponding to different search services; source data in the target data source is synchronized into the target index data;
the returning unit is used for returning the search result.
Optionally, the returning unit is configured to:
and sorting the search results according to the user information included in the query request, and returning the sorted search results.
In a fourth aspect, embodiments of the present application provide an electronic device, comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
acquiring a query request, wherein the query request comprises an object to be queried;
obtaining a search result corresponding to the object to be queried according to the target index data; the target index data is constructed according to index configuration in configuration information, the configuration information further comprises data source configuration, and the data source configuration reflects target data sources corresponding to different search services; source data in the target data source is synchronized into the target index data;
and returning the search result.
In a fifth aspect, embodiments of the present application provide a machine-readable medium having stored thereon instructions, which when executed by one or more processors, cause an apparatus to perform the method of any of the second aspects.
According to the technical scheme, the in-site search platform comprises a management console, an index management module and a search engine. The management console is used for managing configuration information, the configuration information comprises data source configuration and index configuration, and the data source configuration reflects target data sources corresponding to different search services; the index management module is used for constructing target index data according to index configuration and synchronizing source data in a target data source into the target index data; and the search engine is used for returning a search result corresponding to the object to be queried according to the target index data when receiving a query request carrying the object to be queried. Because the target data sources corresponding to different search services are configured on the platform in advance, when certain search service needs to be realized, the search service can be realized only by synchronizing the data in the target data source corresponding to the search service into the target index data, the platform does not need to be repeatedly developed to meet various search service requirements, and the development cost and the operation and maintenance cost are reduced.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is an exemplary diagram of an application scenario of an in-station search platform provided in an embodiment of the present application;
fig. 2 is a structural diagram of an in-station search platform according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram of a data parsing process according to an embodiment of the present application;
fig. 4 is a structural diagram of an in-station search platform provided in an embodiment of the present application;
FIG. 5 is a diagram illustrating a detailed process of updating target index data according to an embodiment of the present application;
fig. 6 is a flowchart of an intra-site search method according to an embodiment of the present application;
fig. 7 is a structural diagram of an intra-site search apparatus according to an embodiment of the present application;
fig. 8 is a structural diagram of a terminal device according to an embodiment of the present application;
fig. 9 is a block diagram of a server according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
At present, most of in-site search platforms are obtained by developing different search services based on open source search engines, and with the increase of search requirements of users, the search platforms are generally required to be developed aiming at new search services, so that a large amount of repeated development is caused, and the development cost and the operation and maintenance cost are higher.
For example, for a search platform developed by a search service for commodity query in an e-commerce scene, when a user wishes to apply the search platform to a social scene such as microblog search, in order to meet the search requirement of the user, a platform developer needs to develop the search platform again based on an open source search engine, so that the search platform can meet the requirement of microblog search in the social scene.
In order to solve the above technical problem, an embodiment of the present application provides an in-site search platform, and it should be noted that in this embodiment, the in-site search platform may be deployed on a server, and the server may be a server in a cluster or an independent server.
The in-site search platform provided by the embodiment of the application provides various search services, for example, provides a quick data search service for various scenes such as advertisements, social contact, e-commerce and the like.
Referring to fig. 1, fig. 1 shows an optional application scenario of the in-site search platform provided in the embodiment of the present application, where the application scenario includes a terminal device 101 and a server 102, and the in-site search platform is deployed on the server 102. The terminal device may be, for example, a smart terminal, a computer, a Personal Digital Assistant (PDA), a tablet computer, or the like.
As the platform is provided with target data sources corresponding to different search services, the platform can meet various search service requirements of users. When a user triggers a query request including an object to be queried through the terminal device 101, the platform on the server 102 is called to perform searching, and a search result corresponding to the object to be queried is returned to the user.
The search result is content related to the object to be queried, such as attribute information of the object to be queried, profile information of the object to be queried, detailed content of the object to be queried, and the like. According to different search services, objects to be queried may be different, and corresponding search results may also be different. For example, when a commodity search service is provided for an e-market scene, an object to be queried may be a commodity, and a search result may be attribute information of the commodity; when the microblog search service is provided for a social scene, the object to be queried may be a person (e.g., a person name "xxx"), the search result may be a microblog published by "xxx", and so on.
Because the target data sources corresponding to different search services are pre-configured on the in-station search platform, when certain search service needs to be realized, the search service can be realized only by synchronizing the data in the target data source corresponding to the search service into the target index data, the platform does not need to be repeatedly developed to meet various search service requirements, and the development cost and the operation and maintenance cost are reduced.
Next, a detailed description will be given of an in-site search platform provided in an embodiment of the present application with reference to the drawings.
Referring to fig. 2, fig. 2 shows a block diagram of an in-site search platform including a management console 201, an index management module 202, and a search engine 203.
The management console 201 is configured to manage configuration information, where the configuration information includes data source configuration and index configuration, and the data source configuration reflects target data sources corresponding to different search services, including an On Line Transaction Processing (OLTP) data source and an On Line Analytical Processing (OLAP) data source, for example, a commodity data source corresponding to a commodity search service in a shopping mall in a tv setting, a microblog data source corresponding to a microblog search service in a social setting, and the like.
The configuration information may be configured in advance by a developer, and the developer may configure target data sources corresponding to different search services, so that when the developer has a certain search service, the developer may automatically access the target data source corresponding to the search service to perform a search. During configuration, the corresponding file address can be input at the data file address and the data file ending mark file, so that data source configuration is completed, and meanwhile, an analysis package can be uploaded on the configuration interface, so that a data analysis mode is customized, and data acquired from a target data source can be analyzed in a subsequent process.
Since there may be incremental or full requirements for each search service, the index configuration may include a full index or an incremental index in order to meet the search service requirements of different users. For example, full-scale information or incremental-scale information may be configured, so that full-scale index data or incremental-scale index data may be constructed according to the index configuration.
It should be noted that, in the actual configuration process of the in-site search platform, in general, before the data source configuration is performed, a search application (configuration application information) may be created, for example, the name of the created search application is "material retrieval", and then a search service for material retrieval may be provided by the in-site search platform. A data synchronization mode, such as incremental data synchronization or full data synchronization, may be selected when creating the search application. After the application information is configured, structural information can be configured, the structural information reflects a data structure of material related data during material searching, field names, field types, word segmentation, index, storage, link and the like of each piece of data can be set, and each piece of data can be deleted and the like.
The index management module 202 is configured to construct target index data according to index configuration, and synchronize source data in a target data source to the target index data. Because target data sources corresponding to different search services are pre-configured on the in-station search platform, when a certain search service needs to be realized, data in the target data source corresponding to the search service needs to be synchronized into target index data.
Since the index configuration may include a full index or an incremental index, when the target index data is constructed and the source data in the target data source is synchronized into the target index data, full data synchronization or incremental data synchronization may be used. The method can prompt the synchronization progress during synchronization, and can also perform operations such as 'modification', 'activation', 'deletion' and the like on the synchronization progress.
The data volume of the full index is quite large, and the time consumption is long, so that the full index can be quickly constructed offline in a MapReduce (MR) task mode; the incremental index can be based on a Flink index task, and incremental index data is constructed in real time through a message queue.
In this embodiment, since multiple target data sources are configured, data patterns of different target data sources may be different, and a pattern for synchronizing source data in the target data sources into target index data may also be different. If the source data and the target index data are in table form, that is, the source data corresponds to a source table, and the target index data corresponds to a target index table, the source data may include a source main table (a type of source table), or include a source main table and a source auxiliary table (a type of source table), where the source main table may be used to store the object, and the source auxiliary table may be used to store the object attribute. For example, in a scenario in which a search service is provided for school management business, "student" is targeted, the "student name" may be stored in the source main table, and "class", "gender", "age", and the like may be targeted for attributes, stored in the source supplementary table; if "class" is used as the object, the "class" may be stored in the source main table, and each "student" may be stored in the source supplementary table as the object attribute.
According to different conditions of a source table corresponding to the source data, the mode for synchronizing the source data in the target data source into the target index data comprises one or more of the following combinations: if the source data comprises a source main table, a source main table can be directly synchronized into a target index table; if the source data comprises a source main table and a source auxiliary table, and one source main table corresponds to one source auxiliary table, synchronizing the source data into one target index table in a broad table form; if the source data comprises a source main table and source auxiliary tables, one source main table corresponds to N source auxiliary tables, for example, a source main table of "class", a source auxiliary table of "student", and one class may include a plurality of students, the source main table of one "class" corresponds to the source auxiliary tables of N "students", and in order to reduce data redundant source data as much as possible, the source data may be synchronized into one target sub-document index table in a parent document manner; if the source data includes a source main table and a source additional table, where N source main tables correspond to one source additional table, for example, a source main table of "student", a source additional table of "class", and one class may include multiple students, then the source main tables of N "students" correspond to one source additional table of "class", and the source data may be synchronized into one target index table in a manner of wide table multi-row index records.
It should be noted that different formats may exist in data of different target data sources, and before source data in a target data source is synchronized into target index data, corresponding parsing modes may exist for different target data sources, and data parsing is completed in a customized manner by uploading a custom parsing package on the in-site search platform. The parsing process can be as shown in fig. 3: s301, configuring various target data sources on the platform, and dynamically managing the target data sources, such as deleting or adding target data sources corresponding to different search services; s302, data pulling is carried out on a target data source to obtain source data, and common logic encapsulation is carried out; s303, defining an analysis mode by uploading an analysis package form through a platform, obtaining various data analysis modes, and abstracting the analysis mode to an off-line/on-line index task logic so as to analyze data before data synchronization; s304, packaging and warehousing the data (common logic packaging); s305, obtaining a target index data format through a defined analysis mode, wherein the target index data format is set by a user.
In addition, the index management module 202 may also provide an Application Programming Interface (API) to provide search services to the outside, so as to meet various search service requirements.
And the search engine 203 is configured to, when receiving a query request carrying an object to be queried, return a search result corresponding to the object to be queried according to the target index data determined by the management console 201, and complete a search service.
In the present embodiment, in order to improve the search capability and meet the high concurrency requirement, the search engine (Solr)203 in the present embodiment may be a Solr cluster (Solr Cloud), that is, a distributed search engine based on Solr and Zookeeper, and uses the Zookeeper as a configuration information center of the Solr Cloud to uniformly manage the configuration of the Solr Cloud, as shown in fig. 4.
Based on the introduction general configuration, the whole process of automatically constructing the in-site search platform can be realized, and the whole process comprises the steps of configuring an OLTP/OLAP target data source; constructing target index data, and synchronizing source data in a target data source to the target index data; data source real-time binlog subscription; adding the incremental index data into a message queue, and recording the offset; a target data source full backup (dump); constructing full index data in an off-line manner; the management console triggers the solr cluster to download the full index data and executes index merging; submitting an increment index task based on the Flink, deploying an increment index cluster, accessing a message queue, and consuming real-time data; and (5) uniformly inquiring service deployment and providing search service for the outside.
According to the technical scheme, the in-site search platform comprises a management console, an index management module and a search engine. The management console is used for managing configuration information, the configuration information comprises data source configuration and index configuration, and the data source configuration reflects target data sources corresponding to different search services; the index management module is used for constructing target index data according to index configuration and synchronizing source data in a target data source into the target index data; and the search engine is used for returning a search result corresponding to the object to be queried according to the target index data when receiving a query request carrying the object to be queried. Because the target data sources corresponding to different search services are configured on the platform in advance, when certain search service needs to be realized, the search service can be realized only by synchronizing the data in the target data source corresponding to the search service into the target index data, the platform does not need to be repeatedly developed to meet various search service requirements, and the development cost and the operation and maintenance cost are reduced. The in-station search function is realized at low cost, and the service iteration is effectively accelerated.
It should be noted that in many scenarios, index data may need to be reconstructed, for example, in a full-text retrieval class scenario, index data generally needs to be reconstructed due to a change of a table structure of source data and a change of a tokenizer and a dictionary class. In this case, the index management module 202 is further configured to reconstruct the index data, and update the target index data according to the reconstructed index data. Correspondingly, the search engine 203 may return a search result corresponding to the object to be queried according to the updated target index data.
When the index data amount is relatively huge, the reconstruction of the index data usually needs to consume several hours or more, and the reconstruction of the index data is obtained by off-line construction in order to ensure that the on-line search service is not influenced.
In order to further ensure that the online service is not affected and the query end is unaware, the index management module 202 may update the target index data by loading the reconstructed index data, determining whether a synchronization process exists for synchronizing source data in a target data source to the target index data, if so, determining whether a difference between a first offset of the target index data and a second offset of the reconstructed index data meets a preset condition, and if so, indicating that the data synchronization progress of the target index data and the reconstructed index data is close to each other.
The detailed process of target index data update can be seen in fig. 5, which includes:
s501, executing an offline index data reconstruction task to obtain reconstructed index data.
S502, the solr cluster loads the reconstruction index data to the local.
S503, judging whether all the solr clusters successfully load the reconstruction index data, if so, executing S504, and if not, ending the process.
S504, constructing a new index set, wherein the index set comprises reconstruction index data.
S505, judging whether a synchronization process for synchronizing the source data in the target data source to the target index data exists, if so, executing S506, and if not, executing S509.
S506, acquiring a first offset of the target index data and a second offset of the reconstruction index data.
And S507, determining whether the difference value between the first offset and the second offset meets a preset condition, if so, executing S508.
S508, stopping using the target index data.
And S509, updating the target index data by using the reconstructed index data, switching the alias to the reconstructed index data, and deleting the target index data.
The distributed cooperation of full index and incremental index is realized through the offline reconstruction of index data, the synchronous state is automatically judged through the online and offline management and control cooperation, alias names are switched and point to the reconstructed index data after the data synchronization is successful, the smooth updating of the target index data at the query end is realized, and the influence on the normal operation of online search service is avoided.
It is understood that, in this embodiment, various triggered tasks, such as an incremental indexing task, a full indexing task, and a reconstruction task of index data, may be monitored by the management console 201, and flow control such as a data synchronization flow may also be controlled by the management console 201. In addition, the management console 201 can also perform dynamic operation and maintenance on the Solr clusters in the Solr Cloud.
Based on this, as shown in fig. 4, the in-station search platform may further perform flow control, manage configuration information, and cluster operation and maintenance by using the management console 201; the index management module 202 includes a full index module 2021 and an incremental index module 2022, and for this purpose, the management console 201 may monitor the triggered task, such as an incremental index task, a full index task, and dynamically configure a full index or an incremental index. The search engine (Solr Cloud)203 may obtain the index data by obtaining the data processing stream from the full index module 2021 and the incremental index module 2022. When receiving a query request through the general query API, returning a search result corresponding to an object to be queried according to the target index data determined by the management console 201, thereby completing a search service.
Based on the station search platform provided in the foregoing, an embodiment of the present application further provides a station search method, with reference to fig. 6, where the method includes:
s601, obtaining a query request, wherein the query request comprises an object to be queried.
S602, obtaining a search result corresponding to the object to be queried according to the target index data.
The target index data is constructed according to index configuration in configuration information, the configuration information further comprises data source configuration, and the data source configuration reflects target data sources corresponding to different search services; source data in the target data source is synchronized into the target index data.
And S603, returning the search result.
In some cases, the requirements of search results may vary from user to user, and the user information may reflect the user's requirements to some extent, such as the user's gender, age, location, transaction history, and so forth. Therefore, in order to return the search results meeting the requirements of the user, the query request can also include user information, so that the search results are returned according to the user information.
In this case, the search result may be returned in S603 by sorting the search results according to the user information included in the query request and returning the sorted search results. For example, the degree of relevance of each search result to the user may be determined according to the user information, and the search results may be returned in the order by sorting in the order of the degrees of relevance from large to small. Or returning the search result with the correlation degree reaching the preset threshold value to the user.
Because the target data sources corresponding to different search services are pre-configured on the in-station search platform used by the search method, when certain search service needs to be realized, the search service can be realized only by synchronizing the data in the target data source corresponding to the search service into the target index data, the platform does not need to be repeatedly developed to meet various search service requirements, and the development cost and the operation and maintenance cost are reduced.
The embodiment of the present application further provides an intra-site search apparatus, referring to fig. 7, the apparatus includes a first obtaining unit 701, a second obtaining unit 702, and a returning unit 703:
the first obtaining unit 701 is configured to obtain a query request, where the query request includes an object to be queried;
the second obtaining unit 702 is configured to obtain a search result corresponding to the object to be queried according to the target index data; the target index data is constructed according to index configuration in configuration information, the configuration information further comprises data source configuration, and the data source configuration reflects target data sources corresponding to different search services; source data in the target data source is synchronized into the target index data;
the returning unit 703 is configured to return the search result.
Optionally, the returning unit is configured to:
and sorting the search results according to the user information included in the query request, and returning the sorted search results.
The present embodiment also provides an electronic device, which may be a terminal device, and fig. 8 is a block diagram illustrating a terminal device 800 according to an exemplary embodiment. For example, the terminal device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.
Referring to fig. 8, terminal device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.
The processing component 802 generally controls overall operation of the terminal device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing elements 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
The memory 804 is configured to store various types of data to support operations at the terminal device 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The multimedia component 808 comprises a screen providing an output interface between the terminal device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. When the terminal device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the apparatus 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
In an exemplary embodiment, the terminal device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.
In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 804 comprising instructions, executable by the processor 820 of the terminal device 800 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
The electronic device provided in this embodiment may also be a server, and fig. 9 is a schematic structural diagram of the server in this embodiment of the present invention. The server 900 may vary widely in configuration or performance and may include one or more Central Processing Units (CPUs) 922 (e.g., one or more processors) and memory 932, one or more storage media 930 (e.g., one or more mass storage devices) storing applications 942 or data 944. Memory 932 and storage media 930 can be, among other things, transient storage or persistent storage. The program stored on the storage medium 930 may include one or more modules (not shown), each of which may include a series of instruction operations for the server. Still further, a central processor 922 may be provided in communication with the storage medium 930 to execute a series of instruction operations in the storage medium 930 on the server 900.
The server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input-output interfaces 958, one or more keyboards 956, and/or one or more operating systems 941, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
A non-transitory computer readable storage medium having instructions therein which, when executed by a processor of a server, enable the server to perform an in-site search method, the method comprising:
acquiring a query request, wherein the query request comprises an object to be queried;
obtaining a search result corresponding to the object to be queried according to the target index data; the target index data is constructed according to index configuration in configuration information, the configuration information further comprises data source configuration, and the data source configuration reflects target data sources corresponding to different search services; source data in the target data source is synchronized into the target index data;
and returning the search result.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium may be at least one of the following media: various media that can store program codes, such as read-only memory (ROM), RAM, magnetic disk, or optical disk.
It should be noted that, in the present specification, all the embodiments are described in a progressive manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus and system embodiments, since they are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described embodiments of the apparatus and system are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The above description is only one specific embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. An in-site search platform, comprising a management console, an index management module, and a search engine:
the management console is used for managing configuration information, the configuration information comprises data source configuration and index configuration, and the data source configuration reflects target data sources corresponding to different search services;
the index management module is used for constructing target index data according to the index configuration and synchronizing source data in the target data source to the target index data;
and the search engine is used for returning a search result corresponding to the object to be queried according to the target index data determined by the management console when receiving a query request carrying the object to be queried.
2. The platform of claim 1, wherein the index management module is further configured to update the target index data based on reconstructed index data;
and the search engine is used for returning the search result corresponding to the object to be inquired according to the updated target index data.
3. The platform of claim 2, wherein the reconstruction index data is constructed offline.
4. The platform of claim 3, wherein the index management module is specifically configured to:
loading the reconstruction index data;
if a synchronization process for synchronizing source data in the target data source to the target index data exists, determining whether a difference value between a first offset of the target index data and a second offset of the reconstructed index data meets a preset condition;
and if so, updating the target index data by using the reconstructed index data.
5. The platform of claim 1, wherein the index configuration comprises a full index or an incremental index.
6. The platform of claim 1, wherein if the source data and the target index data are in table form, the mode for synchronizing the source data in the target data source to the target index data comprises one or more of the following combinations:
the source data comprises a source main table, and one source main table is synchronized to one target index table;
the source data comprises a source main table and a source auxiliary table, one source main table corresponds to one source auxiliary table, and the source data is synchronized into one target index table in a wide table form;
the source data comprises a source main table and source auxiliary tables, one source main table corresponds to N source auxiliary tables, and the source data is synchronized into one target index table in a parent file mode;
the source data comprises a source main table and source auxiliary tables, N source main tables correspond to one source auxiliary table, and the source data is synchronized into one target index table in a manner of multi-row index record of the wide table.
7. An intra-site search method, the method comprising:
acquiring a query request, wherein the query request comprises an object to be queried;
obtaining a search result corresponding to the object to be queried according to the target index data; the target index data is constructed according to index configuration in configuration information, the configuration information further comprises data source configuration, and the data source configuration reflects target data sources corresponding to different search services; source data in the target data source is synchronized into the target index data;
and returning the search result.
8. An in-station search device is characterized by comprising a first acquisition unit, a second acquisition unit and a return unit:
the first obtaining unit is used for obtaining a query request, and the query request comprises an object to be queried;
the second obtaining unit is used for obtaining a search result corresponding to the object to be queried according to the target index data; the target index data is constructed according to index configuration in configuration information, the configuration information further comprises data source configuration, and the data source configuration reflects target data sources corresponding to different search services; source data in the target data source is synchronized into the target index data;
the returning unit is used for returning the search result.
9. An electronic device comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors the one or more programs including instructions for:
acquiring a query request, wherein the query request comprises an object to be queried;
obtaining a search result corresponding to the object to be queried according to the target index data; the target index data is constructed according to index configuration in configuration information, the configuration information further comprises data source configuration, and the data source configuration reflects target data sources corresponding to different search services; source data in the target data source is synchronized into the target index data;
and returning the search result.
10. A machine-readable medium having stored thereon instructions, which when executed by one or more processors, cause an apparatus to perform the method of claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010163274.8A CN113378022A (en) | 2020-03-10 | 2020-03-10 | In-station search platform, search method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010163274.8A CN113378022A (en) | 2020-03-10 | 2020-03-10 | In-station search platform, search method and related device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113378022A true CN113378022A (en) | 2021-09-10 |
Family
ID=77568997
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010163274.8A Pending CN113378022A (en) | 2020-03-10 | 2020-03-10 | In-station search platform, search method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113378022A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114579596A (en) * | 2022-05-06 | 2022-06-03 | 达而观数据(成都)有限公司 | Method and system for updating index data of search engine in real time |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102033910A (en) * | 2010-11-19 | 2011-04-27 | 福建富士通信息软件有限公司 | Enterprise search engine technology based on multiple data resources |
| CN102375860A (en) * | 2010-08-27 | 2012-03-14 | 盛乐信息技术（上海）有限公司 | Cloud-based structured data search method |
| CN103646093A (en) * | 2013-12-18 | 2014-03-19 | 北京博雅立方科技有限公司 | Data processing method and platform for search engines |
| US20140201195A1 (en) * | 2013-01-16 | 2014-07-17 | Google Inc. | Unified searchable storage for resource-constrained and other devices |
| EP2778987A1 (en) * | 2013-03-14 | 2014-09-17 | Open Text S.A. | Systems, methods and computer program products for information integration across disparate information systems |
| CN104778266A (en) * | 2015-04-22 | 2015-07-15 | 无锡天脉聚源传媒科技有限公司 | Multi-data source searching method and device |
| CN105183884A (en) * | 2015-09-24 | 2015-12-23 | 西安未来国际信息股份有限公司 | Search engine system and method based on big data technique |
| CN107103067A (en) * | 2017-04-18 | 2017-08-29 | 北京思特奇信息技术股份有限公司 | A kind of method of data synchronization and system based on search engine |
| CN107943820A (en) * | 2016-10-12 | 2018-04-20 | 阿里巴巴集团控股有限公司 | Searching method, device, terminal device and operating system |
| CN108228743A (en) * | 2017-12-18 | 2018-06-29 | 深圳供电局有限公司 | A kind of real-time big data search engine system |
| CN108363768A (en) * | 2018-02-07 | 2018-08-03 | 深圳壹账通智能科技有限公司 | A kind of document search method, storage medium and server based on Lucene |
| CN108874924A (en) * | 2018-05-31 | 2018-11-23 | 康键信息技术（深圳）有限公司 | Creation method, device and the computer readable storage medium of search service |
| CN110020063A (en) * | 2017-07-18 | 2019-07-16 | 北京京东尚科信息技术有限公司 | Method for vertical search and system |
| CN110825742A (en) * | 2019-10-31 | 2020-02-21 | 北京东方通科技股份有限公司 | Data paging retrieval method and device, electronic equipment and storage medium |
-
2020
- 2020-03-10 CN CN202010163274.8A patent/CN113378022A/en active Pending
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102375860A (en) * | 2010-08-27 | 2012-03-14 | 盛乐信息技术（上海）有限公司 | Cloud-based structured data search method |
| CN102033910A (en) * | 2010-11-19 | 2011-04-27 | 福建富士通信息软件有限公司 | Enterprise search engine technology based on multiple data resources |
| US20140201195A1 (en) * | 2013-01-16 | 2014-07-17 | Google Inc. | Unified searchable storage for resource-constrained and other devices |
| EP2778987A1 (en) * | 2013-03-14 | 2014-09-17 | Open Text S.A. | Systems, methods and computer program products for information integration across disparate information systems |
| CN103646093A (en) * | 2013-12-18 | 2014-03-19 | 北京博雅立方科技有限公司 | Data processing method and platform for search engines |
| CN104778266A (en) * | 2015-04-22 | 2015-07-15 | 无锡天脉聚源传媒科技有限公司 | Multi-data source searching method and device |
| CN105183884A (en) * | 2015-09-24 | 2015-12-23 | 西安未来国际信息股份有限公司 | Search engine system and method based on big data technique |
| CN107943820A (en) * | 2016-10-12 | 2018-04-20 | 阿里巴巴集团控股有限公司 | Searching method, device, terminal device and operating system |
| CN107103067A (en) * | 2017-04-18 | 2017-08-29 | 北京思特奇信息技术股份有限公司 | A kind of method of data synchronization and system based on search engine |
| CN110020063A (en) * | 2017-07-18 | 2019-07-16 | 北京京东尚科信息技术有限公司 | Method for vertical search and system |
| CN108228743A (en) * | 2017-12-18 | 2018-06-29 | 深圳供电局有限公司 | A kind of real-time big data search engine system |
| CN108363768A (en) * | 2018-02-07 | 2018-08-03 | 深圳壹账通智能科技有限公司 | A kind of document search method, storage medium and server based on Lucene |
| CN108874924A (en) * | 2018-05-31 | 2018-11-23 | 康键信息技术（深圳）有限公司 | Creation method, device and the computer readable storage medium of search service |
| CN110825742A (en) * | 2019-10-31 | 2020-02-21 | 北京东方通科技股份有限公司 | Data paging retrieval method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 朴岩，陈远平，及俊川: "基于统一搜索的信息服务平台", 计算机系统应用, vol. 19, no. 11, 15 November 2010 (2010-11-15), pages 132 - 138 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114579596A (en) * | 2022-05-06 | 2022-06-03 | 达而观数据(成都)有限公司 | Method and system for updating index data of search engine in real time |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108780463B (en) | Social graph generation and management | |
| CN110391966B (en) | Message processing method and device and message processing device | |
| CN112784142A (en) | Information recommendation method and device | |
| CN113705210A (en) | Article outline generation method and device for generating article outline | |
| CN111046210A (en) | Information recommendation method and device and electronic equipment | |
| CN111857806A (en) | Permission configuration file updating method and device, electronic equipment and storage medium | |
| US9569451B1 (en) | Folder creation application | |
| CN110633391A (en) | Information searching method and device | |
| CN113378022A (en) | In-station search platform, search method and related device | |
| CN109977293B (en) | Method and device for calculating search result relevance | |
| CN109522286B (en) | Processing method and device of file system | |
| CN110020082B (en) | Searching method and device | |
| CN116048757A (en) | Task processing method, device, electronic equipment and storage medium | |
| CN113128181B (en) | Information processing method and device | |
| CN107515853B (en) | Cell word bank pushing method and device | |
| CN110020206B (en) | Search result ordering method and device | |
| CN108509641B (en) | File backup method, device, server and system | |
| CN113946228A (en) | Statement recommendation method and device, electronic equipment and readable storage medium | |
| CN112988822A (en) | Data query method, device, equipment, readable storage medium and product | |
| CN112256445A (en) | Data processing method, device and equipment based on application program and storage medium | |
| CN110989987A (en) | Portal webpage generation method, portal webpage generation device, client, server and storage medium | |
| CN111694999A (en) | Information processing method and device and electronic equipment | |
| CN110119471B (en) | Method and device for checking consistency of search results | |
| CN113946346B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN112286609B (en) | Method and device for managing shortcut setting items of intelligent terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |