WO2019007010A1 - 分布式搜索及索引更新方法、系统、服务器及计算机设备 - Google Patents

分布式搜索及索引更新方法、系统、服务器及计算机设备 Download PDF

Info

Publication number
WO2019007010A1
WO2019007010A1 PCT/CN2017/120018 CN2017120018W WO2019007010A1 WO 2019007010 A1 WO2019007010 A1 WO 2019007010A1 CN 2017120018 W CN2017120018 W CN 2017120018W WO 2019007010 A1 WO2019007010 A1 WO 2019007010A1
Authority
WO
WIPO (PCT)
Prior art keywords
server
query
index
servers
proxy
Prior art date
Application number
PCT/CN2017/120018
Other languages
English (en)
French (fr)
Inventor
杜若飞
潘文彬
张锡伦
Original Assignee
北京三快在线科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京三快在线科技有限公司 filed Critical 北京三快在线科技有限公司
Priority to JP2019568703A priority Critical patent/JP6967615B2/ja
Priority to US16/622,298 priority patent/US20200210496A1/en
Priority to CA3065118A priority patent/CA3065118C/en
Publication of WO2019007010A1 publication Critical patent/WO2019007010A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/951—Indexing; Web crawling techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
    • H04L41/0668—Management of faults, events, alarms or notifications using network fault recovery by dynamic selection of recovery network elements, e.g. replacement by the most appropriate element after failure
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • H04L67/5682—Policies or rules for updating, deleting or replacing the stored data

Definitions

  • the present invention relates to a distributed search and index update method, system, server and computer device.
  • the search, index, and index maintenance programs can be placed on a single server, or the index can be split across multiple machines and managed by the engine.
  • the search for large amounts of concurrency is large, real-time expansion may not be possible.
  • more and more indexes are needed, and the operation and maintenance costs are also increasing, which affects online stability.
  • a distributed search system with a master slave architecture can be employed.
  • the primary server needs to be elected, when the primary server is abnormal and cannot work, the primary server needs to be re-elected, which may result in the search service being unavailable during the time of re-selecting the primary server, thereby affecting online stability.
  • the embodiments of the present invention provide a distributed search and index update method, system, server, and computer device.
  • a distributed search method comprising: at least one of a plurality of proxy servers obtaining an attribute corresponding to the query request when receiving a query request from a query server Information: the first proxy server queries the configuration management server based on the attribute information to obtain machine information corresponding to the attribute information; the first proxy server sends the at least two engine servers corresponding to the machine information Querying a request; the first proxy server obtains a first query result returned by the at least two engine servers according to the query request; the first proxy server according to at least two of the first query results according to a preset rule Merging into a second query result; and the first proxy server sends the second query result to the query server.
  • an index update method comprising: a master server obtaining a split rule from a configuration management server; the master server sending the split rule to an index creation server, so that the The index creation server splits the index data to be created according to the splitting rule; the master server obtains index configuration information that represents a result of the splitting; the master server obtains an index based on the index configuration information Data; the master server stores the index data in a corresponding at least two engine servers of the plurality of engine servers.
  • a proxy server including: a communication unit, configured to receive a query request from a query server; and a processing unit, configured to obtain attribute information corresponding to the query request, based on the attribute information Querying the configuration management server to obtain machine information corresponding to the attribute information, and determining at least two engine servers corresponding to the machine information.
  • the communication unit is further configured to send the query request to the at least two engine servers to obtain a first query result returned by the at least two engine servers according to the query request.
  • the processing unit is further configured to combine at least two of the first query results according to a preset rule to obtain a second query result.
  • the communication unit is further configured to send the second query result to the query server.
  • a master control server includes: a master control module, configured to obtain a split rule from a configuration management server; and a notification module, configured to send the split rule to an index creation server, to And causing the index creation server to split the index data to be created according to the splitting rule, and obtain index configuration information that represents a result of the splitting.
  • the main control module is further configured to acquire index data based on the index configuration information, and store the index data into at least two engine servers corresponding to the plurality of engine servers.
  • a distributed search system comprising: a configuration management server for managing configuration information and machine information, the configuration information including a split rule, the machine information characterizing the plurality of Information of the engine server; a query server for obtaining a query request of the terminal device; a plurality of proxy servers; and a plurality of engine servers, each of the plurality of engine servers for storing the satisfying the splitting rule Index data.
  • the at least one of the plurality of proxy servers receives the query request from the query server, querying the configuration management server based on the attribute information of the query request, determining the plurality of At least two first engine servers of the engine servers, and sending the query request to the at least two first engine servers.
  • the at least two first engine servers each reply to the first query result in response to receiving the query request.
  • the at least one first proxy server merges at least two of the first query results into a second query result and sends to the query server to return the second query result to the query by the query server Terminal Equipment.
  • a computer apparatus comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor executes the program The steps of the above distributed search method.
  • a computer apparatus comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor is implemented when the program is executed.
  • a plurality of proxy servers are coupled to the distributed architecture of the query server and the engine server, and the query request of the query server may be sent to at least one proxy server of the plurality of proxy services, and the at least one proxy server
  • the query results are obtained in at least two corresponding engine servers. Since multiple proxy servers have a parallel relationship with each other, one proxy server cannot work and can work through other proxy servers, which can effectively prevent the search service from being unavailable for a short time due to the need to reselect the primary device when the primary device fails to work.
  • the proxy server since the configuration management server, the index creation server, and the engine server are linked and updated by the main control server, the proxy server does not need to undertake update and maintenance of the index, which can greatly reduce the burden on the proxy server.
  • FIG. 1 is a schematic flowchart diagram of an index update method according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of an application architecture and data interaction of a distributed search method according to an embodiment of the present invention.
  • FIG. 3 is a schematic flowchart diagram of a distributed search method according to another embodiment of the present invention.
  • FIG. 4 is a schematic diagram of an application architecture and data interaction of a distributed search method according to another embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a proxy server according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a structure of a main control server according to an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
  • FIG. 1 is a schematic flowchart diagram of an index update method according to an embodiment of the present invention. As shown in FIG. 1, the method may include:
  • Step 101 The master server sends the split rule obtained from the configuration management server to the index creation server, so that the index creation server splits the index data to be created into multiple index configuration information according to the split rule.
  • Step 102 The master server obtains index configuration information from the index creation server.
  • Step 103 The master server acquires index data based on the index configuration information.
  • Step 104 The master server stores the index data in a corresponding first engine server of the plurality of engine servers to update index data stored in the first engine server.
  • the index update method in this embodiment may be applied to a master server, which may be a server or a server cluster.
  • the main control server may include a main control module 210, a notification module 220, and a plurality of client modules 230.
  • the main control module 210 can be responsible for unified scheduling and communicate with the configuration management server 240.
  • the notification module 220 can be responsible for notifying the index creation server 250.
  • the number of the notification modules 220 may be one or more.
  • the plurality of notification modules 220 may distinguish based on the type of the service to respectively notify the related information created by the index belonging to the corresponding service type.
  • the number of the client modules 230 may be the same as the number of the engine servers 260, and each client module 230 corresponds to an engine server 260, and the client module 230 may be configured to pull index data according to the indication of the master module 210. The pulled index data is stored in the corresponding engine server 260.
  • the main control module 210 and each notification module 220 can be implemented by a separate server.
  • the client module 230 can be located in the corresponding engine server 260 and implement corresponding functions through the corresponding engine server 260. In an actual application, the main control module 210 can configure an alternate main control module.
  • Each notification module 220 can also configure a corresponding alternate notification module.
  • Each client module 230 can also configure a corresponding alternate client module. In this way, when the main module fails to work, the corresponding function can be continued through the corresponding standby module.
  • the configuration management server 240 is for managing configuration information and machine information.
  • the machine information may characterize information of the plurality of engine servers 260.
  • the information of the engine server 260 may include information such as an IP address and a port of the engine server.
  • the machine information may be represented by a list of machines containing information of the engine server 260 described above.
  • the configuration information may include at least a service identifier, machine configuration information, configured rule information, and the like.
  • the machine configuration information may specifically include a machine list, that is, information including an IP address and a port of the engine server 260.
  • the rule information includes any operation rules required in the search process, and at least includes a split rule required for index creation, an index creation rule, and notification information of which one or several types of service types are to be created by the notification module.
  • the configuration rules, the configuration rules of the index data corresponding to which the client module is pulled, or the like, are of course not limited to the above rules.
  • the master server obtains the splitting rules in the configuration information from the configuration management server 240, and sends the splitting rules to the index creation server 250.
  • the specific control module 210 obtains the splitting rules from the configuration management server 240.
  • An index creation rule is also included; the main control module 210 sends the split rule and the index creation rule to the notification module 220; the notification module 220 sends the split rule and the index creation rule to an index creation server 250.
  • the number of the notification module 220 may be multiple according to different types of service types, and the main control module 210 may obtain and refer to the configuration management server 240 according to the service type of the index data to be created.
  • a splitting rule and an index creation rule that match the service type, and send the split rule and the index creation rule to the notification module 220 that matches the service type.
  • the notification module 220 sends the split rule and the index creation rule to the index creation server 250.
  • the index creation server 250 creates an index according to the index creation rule, and further splits the created index data according to the split rule.
  • the splitting parameter may include a splitting parameter, and the splitting parameter may specifically include a splitting quantity, where the splitting quantity is used to represent the splitting quantity of the index data. For example, when the number of splits is N, the representation splits the index data into N index sub-data. N may be a positive integer greater than or equal to 2 to indicate that the created index data is distributed and stored in at least two engine servers 260.
  • search_online_dis_1 and search_online_dis_2 are a complete index of data that can be split into two indexed data according to the splitting rules. The two index data can be stored in an engine server separately.
  • the index creation server 250 may generate index configuration information based on the created and split index data.
  • the index configuration information may include a plurality, or the index configuration information may include a plurality of index configuration sub-information.
  • the plurality of index configuration information or the plurality of index configuration sub-informations may represent a split result of the index data, and include an engine server 260 corresponding to each split index sub-data.
  • Each index configuration information or each index configuration sub-information is used to indicate which index data the corresponding engine server 260 acquires and stores.
  • the notification module 220 can obtain the index configuration information, and send the index configuration information to the main control module 210, so that the main control module 210 further instructs the corresponding client module 230 to pull the index data. .
  • the main control server acquires the index data based on the index configuration information, and stores the index data in the at least two engine servers, which may include: the main control module 210 indicates the first client based on the index configuration information.
  • the module 230 obtains index data; the first client module 230 is any client module corresponding to the split result included in the index configuration information; the first client module 230 stores the acquired index data.
  • the main control module 210 may indicate the first client module 230 corresponding to the engine server 260 according to any index configuration information or an engine server 260 corresponding to the index data included in any index configuration sub-information. In this way, the first client module 230 can pull the corresponding index data based on the indication of the main control module 210, and store the extracted index data into the engine server 260 corresponding to the first client module 230. .
  • the distributed search method in this embodiment is a pull update process of index data, which can be specifically used as an offline data processing process.
  • the data processing process combined with each server and each module can be as follows:
  • Step 21 The main control module 210 obtains a split rule and an index creation rule from the configuration management server 240.
  • the main control module 210 may obtain a splitting rule and an index creation rule that match the service type according to the service type of the index data to be created.
  • Step 22 The main control module 210 sends the splitting rule and the index creation rule to the notification module 220.
  • Step 23 The notification module 220 sends the split rule and the index creation rule to the index creation server 250.
  • the index creation server 250 may create index data according to an index creation rule, and split the index data into N index sub-data according to the split rule. Moreover, the index creation server 250 may generate a plurality of index configuration information or a plurality of index configuration sub-informments based on the created and split index data. Each of the index configuration information or each of the index configuration sub-information may represent a split result of the index data, and include an engine server 260 corresponding to each split index data, so that the corresponding engine server 260 may be represented. Corresponding to which index data is obtained and stored.
  • Step 24 The index creation server 250 sends the index configuration information to the notification module 220.
  • Step 25 The notification module 220 sends the index configuration information to the main control module 210.
  • the number of the notification modules 220 may be multiple.
  • the plurality of notification modules 220 can perform function configuration according to the service type, that is, different notification modules perform notification functions of the corresponding service types.
  • the main control module 210 can obtain the splitting rule and the index creation rule according to the service type, and send the obtained splitting rule and the index creating rule to the notification module 220 that matches the service type.
  • the index creation server 250 can send the index configuration information to the notification module 220 that matches the type of service. It can be understood that multiple notification modules can work in parallel.
  • Step 26 The main control module 210 instructs the client module 230 according to the index configuration information.
  • the main control module 210 may indicate that the engine server 260 corresponds to any index configuration information of the plurality of index configuration information or the index data corresponding to the index data included in any index configuration sub-information.
  • Client module 230 In this way, the client module 230 can pull the corresponding index data based on the indication of the main control module 210, and store the extracted index data into the engine server 260 corresponding to the client module 230.
  • the engine server 260 only needs to load the corresponding index data, and the update function of the index data is mainly implemented by the main control server (specifically, the client module in the main control server), which can be greatly reduced.
  • the index data distribution is stored in the plurality of engine servers 260, and the memory usage of the engine server 260 can be greatly reduced during the search process, thereby effectively improving the efficiency of the search, reducing the response time of the search, and improving the operation experience of the user.
  • FIG. 3 is a schematic flow chart of a distributed search method according to another embodiment of the present invention. As shown in FIG. 3, the method can include:
  • Step 301 The first proxy server of the multiple proxy servers receives the query request of the query server, and obtains the attribute information corresponding to the query request.
  • Step 302 The first proxy server queries the configuration management server based on the attribute information to obtain machine information corresponding to the attribute information.
  • Step 303 The first proxy server sends the query request to at least two engine servers corresponding to the machine information to obtain a first query result returned by each of the at least two engine servers according to the query request.
  • Step 304 The first proxy server merges at least two of the first query results into a second query result according to a preset rule.
  • Step 305 The first proxy server sends the second query result to the query server.
  • FIG. 4 is a schematic diagram of an application architecture and data interaction of a distributed search method according to an embodiment of the present invention. As shown in FIG. 4, in the embodiment, the number of proxy servers is two as an example.
  • the query server 410 may send the query request to at least one of the plurality of proxy servers 420 according to a preset rule.
  • the preset rule may be a polling rule or a random rule or the like.
  • a plurality of proxy servers 420 may be numbered in advance, and the polling rules may be sequentially selecting one or more proxy servers as the first to be sent a query request based on the number order of the plurality of proxy servers 420.
  • Proxy server For example, to send a query request to a proxy server, when the query server 410 receives the first query request, the first query request may be sent to the proxy server 420 numbered 1; the query server 410 receives the second query.
  • the second query request can be sent to the proxy server 420 numbered 2, and so on.
  • the first query request and the second query request may be determined according to a data receiving time.
  • the random rule may be that the received query request is sent to the corresponding at least one proxy server 420 according to a preset random algorithm.
  • the first proxy server 421 obtains the attribute information corresponding to the query request, and the attribute information may be a service type corresponding to the query request, so that the device information may be requested from the configuration management server 240 based on the service type.
  • the configuration management server 240 For details of the configuration management server 240, refer to the above, and details are not described herein again. Further, based on the above description, when the index data in the engine server 260 is updated and stored, the index data can be split based on the split rule. Therefore, index data belonging to the same service type may be stored in at least two engine servers 260.
  • the first proxy server queries the configuration management server 240 to obtain the machine information corresponding to the attribute information.
  • the identifier of the at least two engine servers 260 may indicate that the corresponding index data in the query request is stored in the at least two engine servers.
  • the machine information can be implemented by a machine list. Therefore, the first proxy server 421 may send the query request to the corresponding at least two engine servers 260 according to the machine information to obtain keywords or keywords, associated keywords or associations included in the query request.
  • the index data corresponding to the keyword may be implemented by a machine list.
  • the first proxy server 421 obtains the first query result returned by the at least two engine servers 260, and may include: the first proxy server 421 obtains a first query that satisfies a pre-configured truncation parameter. result.
  • the truncation parameter indicates the number of index data in the query result returned by any engine server 260. For example, if the query result obtained by an engine server 260 includes 1000 index data and the truncation parameter is 600, the engine server 260 returns the first 600 index data of the 1000 index data. This greatly reduces search latency and improves query rate per second (QPS, Query Per Second).
  • the truncation parameter can be configured by the configuration management server 240, and the truncation parameter is obtained by the main control module 210 in the main control server, and sent to each engine server 260 for configuration.
  • the first proxy server 421 obtains the first query result returned by the at least two engine servers 260, and merges and sorts the obtained at least two first query results according to a preset rule to generate a second query result. And sending the second query result to the query server 410, and the query server 410 sends the result to the terminal device to output the display to the user.
  • the distributed search method of this embodiment is a search query process, which can be specifically used as an online data processing process.
  • the data processing process combined with each server is as follows:
  • Step 41 The query server 410 obtains a query request of the terminal device.
  • Step 42 The query server 410 sends the query request to at least one of the plurality of proxy servers 420.
  • the first proxy server 421 may be a proxy server corresponding to the service type of the query request in the plurality of proxy servers 420, or may be a proxy server determined based on a preset rule (for example, a polling rule or a random rule, etc.).
  • the query server 410 may analyze the received query request, on the one hand, obtain keywords or keywords in the query request, and on the other hand, obtain the association relationship between the keywords or keywords.
  • the associated keyword or associated keyword that is, the intent recognition of the query request. For example, if the keyword or keyword included in the query request is a restaurant name, the keyword or keyword may be intently identified to obtain an associated keyword or the associated keyword is a order or take-out.
  • the keyword or the keyword included in the query request is a character string, and the pin code of the Chinese word is determined by the intention identification, and the corresponding associated keyword or the associated keyword may be the Chinese word, etc. .
  • the query server 410 may further generate at least one query request based on the at least one keyword obtained after the intent recognition, and send the at least one query request to the corresponding at least one first proxy server 421.
  • Step 43 The first proxy server 421 requests the configuration management server 240 for the machine list based on the attribute information (for example, the service type) of the query request, so as to obtain the information of the engine server 260 where the index data corresponding to the query request is located.
  • the attribute information for example, the service type
  • Step 44 The first proxy server 421 sends a query request based on at least two engine servers 260 corresponding to the obtained machine list.
  • Step 45 The at least two engine servers 260 load the index data based on the content in the query request, and return the query result to the first proxy server 421.
  • the engine server 260 can control the number of index data in the query result based on the pre-configured truncation parameters, thereby reducing the query delay and improving the QPS.
  • Step 46 The first proxy server 421 merges and sorts the obtained at least two query results according to a preset rule to generate a final query result, and sends the final query result to the query server 410.
  • Step 47 The query server 410 sends the final query result to the terminal device for the terminal device to output and display to the user.
  • the query request of the query server 410 can be sent to the corresponding at least one proxy server 420, from the at least one proxy server 420 from the respective at least two engine servers 260. Get the query results.
  • the plurality of proxy servers 420 may have the same function, and the plurality of proxy servers 420 may have a parallel relationship with each other. In this way, one proxy server 420 can not work and can also work through other proxy servers 420, which can effectively avoid the problem that the search service is unavailable in a short time due to the need to reselect the primary device when the primary device fails to work.
  • the proxy server 420 does not need to undertake update maintenance of the index, the burden on the proxy server 420 is greatly reduced.
  • the embodiment of the present invention further provides a distributed search system, which may be specifically referred to FIG. 4 and FIG. 2, and the system may include a configuration management server 240, a query server 410, a plurality of proxy servers 420, and more.
  • Engine server 260 may be specifically referred to FIG. 4 and FIG. 2, and the system may include a configuration management server 240, a query server 410, a plurality of proxy servers 420, and more.
  • Engine server 260 may be specifically referred to FIG. 4 and FIG. 2, and the system may include a configuration management server 240, a query server 410, a plurality of proxy servers 420, and more.
  • Engine server 260 may be specifically referred to FIG. 4 and FIG.
  • the configuration management server 240 can be used to manage configuration information and machine information.
  • the configuration information may include a split rule.
  • the machine information may characterize information of the plurality of engine servers.
  • the proxy server 420 is configured to obtain the attribute information corresponding to the query request when receiving the query request sent by the query server 410, and query the configuration management server 240 to obtain the attribute information based on the attribute information. Corresponding machine information, so that the query request can be sent to at least two engine servers 260 corresponding to the machine information. And, after obtaining the first query result returned by the at least two engine servers 260, the proxy server 420 may merge the at least two first query results into a second query result according to a preset rule, and the second query The result is sent to the query server 410.
  • the query server 410 is configured to send the query request to the proxy server 420 when obtaining a query request of the terminal device, and send the second query result to the second query result when the second query result is received Terminal Equipment.
  • Each of the plurality of engine servers 260 is operative to store index data that satisfies the splitting rules and to reply to the first query result upon receipt of the query request.
  • the system may further include a master server and an index creation server 250.
  • the master server may be configured to obtain a split rule from the configuration management server 240 and send the split rule to the index creation server 250.
  • the master server may be further configured to obtain index configuration information that is sent by the index creation server 250 to represent the split result, acquire index data based on the index configuration information, and store the index data in a plurality of engine servers 260. At least two of the first engine servers.
  • the index creation server 250 is configured to split the index data to be created based on the splitting rule, and send index configuration information that represents the split result to the master server.
  • the proxy server 420 may obtain a query result that satisfies a pre-configured truncation parameter.
  • the distributed search system of the embodiment of the present invention connects the main search server, the configuration management server, the index creation server, the query server, and the engine server through a plurality of proxy servers, and the query function and the index update maintenance function respectively pass through the proxy server and
  • the implementation of the main control server can greatly improve the scalability of the distributed search system and the stability of the system. In practical applications, the first 50% time query delay is reduced by 50%, the first 90% time query delay is reduced by 54.5%, and the first 99% time query delay is reduced by 46%, which improves the user experience. .
  • FIG. 5 is a schematic structural diagram of a proxy server according to an embodiment of the present invention.
  • the proxy server may include a communication unit 51 and a processing unit 52.
  • the communication unit 51 can be configured to receive a query request from the query server and send the query request to the processing unit 52.
  • the communication unit 51 is further configured to send the query request to at least two engine servers determined by the processing unit 52, obtain a first query result returned by the at least two engine servers, and merge the processing unit 52 The second query result is sent to the query server.
  • the processing unit 52 is configured to obtain attribute information corresponding to the query request, and query a configuration management server based on the obtained attribute information to obtain the at least two engine servers corresponding to the attribute information.
  • the processing unit 52 may also combine at least two first query results obtained by the communication unit 51 according to a preset rule to obtain a second query result.
  • the communication unit 51 can obtain a query result that satisfies a pre-configured truncation parameter.
  • the processing unit 52 in the proxy server may be a central processing unit (CPU) in the proxy server, a digital signal processor (DSP), a micro control unit (MCU), or Field-Programmable Gate Array (FPGA) implementation.
  • the communication unit 51 in the proxy server can be implemented by a communication module (including: a basic communication suite, an operating system, a communication module, a standardized interface and a protocol, etc.) and a transceiver antenna.
  • FIG. 6 is a schematic structural diagram of a main control server according to an embodiment of the present invention.
  • the main control server may include a main control module 61 and a notification module 62.
  • the main control module 61 is configured to obtain a splitting rule from the configuration management server, and send the splitting rule to the notification module 62.
  • the main control module 61 may further acquire index data based on the index configuration information sent by the notification module 62, and store the index data in at least two engine servers corresponding to the plurality of engine servers.
  • the notification module 62 is configured to send the splitting rule to the index creation server, so that the index creation server splits the index data to be created according to the splitting rule.
  • the notification module 62 may also obtain index configuration information that characterizes the split result, and send the index configuration information to the main control module 61.
  • the main control server may further include multiple client modules 63.
  • the plurality of client modules 63 can be in one-to-one correspondence with a plurality of engine servers.
  • the client module 63 may be configured to obtain the index data by the client module 63 corresponding to the split result included in the index configuration information, based on the index configuration information sent by the notification module 62.
  • the main control module 61 may include: a first communication submodule, configured to communicate with the configuration management server to obtain the splitting rule from the configuration management server; and a second communication submodule, Used to communicate with the notification module 62 to send the splitting rule to the notification module 62 and obtain the index configuration information from the notification module 62; and a third communication sub-module for interacting with the client module 63 communicating to instruct the client module 63 to acquire the index data based on the index configuration information.
  • the notification module 62 is configured to send the splitting rule to the index creation server, and send the index configuration information to the main control module 61 after obtaining the index configuration information that represents the split result.
  • the notification module 62 may include: a first communication module, configured to communicate with the main control module 61 to obtain the splitting rule from the main control module 61 and to the main control module Sending the index configuration information; and the second communication module is configured to communicate with the index creation server to send the split rule to the index creation server and obtain the index configuration from the index creation server information.
  • the client module 63 may acquire index data based on the indication of the main control module 61 and store the index data into a corresponding engine server.
  • the client module 63 may include: a first communication submodule for instructing the main control module 61 to accept the indication of the main control module 61; and a processing module for responding to the The indication of the main control module 61 acquires index data based on the index configuration information; and the second communication sub-module is configured to communicate with the engine server to store the index data into a corresponding engine server.
  • the master server may be a server cluster.
  • the main control module 61 is mainly responsible for unified scheduling.
  • the notification module 62 is primarily responsible for communicating with the index creation server.
  • the number of the notification modules 62 may be at least one.
  • the at least one notification module 62 can distinguish based on the type of the service, for example, each notification module 62 is configured to notify relevant information of the index creation belonging to the corresponding service type.
  • the number of the client modules 63 may be the same as the number of engine servers, and each client module 63 corresponds to one engine server.
  • the client module 63 can be configured to pull the index data according to the indication of the main control module 61, and store the pulled index data into the corresponding engine server.
  • the main control module 61 and each notification module 62 can be implemented by a separate server.
  • the client module 63 can be located in a corresponding engine server and implement corresponding functions through a corresponding engine server.
  • the main control module 61 can configure a standby main control module; each notification module 62 can also configure a corresponding alternate notification module.
  • Each client module 63 can also configure a corresponding alternate client module 63. In this way, when the main module fails to work, the corresponding function can be continued through the corresponding standby module.
  • FIG. 7 is a schematic structural diagram of a computer device according to an embodiment of the present invention, as shown in FIG. 7 .
  • the computer device can include a processor 71, a memory 72, and at least one external communication interface 73.
  • the processor 71, the memory 72, and the external communication interface 73 can all be connected by a bus 74.
  • a computer program executable on the processor 71 is stored in the memory 72.
  • the processor 71 When the computer device is acting as a proxy server, the processor 71 performs the following steps: receiving a query request from the query server; obtaining attribute information corresponding to the query request; and querying based on the attribute information Configuring a management server, obtaining machine information corresponding to the attribute information; sending the query request to at least two engine servers corresponding to the machine information; obtaining a first query returned by the at least two engine servers according to the query request a result; combining at least two of the first query results into a second query result according to a preset rule; and transmitting the second query result to the query server.
  • the processor 71 can implement the specific functions of the communication unit 51 and the processing unit 52 in the proxy server shown in FIG. 5 by executing a program.
  • the processor 71 executes the program, the following steps are implemented: obtaining a query result that satisfies a pre-configured truncation parameter.
  • the processor 71 implements the following steps: obtaining a split rule from the configuration management server; sending the split rule to the index creation server, so that the index creation server is based on The splitting rule splits the index data to be created; obtains index configuration information that represents the split result; acquires index data based on the index configuration information; and stores the index data in a corresponding one of the plurality of engine servers At least two of the first engine servers.
  • the processor 71 can implement the specific functions of the main control module 61, the notification module 62, and the client module 63 in the main control server shown in FIG. 6 by executing a program.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner such as: multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored or not executed.
  • the coupling, or direct coupling, or communication connection of the components shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical, mechanical or other forms. of.
  • the units described above as separate components may or may not be physically separated, and the components displayed as the unit may or may not be physical units, that is, may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • the various servers described above may be a physical hardware machine or a software module running in a server cluster.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated into one unit;
  • the unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
  • the foregoing program may be stored in a computer readable storage medium, and the program is executed when executed.
  • the foregoing storage device includes the following steps: the foregoing storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
  • ROM read-only memory
  • RAM random access memory
  • magnetic disk or an optical disk.
  • optical disk A medium that can store program code.
  • the above-described integrated unit of the present invention may be stored in a computer readable storage medium if it is implemented in the form of a software function module and sold or used as a standalone product.
  • the technical solution of the embodiments of the present invention may be embodied in the form of a software product in essence or in the form of a software product stored in a storage medium, including a plurality of instructions.
  • a computer device (which may be a personal computer, server, or network device, etc.) is caused to perform all or part of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes various media that can store program codes, such as a mobile storage device, a ROM, a RAM, a magnetic disk, or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种分布式搜索及索引更新方法、系统、服务器及计算机设备。多个代理服务器中的至少一个代理服务器在接收到查询服务器转发的来自终端设备的查询请求后，可根据所述查询请求对应的属性信息查询配置管理服务器，以获得所述属性信息对应的机器信息，并向所述机器信息对应的至少两个引擎服务器发送所述查询请求。并且，所述至少一个代理服务器在获得所述至少两个引擎服务器各自响应于所述查询请求而返回的第一查询结果后，可将至少两个所述第一查询结果按照预设规则合并为第二查询结果，并将所述第二查询结果发送至所述查询服务器，以由所述查询服务器将所述第二查询结果返回给所述终端设备。

Description

分布式搜索及索引更新方法、系统、服务器及计算机设备
相关申请的交叉引用
本专利申请要求于2017年7月5日提交的、申请号为201710540135.0、发明名称为“分布式搜索及索引更新方法、系统、服务器及计算机设备”的中国专利申请的优先权，该申请的全文以引用的方式并入本文中。
技术领域
本发明涉及分布式搜索及索引更新方法、系统、服务器及计算机设备。
背景技术
随着移动互联网的发展，人们可以方便地通过移动设备访问网络，获取网络服务，由此而兴起了一批线上到线下(O2O，Online-to-Offline)本地生活化服务。但随着业务的爆发性增长，搜索引擎需要查询的数据量越来越大，单机内存已存储不下这些数据，使得系统稳定性越来越低，而且查询请求的延时也越来越大，使得用户体验也越来越差。
可将搜索、索引和索引维护程序放在一台服务器上，或者将索引拆分到多台机器上并由引擎这块负责管理索引。然而，当搜索并发量大的情况下，可能无法进行实时扩展。并且，随着业务量越来越多，需要的索引数也越来越多，运维成本也越来越大，并影响线上稳定性。
可采用主(master)从(slave)结构的分布式搜索系统。然而，由于主服务器需要选举，当主服务器出现异常无法工作时，需要重新选举主服务器，这就可能导致在重新选取主服务器的时间内搜索服务不可用，从而影响了线上稳定性。
发明内容
为解决现有存在的技术问题，本发明实施例提供一种分布式搜索及索引更新方法、系统、服务器及计算机设备。
根据本发明的第一方面，提供了一种分布式搜索方法，包括：多个代理服务器中的至少一个第一代理服务器在接收到来自查询服务器的查询请求时，获得所述查询请求对应的属性信息；所述第一代理服务器基于所述属性信息查询配置管理服务器，以获得所述属性信息对应的机器信息；所述第一代理服务器向所述机器信息对应的至少两个引擎服务器发送所述 查询请求；所述第一代理服务器获得所述至少两个引擎服务器根据所述查询请求而返回的第一查询结果；所述第一代理服务器将至少两个所述第一查询结果按照预设规则合并为第二查询结果；以及所述第一代理服务器将所述第二查询结果发送至所述查询服务器。
根据本发明的第二方面，提供了一种索引更新方法，包括：主控服务器从配置管理服务器获得拆分规则；所述主控服务器发送所述拆分规则至索引创建服务器，以使所述索引创建服务器根据所述拆分规则将待创建的索引数据进行拆分；所述主控服务器获得表征所述拆分的结果的索引配置信息；所述主控服务器基于所述索引配置信息获取索引数据；所述主控服务器将所述索引数据存储至多个引擎服务器中对应的至少两个引擎服务器中。
根据本发明的第三方面，提供了一种代理服务器，包括：通讯单元，用于接收来自查询服务器的查询请求；处理单元，用于获得所述查询请求对应的属性信息，基于所述属性信息查询配置管理服务器以获得所述属性信息对应的机器信息，并确定所述机器信息对应的至少两个引擎服务器。其中，所述通讯单元还用于向所述至少两个引擎服务器发送所述查询请求，以获得所述至少两个引擎服务器根据所述查询请求而返回的第一查询结果。所述处理单元还用于将至少两个所述第一查询结果按照预设规则进行合并，得到第二查询结果。所述通讯单元还用于将所述第二查询结果发送至所述查询服务器。
根据本发明的第四方面，提供了一种主控服务器，包括：主控模块，用于从配置管理服务器获得拆分规则；通知模块，用于发送所述拆分规则至索引创建服务器，以使所述索引创建服务器根据所述拆分规则将待创建的索引数据进行拆分，并获得表征所述拆分的结果的索引配置信息。其中，所述主控模块还用于基于所述索引配置信息获取索引数据，并将所述索引数据存储至多个引擎服务器中对应的至少两个引擎服务器中。
根据本发明的第五方面，提供了一种分布式搜索系统，包括：配置管理服务器，用于管理配置信息和机器信息，所述配置信息包括拆分规则，所述机器信息表征所述多个引擎服务器的信息；查询服务器，用于获得终端设备的查询请求；多个代理服务器；以及多个引擎服务器，所述多个引擎服务器中的每个引擎服务器用于存储满足所述拆分规则的索引数据。其中，所述多个代理服务器中的至少一个第一代理服务器接收到来自所述查询服务器的所述查询请求后，通过基于所述查询请求的属性信息查询所述配置管理服务器，确定所述多个引擎服务器中的至少两个第一引擎服务器，并向所述至少两个第一引擎服务器发送所述查询请求。所述至少两个第一引擎服务器各自响应于收到所述查询请求回复第一查询结果。所述至少一个第一代理服务器将至少两个所述第一查询结果合并为第二查询结果，并发送至所述查询服务器，以由所述查询服务器将所述第二查询结果返回至所述终端设备。
根据本发明的第六方面，提供了一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时实现上述分布式搜索方法的步骤。
根据本发明的第七方面，提供了一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时实现上述索引更新方法的步骤。
采用本发明实施例的技术方案，通过多个代理服务器联结查询服务器和引擎服务器的分布式架构，查询服务器的查询请求可被发送至多个代理服务中的至少一个代理服务器，由至少一个代理服务器从相应的至少两个引擎服务器中获得查询结果。由于多个代理服务器相互之间具有并行关系，一个代理服务器无法工作还可通过其他代理服务器工作，可有效避免主设备无法工作时因需要重新选择主设备而导致的短时间内搜索服务不可用。另外，由于通过主控服务器联结配置管理服务器、索引创建服务器和引擎服务器进行索引数据的更新维护，代理服务器无需承担索引的更新维护，可大大降低了代理服务器的负担。
附图说明
图1为本发明一实施例的索引更新方法的流程示意图。
图2为本发明一实施例中的分布式搜索方法的应用架构以及数据交互示意图。
图3为本发明另一实施例的分布式搜索方法的流程示意图。
图4为本发明另一实施例中的分布式搜索方法的应用架构以及数据交互示意图。
图5为本发明一实施例的代理服务器的组成结构示意图。
图6为本发明一实施例的主控服务器的组成结构示意图。
图7为本发明实施例的计算机设备的组成结构示意图。
具体实施方式
下面结合附图及具体实施例对本发明作进一步详细的说明。
图1为本发明一实施例的索引更新方法的流程示意图。如图1所示，所述方法可包括：
步骤101：主控服务器将从配置管理服务器获得的拆分规则发送至索引创建服务器，以使索引创建服务器根据所述拆分规则将待创建的索引数据拆分为多个索引配置信息。
步骤102：所述主控服务器从所述索引创建服务器获得索引配置信息。
步骤103：所述主控服务器基于所述索引配置信息获取索引数据。
步骤104，所述主控服务器将所述索引数据存储至多个引擎服务器中对应的第一引擎服务器中，以更新所述第一引擎服务器中存储的索引数据。
本实施例的索引更新方法可应用于主控服务器中，所述主控服务器具体可以是服务器或服务器集群。在一实施例中，所述主控服务器可包括主控模块210、通知模块220和多个客户端模块230，具体可参照图2所示。所述主控模块210可负责统一调度，并与配置管理服务器240通信。所述通知模块220可负责通知索引创建服务器250。所述通知模块220的数量可以是一个或多个。例如，多个通知模块220可基于业务类型的不同进行区分，以分别用于通知属于相应业务类型的索引创建的相关信息。所述客户端模块230的数量可与引擎服务器260的数量相同，且每个客户端模块230对应一个引擎服务器260，所述客户端模块230可用于根据主控模块210的指示拉取索引数据，将拉取的索引数据存储至对应的引擎服务器260中。所述主控模块210、每个通知模块220可通过独立的服务器实现。所述客户端模块230可位于对应的引擎服务器260中，并通过对应的引擎服务器260实现相应功能。在实际应用中，所述主控模块210可配置备用主控模块。每个通知模块220也可配置对应的备用通知模块。每个客户端模块230也可配置对应的备用客户端模块。这样，主模块出现异常无法工作时，可通过相应的备用模块继续执行相应的功能。
配置管理服务器240用于管理配置信息和机器信息。所述机器信息可表征所述多个引擎服务器260的信息。其中，引擎服务器260的信息可包括引擎服务器的IP地址和端口等信息。作为一种示例，所述机器信息可通过包含有上述引擎服务器260的信息的机器列表表示。所述配置信息可至少包括业务标识、机器配置信息以及配置的规则信息等等。其中，所述机器配置信息可具体包括机器列表，也即包括引擎服务器260的IP地址和端口等信息。所述规则信息包括在搜索过程中所需要的任何操作规则，至少包括索引创建时所需要的拆分规则、索引创建规则、通知模块执行哪一种或几种业务类型的索引创建的通知信息的配置规则、客户端模块执行拉取哪一种或几种业务类型对应的索引数据的配置规则等等，当然不限于上述几种规则。
主控服务器从配置管理服务器240中获得配置信息中的拆分规则，将所述拆分规则发送至索引创建服务器250，可具体包括：主控模块210从配置管理服务器240获得拆分规则，当然也包括索引创建规则；主控模块210发送所述拆分规则和所述索引创建规则至所述通知模块220；所述通知模块220发送所述拆分规则和所述索引创建规则至索引创建服务器250。 其中，所述通知模块220的数量可依据业务类型的不同配置有多个，则所述主控模块210可依据待创建的索引数据的业务类型，从所述配置管理服务器240中获得与所述业务类型相匹配的拆分规则和索引创建规则，并将所述拆分规则和所述索引创建规则发送至与所述业务类型相匹配的通知模块220。所述通知模块220再将所述拆分规则和所述索引创建规则发送至所述索引创建服务器250。
所述索引创建服务器250依据所述索引创建规则创建索引，进一步依据所述拆分规则将创建的索引数据进行拆分。其中，所述拆分规则中可包括拆分参数，所述拆分参数具体可包括拆分数量，所述拆分数量用于表征索引数据的拆分数量。例如，所述拆分数量为N时，表征将索引数据拆分为N份索引子数据。N可为大于等于2的正整数，以表示创建的索引数据会分布存储在至少两个引擎服务器260中。例如，search_online_dis_1和search_online_dis_2是一份完整的索引数据，可依据拆分规则将其水平拆分为两个索引数据。所述两个索引数据可分别存储在一引擎服务器中。
进一步地，所述索引创建服务器250可基于创建完成、且拆分后的索引数据生成索引配置信息。所述索引配置信息可包括多个，或者所述索引配置信息可包括多个索引配置子信息。多个所述索引配置信息或所述多个索引配置子信息可表征索引数据的拆分结果，以及包括拆分后的每个索引子数据对应的引擎服务器260。每个索引配置信息或者每个索引配置子信息用于表示相应的引擎服务器260对应获取并存储哪些索引数据。则所述通知模块220可获得索引配置信息，并将所述索引配置信息发送至所述主控模块210，以由所述主控模块210进一步指示相应的客户端模块230进行索引数据的拉取。
所述主控服务器基于所述索引配置信息获取索引数据，将所述索引数据存储至至少两个引擎服务器中，可具体包括：所述主控模块210基于所述索引配置信息指示第一客户端模块230获取索引数据；所述第一客户端模块230为所述索引配置信息中包括的拆分结果对应的任一客户端模块；所述第一客户端模块230将获取的所述索引数据存储至与所述第一客户端模块230对应的引擎服务器260中。具体的，所述主控模块210可依据任一索引配置信息或者任一索引配置子信息所包括的索引数据对应的引擎服务器260，来指示所述引擎服务器260对应的第一客户端模块230。这样，所述第一客户端模块230可基于所述主控模块210的指示拉取相应的索引数据，并将拉取的索引数据存储至所述第一客户端模块230对应的引擎服务器260中。
可以理解，本实施例的分布式搜索方法为索引数据的拉取更新过程，其具体可以作为离线的数据处理过程。参照图2所示，结合各服务器和各模块的数据处理过程可如下：
步骤21：主控模块210从配置管理服务器240获得拆分规则和索引创建规则。作为一种实施方式，所述主控模块210可根据待创建的索引数据的业务类型获得与所述业务类型相匹配的拆分规则和索引创建规则。
步骤22：所述主控模块210将所述拆分规则和索引创建规则发送至通知模块220。
步骤23：所述通知模块220将所述拆分规则和索引创建规则发送至索引创建服务器250。
所述索引创建服务器250可依据索引创建规则创建索引数据，以及依据拆分规则将索引数据拆分为N个索引子数据。并且，所述索引创建服务器250可基于创建完成、且拆分后的索引数据生成多个索引配置信息或者多个索引配置子信息。每个所述索引配置信息或每个所述索引配置子信息可表征索引数据的拆分结果，并包括拆分后的每个索引子数据对应的引擎服务器260，从而可表示相应的引擎服务器260对应获取并存储哪些索引数据。
步骤24：索引创建服务器250将索引配置信息发送至所述通知模块220。
步骤25：所述通知模块220将所述索引配置信息发送至所述主控模块210。其中，所述通知模块220的数量可以为多个。多个通知模块220可依据业务类型进行功能配置，即不同的通知模块执行相应业务类型的通知功能。这样，主控模块210可依据业务类型获得拆分规则和索引创建规则，并将获得的拆分规则和索引创建规则发送至与业务类型相匹配的通知模块220。相应的，所述索引创建服务器250可将索引配置信息发送至与业务类型相匹配的通知模块220。可以理解的是，多个通知模块可并行工作。
步骤26：主控模块210依据索引配置信息指示客户端模块230。作为一种实施方式，所述主控模块210可依据多个索引配置信息中的任一索引配置信息或者任一索引配置子信息所包括的索引数据对应的引擎服务器260指示所述引擎服务器260对应的客户端模块230。这样，所述客户端模块230可基于所述主控模块210的指示拉取相应的索引数据，并将拉取的索引数据存储至所述客户端模块230对应的引擎服务器260中。
采用本发明实施例的技术方案，引擎服务器260只需要加载对应的索引数据即可，索引数据的更新功能主要通过主控服务器(具体可以是主控服务器中的客户端模块)实现，可大大降低服务器的负担。索引数据分布存储在多个引擎服务器260中，在搜索过程中可大大降低了引擎服务器260的内存使用，从而可有效提升了搜索的效率，减少了搜索的响应时间，提升了用户的操作体验。
本发明实施例还提供了一种分布搜索方法。图3为本发明另一实施例的分布式搜索方 法的流程示意图。如图3所示，所述方法可包括：
步骤301：多个代理服务器中的第一代理服务器接收到查询服务器的查询请求，获得所述查询请求对应的属性信息。
步骤302：所述第一代理服务器基于所述属性信息查询配置管理服务器，以获得所述属性信息对应的机器信息。
步骤303：所述第一代理服务器向所述机器信息对应的至少两个引擎服务器发送所述查询请求，以获得所述至少两个引擎服务器各自根据所述查询请求而返回的第一查询结果。
步骤304：所述第一代理服务器将至少两个所述第一查询结果按照预设规则合并为第二查询结果。
步骤305：所述第一代理服务器将所述第二查询结果发送至所述查询服务器。
本实施例的分布式搜索方法可应用于多个代理服务器，并且所述多个代理服务器中的每个代理服务器可具有相同的功能。图4为本发明实施例的分布式搜索方法的应用架构以及数据交互示意图。如图4所示，在本实施例中以代理服务器的数量为两个为例进行说明。
查询服务器410接收到来自用户的终端设备的查询请求后，可按照预设规则将所述查询请求发送至多个代理服务器420中的至少一个第一代理服务器421。其中，所述预设规则可以是轮询规则或者随机规则等等。在实际应用中，可预先对多个代理服务器420进行编号，所述轮询规则可以是基于所述多个代理服务器420的编号顺序依次选择一个或多个代理服务器作为待发送查询请求的第一代理服务器。例如，以将查询请求发送至一个代理服务器为例，查询服务器410接收到第一查询请求时，可发送所述第一查询请求至编号为1的代理服务器420；查询服务器410接收到第二查询请求时，可发送所述第二查询请求至编号为2的代理服务器420，以此类推。所述第一查询请求和所述第二查询请求可根据数据接收时间确定。而所述随机规则可以是将接收到的查询请求按照预设随机算法发送至相对应的至少一个代理服务器420。
所述第一代理服务器421获得所述查询请求对应的属性信息，所述属性信息可以为所述查询请求对应的业务类型，从而可基于所述业务类型向配置管理服务器240请求机器信息。所述配置管理服务器240的具体介绍可参见以上，在此不再赘述。此外，基于以上描述可知，在对引擎服务器260中的索引数据进行更新存储时，可基于拆分规则对索引数据进行拆分。因此，属于同一业务类型的索引数据可能存储在至少两个引擎服务器260中。
基于此，本实施例中，第一代理服务器查询配置管理服务器240，以获得所述属性信 息对应的机器信息。其中，所述机器信息中可包括至少两个引擎服务器260的标识，所述至少两个引擎服务器260的标识可表明所述查询请求中对应的索引数据存储在所述至少两个引擎服务器中。在具体实现过程中，所述机器信息可通过一机器列表实现。因此，所述第一代理服务器421可根据所述机器信息向对应的至少两个引擎服务器260发送所述查询请求，以获得所述查询请求中包括的关键字或关键词、关联关键字或关联关键词对应的索引数据。
作为一种实施方式，所述第一代理服务器421获得所述至少两个引擎服务器260返回的第一查询结果，可包括：所述第一代理服务器421获得满足预先配置的截断参数的第一查询结果。
具体的，所述截断参数表示任一引擎服务器260返回的查询结果中的索引数据的数量。例如，若一引擎服务器260获得的查询结果包括1000个索引数据，而所述截断参数为600，则该引擎服务器260返回所述1000个索引数据中的前600个索引数据。这样可大大降低搜索时延以及提升每秒查询率(QPS，Query Per Second)。其中，所述截断参数可由配置管理服务器240进行配置，由主控服务器中的主控模块210获得该截断参数，并发送至各引擎服务器260中进行配置。
本实施例中，所述第一代理服务器421获得至少两个引擎服务器260返回的第一查询结果，将获得的至少两个第一查询结果按照预设规则进行合并以及排序以生成第二查询结果，并将所述第二查询结果发送至所述查询服务器410，由所述查询服务器410发送至终端设备以输出显示给用户。
可以理解为，本实施例的分布式搜索方法为搜索查询过程，其具体可以作为在线的数据处理过程。参照图4所示，结合各服务器的数据处理过程如下：
步骤41：查询服务器410获得终端设备的查询请求。
步骤42：查询服务器410将查询请求发送至多个代理服务器420中的至少一个第一代理服务器421。其中，所述第一代理服务器421可以是多个代理服务器420中与查询请求的业务类型对应的代理服务器，也可以是基于预设规则(例如轮询规则或者随机规则等)确定的代理服务器。
其中，所述查询服务器410可对接收到的所述查询请求进行分析，一方面，获得所述查询请求中的关键字或关键词，另一方面，获得所述关键字或关键词具有关联关系的关联关键字或关联关键词，也即对所述查询请求进行意图识别。例如，所述查询请求中包括的关键字或关键词为一餐厅名字，则对所述关键词或关键词进行意图识别可获得关联关键字或关联 关键词为订餐或外卖等等。再例如，所述查询请求中包括的关键字或关键词为字符串，通过意图识别确定该字符串为一中文词语的拼音，则对应的关联关键字或关联关键词可以为该中文词语等等。查询服务器410可进一步基于意图识别后获得的至少一个关键词生成至少一个查询请求，并将所述至少一个查询请求分别发送至对应的至少一个第一代理服务器421。
步骤43：第一代理服务器421基于所述查询请求的属性信息(例如业务类型)向配置管理服务器240请求机器列表，从而获得所述查询请求对应的索引数据所在的引擎服务器260的信息。
步骤44：所述第一代理服务器421基于获得的机器列表相对应的至少两个引擎服务器260发送查询请求。
步骤45：所述至少两个引擎服务器260基于所述查询请求中的内容加载索引数据，并向所述第一代理服务器421返回查询结果。其中，所述引擎服务器260可基于预先配置的截断参数控制查询结果中的索引数据的数量，从而减少查询延时以及提升QPS。
步骤46：所述第一代理服务器421将获得的至少两个查询结果按照预设规则进行合并以及排序以生成最终查询结果，将所述最终查询结果发送至所述查询服务器410。
步骤47：所述查询服务器410将所述最终查询结果发送至终端设备，以供终端设备输出显示给用户。
通过利用多个代理服务器420联结查询服务器410和引擎服务器260的架构，查询服务器410的查询请求可发送至对应的至少一个代理服务器420，由至少一个代理服务器420从相应的至少两个引擎服务器260中获得查询结果。其中，多个代理服务器420可具有相同的功能，多个代理服务器420相互之间可具有并行关系。这样，一个代理服务器420无法工作还可通过其他代理服务器420工作，可有效避免了主设备无法工作时因需要重新选择主设备而导致短时间内搜索服务不可用的问题。并且，由于代理服务器420无需承担索引的更新维护，大大降低了代理服务器420的负担。
本发明实施例还提供了一种分布搜索系统，所述分布搜索系统具体可参照图4和图2所示，所述系统可包括配置管理服务器240、查询服务器410、多个代理服务器420和多个引擎服务器260。
其中，所述配置管理服务器240可用于管理配置信息和机器信息。所述配置信息可包括拆分规则。所述机器信息可表征所述多个引擎服务器的信息。
所述代理服务器420可用于在接收到所述查询服务器410发送的查询请求时获得所述 查询请求对应的属性信息，并基于所述属性信息查询所述配置管理服务器240，以获得所述属性信息对应的机器信息，从而可向所述机器信息对应的至少两个引擎服务器260发送所述查询请求。并且，在获得所述至少两个引擎服务器260返回的第一查询结果后，代理服务器420可将至少两个第一查询结果按照预设规则合并为第二查询结果，并将所述第二查询结果发送至所述查询服务器410。
所述查询服务器410可用于在获得终端设备的查询请求时将所述查询请求发送至所述代理服务器420，并在收到所述第二查询结果时将所述第二查询结果发送至所述终端设备。
所述多个引擎服务器260中的每个引擎服务器可用于存储满足所述拆分规则的索引数据，并在收到所述查询请求时回复所述第一查询结果。
本实施例中，所述系统还可包括主控服务器和索引创建服务器250。其中，所述主控服务器可用于从所述配置管理服务器240获得拆分规则，并发送所述拆分规则至索引创建服务器250。此外，主控服务器还可用于获得所述索引创建服务器250发送的表征拆分结果的索引配置信息，基于所述索引配置信息获取索引数据，并将所述索引数据存储至多个引擎服务器260中对应的至少两个第一引擎服务器中。所述索引创建服务器250可用于基于所述拆分规则将待创建的索引数据进行拆分，并将表征拆分结果的索引配置信息发送至所述主控服务器。
作为一种实施方式，所述代理服务器420可获得满足预先配置的截断参数的查询结果。
本发明实施例的分布搜索系统，通过多个代理服务器联结主控服务器、配置管理服务器、索引创建服务器、查询服务器和引擎服务器的分布搜索架构，将查询功能以及索引更新维护功能分别通过代理服务器以及主控服务器实现，可大大提升了分布式搜索系统的可扩展性以及系统的稳定性。在实际应用中，通过线上以单个索引统计，前50％时间查询延时减少50％，前90％时间查询延时减少54.5％，前99％时间查询延时减少46％，提高了用户体验。
本发明实施例还提供了一种代理服务器。如图5为本发明实施例的代理服务器的组成结构示意图，如图5所示，所述代理服务器可包括通讯单元51和处理单元52。
其中，所述通讯单元51可用于接收来自查询服务器的查询请求，并将所述查询请求发送至所述处理单元52。通讯单元51还可用于向所述处理单元52确定的至少两个引擎服务器发送所述查询请求，获得所述至少两个引擎服务器返回的第一查询结果，并将所述处理单元52合并的第二查询结果发送至所述查询服务器。
所述处理单元52可用于获得所述查询请求对应的属性信息，并基于获得的属性信息查询配置管理服务器，以获得所述属性信息对应的所述至少两个引擎服务器。所述处理单元52还可将至少两个由所述通讯单元51获得的所述第一查询结果按照预设规则合并得到第二查询结果。
作为一种实施方式，所述通讯单元51可获得满足预先配置的截断参数的查询结果。
所述代理服务器中的处理单元52可由所述代理服务器中的中央处理器(CPU，Central Processing Unit)、数字信号处理器(DSP，Digital Signal Processor)、微控制单元(MCU，Microcontroller Unit)或可编程门阵列(FPGA，Field－Programmable Gate Array)实现。所述代理服务器中的通讯单元51可通过通信模组(包含：基础通信套件、操作系统、通信模块、标准化接口和协议等)及收发天线实现。
本发明实施例还提供了一种主控服务器。图6为本发明实施例的主控服务器的组成结构示意图，如图6所示，所述主控服务器可包括主控模块61和通知模块62。
其中，所述主控模块61可用于从配置管理服务器获得拆分规则，并发送所述拆分规则至所述通知模块62。所述主控模块61还可基于所述通知模块62发送的索引配置信息获取索引数据，并将所述索引数据存储至多个引擎服务器中对应的至少两个引擎服务器中。
所述通知模块62可用于发送所述拆分规则至索引创建服务器，以使所述索引创建服务器根据所述拆分规则将待创建的索引数据进行拆分。此外，通知模块62还可获得表征拆分结果的索引配置信息，并发送所述索引配置信息至所述主控模块61。
本实施例中，所述主控服务器还可包括多个客户端模块63。所述多个客户端模块63可与多个引擎服务器一一对应。其中，所述主控模块61可基于所述通知模块62发送的索引配置信息，指示所述索引配置信息中包括的拆分结果对应的客户端模块63获取索引数据。在实际应用中，所述主控模块61可包括：第一通信子模块，用以与所述配置管理服务器通信，以从所述配置管理服务器获得所述拆分规则；第二通信子模块，用以与所述通知模块62通信，以向所述通知模块62发送所述拆分规则并从所述通知模块62获取所述索引配置信息；以及第三通信子模块，用以与客户端模块63通信，以指示所述客户端模块63基于所述索引配置信息获取所述索引数据。
所述通知模块62可用于发送所述拆分规则至索引创建服务器，并在获得表征拆分结果的索引配置信息后发送所述索引配置信息至所述主控模块61。在实际应用中，所述通知模块62可包括：第一通信模块，用以与所述主控模块61通信，以从所述主控模块61获得所述 拆分规则并向所述主控模块61发送所述索引配置信息；以及，第二通信模块，用以与所述索引创建服务器通信，以向所述索引创建服务器发送所述拆分规则并从所述索引创建服务器获得所述索引配置信息。
所述客户端模块63可基于所述主控模块61的指示获取索引数据，并将所述索引数据存储至对应的引擎服务器中。在实际应用中，所述客户端模块63可包括：第一通信子模块，用以与所述主控模块61，以接受所述主控模块61的指示；处理模块，用于响应于所述主控模块61的指示基于所述索引配置信息获取索引数据；以及，第二通信子模块，用以与所述引擎服务器通信，以将所述索引数据存储至对应的引擎服务器中。
本实施例中，所述主控服务器可以是服务器集群。所述主控模块61主要负责统一调度。所述通知模块62主要负责与索引创建服务器通信。所述通知模块62的数量可以是至少一个。至少一个通知模块62可基于业务类型的不同进行区分，例如每个通知模块62用于通知属于相应业务类型的索引创建的相关信息。所述客户端模块63的数量可与引擎服务器的数量相同，且每个客户端模块63对应一个引擎服务器。所述客户端模块63可用于根据主控模块61的指示拉取索引数据，并将拉取的索引数据存储至对应的引擎服务器中。所述主控模块61、每个通知模块62可通过独立的服务器实现。所述客户端模块63可位于对应的引擎服务器中，并通过对应的引擎服务器实现相应功能。在实际应用中，所述主控模块61可配置备用主控模块；每个通知模块62也可配置对应的备用通知模块。每个客户端模块63也可配置对应的备用客户端模块63。这样，主模块出现异常无法工作时，可通过相应的备用模块继续执行相应的功能。
本发明实施例还提供了一种计算机设备，图7为本发明实施例的计算机设备的组成结构示意图，如图7所示。所述计算机设备可包括处理器71、存储器72以及至少一个外部通信接口73。所述处理器71、存储器72以及外部通信接口73均可通过总线74连接。所述存储器72中存储有可在处理器71上运行的计算机程序。
其中，在所述计算机设备作为代理服务器时，所述处理器71执行所述程序时实现以下步骤：接收来自查询服务器的查询请求；获得所述查询请求对应的属性信息；基于所述属性信息查询配置管理服务器，获得所述属性信息对应的机器信息；向所述机器信息对应的至少两个引擎服务器发送所述查询请求；获得所述至少两个引擎服务器根据所述查询请求返回的第一查询结果；将至少两个所述第一查询结果按照预设规则合并为第二查询结果；以及，将所述第二查询结果发送至所述查询服务器。换言之，处理器71可通过执行程序实现图5所示代理服务器中的通讯单元51和处理单元52的具体功能。
作为一种实施方式，所述处理器71执行所述程序时实现以下步骤：获得满足预先配置的截断参数的查询结果。
在计算机设备作为主控服务器时，所述处理器71执行所述程序时实现以下步骤：从配置管理服务器获得拆分规则；发送所述拆分规则至索引创建服务器，以使索引创建服务器根据所述拆分规则将待创建的索引数据进行拆分；获得表征拆分结果的索引配置信息；基于所述索引配置信息获取索引数据；以及，将所述索引数据存储至多个引擎服务器中相对应的至少两个第一引擎服务器中。换言之，处理器71可通过执行程序实现图6所示主控服务器中的主控模块61、通知模块62和客户端模块63的具体功能。
这里需要指出的是：以上涉及计算机设备的描述，与上述方法描述是类似的，同方法的有益效果描述，不做赘述。对于本发明计算机设备实施例中未披露的技术细节，请参照本发明方法实施例的描述。
在本申请所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，如：多个单元或组件可以结合，或可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口，设备或单元的间接耦合或通信连接，可以是电性的、机械的或其它形式的。
上述作为分离部件说明的单元可以是、或也可以不是物理上分开的，作为单元显示的部件可以是、或也可以不是物理单元，即可以位于一个地方，也可以分布到多个网络单元上；可以根据实际的需要选择其中的部分或全部单元来实现本实施例方案的目的。例如前面所述的各种服务器，可以是物理上的一台硬件机器，也可以是运行于服务器集群中的一个软件模块。
另外，在本发明各实施例中的各功能单元可以全部集成在一个处理单元中，也可以是各单元分别单独作为一个单元，也可以两个或两个以上单元集成在一个单元中；上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。
本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于一计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：移动存储设备、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟 或者光盘等各种可以存储程序代码的介质。
或者，本发明上述集成的单元如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机、服务器、或者网络设备等)执行本发明各个实施例所述方法的全部或部分。而前述的存储介质包括：移动存储设备、ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以所述权利要求的保护范围为准。

Claims (15)

  1. 一种分布式搜索方法，包括：
    多个代理服务器中的至少一个第一代理服务器在接收到来自查询服务器的查询请求时，获得所述查询请求对应的属性信息；
    所述第一代理服务器基于所述属性信息查询配置管理服务器，以获得所述属性信息对应的机器信息；
    所述第一代理服务器向所述机器信息对应的至少两个引擎服务器发送所述查询请求；
    所述第一代理服务器获得所述至少两个引擎服务器根据所述查询请求而返回的第一查询结果；
    所述第一代理服务器将至少两个所述第一查询结果按照预设规则合并为第二查询结果；以及
    所述第一代理服务器将所述第二查询结果发送至所述查询服务器。
  2. 根据权利要求1所述的方法，其特征在于，获得所述至少两个引擎服务器根据所述查询请求而返回的所述第一查询结果，包括：
    所述第一代理服务器获得满足预先配置的截断参数的所述第一查询结果。
  3. 根据权利要求1所述的方法，其特征在于，还包括：
    从所述多个代理服务器中选择至少一个业务类型与所述查询请求匹配的代理服务器作为所述第一代理服务器。
  4. 根据权利要求1所述的方法，其特征在于，还包括：
    按预设规则从所述代理服务器中选择至少之一作为所述第一代理服务器，
    其中，所述预设规则包括随机规则和轮询规则。
  5. 一种索引更新方法，包括：
    主控服务器从配置管理服务器获得拆分规则；
    所述主控服务器发送所述拆分规则至索引创建服务器，以使所述索引创建服务器根据所述拆分规则将待创建的索引数据进行拆分；
    所述主控服务器获得表征所述拆分的结果的索引配置信息；
    所述主控服务器基于所述索引配置信息获取索引数据；
    所述主控服务器将所述索引数据存储至多个引擎服务器中对应的至少两个引擎服务器中。
  6. 一种代理服务器，包括：
    通讯单元，用于接收来自查询服务器的查询请求；
    处理单元，用于获得所述查询请求对应的属性信息，基于所述属性信息查询配置管理服务器以获得所述属性信息对应的机器信息，并确定所述机器信息对应的至少两个引擎服务器，
    其中，所述通讯单元还用于向所述至少两个引擎服务器发送所述查询请求，以获得所述至少两个引擎服务器根据所述查询请求而返回的第一查询结果；
    所述处理单元还用于将至少两个所述第一查询结果按照预设规则进行合并，得到第二查询结果；以及
    所述通讯单元还用于将所述第二查询结果发送至所述查询服务器。
  7. 一种主控服务器，包括：
    主控模块，用于从配置管理服务器获得拆分规则；
    通知模块，用于发送所述拆分规则至索引创建服务器，以使所述索引创建服务器根据所述拆分规则将待创建的索引数据进行拆分，并获得表征所述拆分的结果的索引配置信息，
    其中，所述主控模块还用于基于所述索引配置信息获取索引数据，并将所述索引数据存储至多个引擎服务器中对应的至少两个引擎服务器中。
  8. 根据权利要求7所述的主控服务器，其特征在于，所述主控服务器还包括多个客户端模块；所述多个客户端模块与多个引擎服务器一一对应；其中，
    所述主控模块基于所述索引配置信息指示第一客户端模块获取第一索引数据，其中，所述第一客户端模块为所述多个客户端模块中与所述索引配置信息包括的第一拆分结果对应的客户端模块；
    所述第一客户端模块基于所述主控模块的指示获取索引数据，并将所述索引数据存储至对应的引擎服务器中。
  9. 一种分布式搜索系统，包括：
    配置管理服务器，用于管理配置信息和机器信息，所述配置信息包括拆分规则，所述机器信息表征所述多个引擎服务器的信息；
    查询服务器，用于获得终端设备的查询请求；
    多个代理服务器；以及
    多个引擎服务器，所述多个引擎服务器中的每个引擎服务器用于存储满足所述拆分规则的索引数据，
    其中，所述多个代理服务器中的至少一个第一代理服务器接收到来自所述查询服务器的所述查询请求后，通过基于所述查询请求的属性信息查询所述配置管理服务器，确定所述多个引擎服务器中的至少两个第一引擎服务器，并向所述至少两个第一引擎服务器发送所述查询请求；
    所述至少两个第一引擎服务器各自响应于收到所述查询请求回复第一查询结果；
    所述至少一个第一代理服务器将至少两个所述第一查询结果合并为第二查询结果，并发送至所述查询服务器，以由所述查询服务器将所述第二查询结果返回至所述终端设备。
  10. 根据权利要求9所述的系统，其特征在于，所述系统还包括：
    主控服务器，用于从所述配置管理服务器获得所述拆分规则；
    索引创建服务器，用于基于所述主控服务器发送的所述拆分规则将待创建的索引数据进行拆分，并将表征所述拆分的结果的索引配置信息发送至所述主控服务器，
    其中，所述主控服务器基于所述索引配置信息获取索引数据，并将所述索引数据存储至所述多个引擎服务器中对应的至少两个引擎服务器中。
  11. 根据权利要求9所述的系统，其特征在于，所述第一引擎服务器基于预先配置的截断参数向所述第一代理服务器回复所述第一查询结果。
  12. 根据权利要求9所述的系统，其特征在于，所述查询服务器向所述多个代理服务器中业务类型与所述查询请求的业务类型匹配的所述第一代理服务器发送所述查询请求。
  13. 根据权利要求9所述的系统，其特征在于，所述查询服务器按照预设的规则从所述代理服务器中选择至少之一作为所述第一代理服务器，并向所述第一代理服务器发送所述查询请求。
  14. 一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时实现权利要求1至4任一项所述分布式搜索的步骤。
  15. 一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时实现权利要求5所述索引更新方法的步骤。
PCT/CN2017/120018 2017-07-05 2017-12-29 分布式搜索及索引更新方法、系统、服务器及计算机设备 WO2019007010A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2019568703A JP6967615B2 (ja) 2017-07-05 2017-12-29 分散型検索及びインデックスの更新方法、システム、サーバー並びにコンピューター機器
US16/622,298 US20200210496A1 (en) 2017-07-05 2017-12-29 Distributed search method, index update method, system, server, and computer device
CA3065118A CA3065118C (en) 2017-07-05 2017-12-29 Distributed searching and index updating method and system, servers, and computer devices

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710540135.0 2017-07-05
CN201710540135.0A CN107273540B (zh) 2017-07-05 2017-07-05 分布式搜索及索引更新方法、系统、服务器及计算机设备

Publications (1)

Publication Number Publication Date
WO2019007010A1 true WO2019007010A1 (zh) 2019-01-10

Family

ID=60071122

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/120018 WO2019007010A1 (zh) 2017-07-05 2017-12-29 分布式搜索及索引更新方法、系统、服务器及计算机设备

Country Status (6)

Country Link
US (1) US20200210496A1 (zh)
JP (1) JP6967615B2 (zh)
CN (1) CN107273540B (zh)
CA (2) CA3184577A1 (zh)
TW (1) TWI740029B (zh)
WO (1) WO2019007010A1 (zh)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107273540B (zh) * 2017-07-05 2021-09-24 北京三快在线科技有限公司 分布式搜索及索引更新方法、系统、服务器及计算机设备
CN109255072B (zh) * 2018-08-15 2023-04-14 腾讯科技（深圳）有限公司 信息召回方法及装置、计算机存储介质、电子设备
CN109409924B (zh) * 2018-09-03 2023-04-18 平安科技（深圳）有限公司 账户评分系统、方法、服务器和计算机可读存储介质
CN109918472A (zh) * 2019-02-27 2019-06-21 北京百度网讯科技有限公司 存储和查询数据的方法、装置、设备和介质
CN111405039A (zh) * 2020-03-16 2020-07-10 深圳市网心科技有限公司 一种数据透传方法、装置、系统和客户端及服务器
CN111417119A (zh) * 2020-03-16 2020-07-14 纳瓦电子（上海）有限公司 无线级联的方法
CN111858585A (zh) * 2020-06-30 2020-10-30 深圳幂度信息科技有限公司 区块链策略处理装置、计算机可读存储介质及终端设备
CN111931033A (zh) * 2020-08-11 2020-11-13 深圳市欢太科技有限公司 一种检索方法、检索装置及服务器
CN113761079A (zh) * 2021-01-21 2021-12-07 北京沃东天骏信息技术有限公司 数据访问方法、系统和存储介质
CN113438304B (zh) * 2021-06-23 2023-04-07 平安消费金融有限公司 基于数据库集群的数据查询方法、装置、服务器及介质
CN113535730A (zh) * 2021-07-21 2021-10-22 挂号网（杭州）科技有限公司 搜索引擎的索引更新方法及系统、电子设备、存储介质
CN113641796A (zh) * 2021-08-30 2021-11-12 平安医疗健康管理股份有限公司 数据搜索方法、系统及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1571398A (zh) * 2004-04-29 2005-01-26 上海交通大学 基于代理映射的网络安全隔离与信息交换系统及方法
CN101950300A (zh) * 2010-09-20 2011-01-19 华南理工大学 一种分层结构、分布式搜索引擎系统及其实现方法
US20150193404A1 (en) * 2013-01-07 2015-07-09 Google Inc. Operational transformations proxy for thin clients
CN105978948A (zh) * 2016-04-27 2016-09-28 努比亚技术有限公司 一种云服务的方法和系统
CN107273540A (zh) * 2017-07-05 2017-10-20 北京三快在线科技有限公司 分布式搜索及索引更新方法、系统、服务器及计算机设备

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8880489B2 (en) * 2005-08-04 2014-11-04 Hewlett-Packard Development Company, L.P. Discovery across multiple registries
CN102375853A (zh) * 2010-08-24 2012-03-14 中国移动通信集团公司 分布式数据库系统、在其中建立索引的方法和查询方法
CN102033912A (zh) * 2010-11-25 2011-04-27 北京北纬点易信息技术有限公司 一种分布式数据库访问方法及系统
US9774676B2 (en) * 2012-05-21 2017-09-26 Google Inc. Storing and moving data in a distributed storage system
CN104978337A (zh) * 2014-04-08 2015-10-14 张军 一种基于软件定义网络的分布式视频搜索引擎框架
CN105320527A (zh) * 2014-06-12 2016-02-10 中兴通讯股份有限公司 基于zookeeper分布式搜索引擎的配置文件更新方法、装置及系统
US9760591B2 (en) * 2015-05-14 2017-09-12 Walleye Software, LLC Dynamic code loading
CN105187551A (zh) * 2015-09-29 2015-12-23 成都四象联创科技有限公司 基于云平台的分布式计算方法
CN106776694A (zh) * 2016-11-11 2017-05-31 张军 一种基于软件定义的网络分布式图片搜索引擎框架

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1571398A (zh) * 2004-04-29 2005-01-26 上海交通大学 基于代理映射的网络安全隔离与信息交换系统及方法
CN101950300A (zh) * 2010-09-20 2011-01-19 华南理工大学 一种分层结构、分布式搜索引擎系统及其实现方法
US20150193404A1 (en) * 2013-01-07 2015-07-09 Google Inc. Operational transformations proxy for thin clients
CN105978948A (zh) * 2016-04-27 2016-09-28 努比亚技术有限公司 一种云服务的方法和系统
CN107273540A (zh) * 2017-07-05 2017-10-20 北京三快在线科技有限公司 分布式搜索及索引更新方法、系统、服务器及计算机设备

Also Published As

Publication number Publication date
CN107273540A (zh) 2017-10-20
US20200210496A1 (en) 2020-07-02
TW201907324A (zh) 2019-02-16
TWI740029B (zh) 2021-09-21
CN107273540B (zh) 2021-09-24
CA3065118A1 (en) 2019-01-10
CA3065118C (en) 2024-03-26
JP2020523700A (ja) 2020-08-06
JP6967615B2 (ja) 2021-11-17
CA3184577A1 (en) 2019-01-10

Similar Documents

Publication Publication Date Title
TWI740029B (zh) 分散式搜索及索引更新方法、系統、伺服器及電腦設備
CN110489417B (zh) 一种数据处理方法及相关设备
CN107508795B (zh) 跨容器集群的访问处理装置及方法
US20160330067A1 (en) Network Service Fault Handling Method, Service Management System, and System Management Module
WO2019153488A1 (zh) 服务配置管理方法、装置、存储介质和服务器
WO2018040722A1 (zh) 表数据查询方法及装置
US20200349151A1 (en) Methods, systems, and computer readable mediums for performing an aggregated free-form query
US11762775B2 (en) Systems and methods for implementing overlapping data caching for object application program interfaces
US10474185B2 (en) Timestamp alignment across a plurality of computing devices
CN111930770A (zh) 数据查询方法、装置及电子设备
US20180123935A1 (en) Network Configuration Management System
CN113271359A (zh) 刷新缓存数据的方法、装置、电子设备和存储介质
WO2019062156A1 (zh) 存储过程的执行方法、装置及存储介质
CN110597808B (zh) 分布式数据库表连接方法、装置、系统、服务器及介质
US9489652B1 (en) Obtaining and running a local query on a computing device
US20220067047A1 (en) Method, apparatus, device and storage medium for generating and processing a distributed graph database
WO2023024057A1 (zh) 跨域授权处理方法及跨域调用处理方法
CN112910796B (zh) 流量管理方法、装置、设备、存储介质以及程序产品
JP5402066B2 (ja) 情報検索システム、情報検索装置、情報検索プログラム及び情報検索方法
CN112148925A (zh) 用户标识关联查询方法、装置、设备及可读存储介质
US11853814B2 (en) Automatically generating events
EP3993366B1 (en) Network load balancer, request message distribution method, program product and system
US20240179059A1 (en) Systems and Methods for Storing and Transporting Configuration Data to Edge Servers
US20230131871A1 (en) Method, system and apparatus for forwarding data in virtual network and program product
WO2022031887A1 (en) Improved database management systems and methods using data normalization and defragmentation techniques

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 3065118

Country of ref document: CA

ENP Entry into the national phase

Ref document number: 2019568703

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17917036

Country of ref document: EP

Kind code of ref document: A1