WO2020134786A1 - 服务器的扩容方法及装置、服务器、存储介质 - Google Patents

服务器的扩容方法及装置、服务器、存储介质 Download PDF

Info

Publication number
WO2020134786A1
WO2020134786A1 PCT/CN2019/120668 CN2019120668W WO2020134786A1 WO 2020134786 A1 WO2020134786 A1 WO 2020134786A1 CN 2019120668 W CN2019120668 W CN 2019120668W WO 2020134786 A1 WO2020134786 A1 WO 2020134786A1
Authority
WO
WIPO (PCT)
Prior art keywords
host
solr
solr cluster
performance
current
Prior art date
Application number
PCT/CN2019/120668
Other languages
English (en)
French (fr)
Inventor
吴科桦
何宜梅
邢鹏
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2020134786A1 publication Critical patent/WO2020134786A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/04—Network management architectures or arrangements
    • H04L41/044—Network management architectures or arrangements comprising hierarchical management structures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0894—Policy-based network configuration management
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0803—Configuration setting
    • H04L41/0823—Configuration setting characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0896—Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/28—Restricting access to network management systems or functions, e.g. using authorisation function to access network configuration
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Definitions

  • the embodiments of the present invention relate to a distributed search engine technology, and in particular, to a server capacity expansion method and device, a server, and a storage medium.
  • Solr Enterprise Search Application Server
  • Solr is a high-performance full-text search service based on the full-text search engine (Lucene) developed in the computer programming language (Java). It is expanded on the basis of Lucene and provides more Rich query language, support configurable and extensible, and optimize query performance, at the same time it provides a perfect function management interface, is a very good full-text search service.
  • Solr cluster is a distributed system, it has one or more shards, each shard has a hash range, all shards form a complete hash ring, the range of the hash ring is 0x00-0xfffffff, Sharding is a concept of virtual scope.
  • Each shard corresponds to one or more replicas, and the replicas are physical nodes that actually exist on the host.
  • an embodiment of the present invention provides a server capacity expansion method.
  • the method includes: acquiring security level information of a current enterprise-level search application server Solr cluster; based on the security level information, generating a A security level strategy for expanding capacity at the security level; acquiring performance level information of the current Solr cluster; based on the performance level information, generating a performance level strategy for expanding the performance level of the Solr cluster; according to the security level strategy Expand the Solr cluster with the performance level strategy.
  • an embodiment of the present invention provides a server capacity expansion device.
  • the device includes: a first acquiring unit configured to acquire security level information of a current enterprise search application server Solr cluster; a first generating unit configured to be based on The security level information generates a security level strategy for expanding the security level of the Solr cluster; the second obtaining unit is configured to obtain performance level information of the current Solr cluster; the second generating unit is configured based on the The performance level information generates a performance level strategy for expanding the Solr cluster at the performance level; the capacity expansion unit is configured to expand the Solr cluster according to the security level strategy and the performance level strategy.
  • an embodiment of the present invention provides a server expansion server, including a memory and a processor, where the memory stores a computer program that can run on the processor, and the processor implements the program to implement the foregoing Steps in the method of generating expansion strategies.
  • an embodiment of the present invention provides a computer-readable storage medium in which a computer program is stored, and the computer program is configured to execute the steps in the above-described server capacity expansion method when executed.
  • an embodiment of the present invention provides a computer program product.
  • the computer program product includes a computer program stored on a non-transitory computer-readable storage medium.
  • the computer program includes program instructions. When the program instructions are When executed by a computer, the computer is caused to perform the method described in the above aspects.
  • FIG. 1 is a schematic flowchart of an implementation of a server capacity expansion method according to an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of an implementation of another server capacity expansion method according to an embodiment of the present invention.
  • FIG. 3A is a schematic flowchart of another server capacity expansion method according to an embodiment of the present invention.
  • FIG. 3B is a schematic diagram of an implementation process of a method for determining a weight value
  • FIG. 4 is a schematic flowchart of another server expansion method according to an embodiment of the present invention.
  • 5A is a schematic diagram of an implementation process of a method for generating a security level expansion strategy according to an embodiment of the present invention
  • 5B is a schematic diagram of an implementation process of a method for generating a capacity expansion strategy according to an embodiment of the present invention
  • 6A is a schematic diagram of a system for expanding a Solr cluster server based on an adaptive balancing strategy according to an embodiment of the present invention
  • FIG. 6B is a schematic diagram of a flow for implementing expansion of a server provided by an embodiment of the present invention.
  • 6C is a schematic diagram of a principle for generating a Solr cluster after expansion according to an embodiment of the present invention.
  • 6D is a schematic diagram of a principle for providing a capacity expansion strategy at a performance level according to an embodiment of the present invention
  • FIG. 7 is a schematic structural diagram of a server capacity expansion device according to an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of a hardware entity for generating an expansion strategy server in an embodiment of the present invention.
  • the current expansion method is only based on the current Solr cluster sharding and replica status, combined with the newly added Solr host, manual segmentation. This expansion method has the following defects:
  • the expansion factor does not consider the security factor of the Solr cluster copy factor
  • the embodiment of the present invention proposes a server expansion method and system based on an adaptive balancing strategy, so that when a Solr cluster encounters a security or performance bottleneck, it can be based on the characteristics of the Solr cluster and its own characteristics of the cluster environment , Combined with certain expansion rules, automatically plan and execute the expansion strategy, so that the security and performance of the Solr cluster after expansion are more optimized.
  • An embodiment of the present invention provides a server capacity expansion method.
  • the method is applied to a server.
  • the function implemented by the method can be implemented by a processor in a terminal invoking a program code.
  • the program code can be stored in a computer storage medium and can be seen.
  • the server includes at least a processor and a storage medium.
  • FIG. 1 is a schematic diagram of an implementation process of a server capacity expansion method according to an embodiment of the present invention.
  • the method includes: step S101, the server obtains security level information of the current Solr cluster; here, the server is not limited here. It can be one server or multiple servers.
  • the security level information includes but is not limited to the basic information of the Solr cluster and the transmission speed of each Solr host network during data transmission.
  • the basic information of the Solr cluster may be Solr cluster's replica information, Solr cluster's replica factor, the current Solr cluster's shard number, the hash range of each shard, and each shard corresponds to the distribution of replicas on each Solr host.
  • Obtaining the security level information of the current Solr cluster may be obtaining the copy information of the current Solr cluster or obtaining the transmission speed of each Solr host network of the current Solr cluster for data transmission.
  • Step S102 based on the security level information, the server generates a security level strategy for expanding the Solr cluster at the security level; based on the security level information, generates a security level strategy for expanding the Solr cluster at the security level
  • the security level strategy is illustrated here for ease of understanding.
  • the security level information is the copy information of the Solr cluster.
  • the server obtains the copy factor in the current Solr cluster copy information and analyzes its security level. Is the copy factor greater than 1, If it is greater than 1, the cluster security is high.
  • the server does not need to generate a strategy to increase the copy factor of the Solr cluster; otherwise, if the copy factor is equal to 1, the Solr cluster is unstable, and it may be that a copy of the index data appears If the host is damaged or the network of the host is disconnected, the copy will be unavailable, which will directly cause some services of the Solr cluster to be unavailable.
  • the server generates a strategy to increase the copy factor of the Solr cluster, that is, based on the copy information in the security level information To generate a security level policy for expanding the Solr cluster at the security level.
  • a security level strategy for expanding the Solr cluster at the security level is generated.
  • the security level information is used for data transmission for each host of the Solr cluster.
  • the transmission speed of each host in the current Solr cluster for data transmission is obtained, and the security weight of each Solr host is counted. You can read the latest N data sampling records of each Solr host for network transmission.
  • the security weight of each Solr host is calculated according to the rules, for example, the total Good weight is 1, Bad above M% has a weight of 0, M% is a tolerance, and can be set according to the actual situation of network transmission; finally, a copy of the Solr cluster on a host with a security weight of 0 is moved to a host with a security weight of 1
  • the above strategy that is, the copy transfer security host strategy, or for Solr hosts with frequent network disconnection or slow network transmission speed and other poor network stability, try not to deploy Solr copies. If there are existing copies on such hosts, you can move To other hosts with good network stability. Therefore, based on the copy information in the security layer information, a security layer policy for expanding the security level of the Solr cluster is generated.
  • Step S103 the server obtains the performance level information of the current Solr cluster; here, the performance level information includes but is not limited to the time-consuming data sampling of the read and write indexes of each Solr host, the system resource status of each Solr host, and co-located The status of components, the current number of Solr cluster shards, the hash range of each shard, and the distribution of replica hosts corresponding to each shard.
  • Obtaining the performance level information of the current Solr cluster can be the speed at which each host of the current Solr cluster reads and writes the index and the system load of each host.
  • Step S104 the server generates a performance level strategy for expanding the Solr cluster at the performance level based on the performance level information; here, based on the performance level information, generates a performance level strategy for the Solr cluster at the performance level
  • the performance-level strategy for capacity expansion is illustrated here for ease of understanding. For example, based on the performance-level information, the speed of reading and writing the index for each host in the current Solr cluster based on the performance-level information and each The system load of the host, based on the speed at which each host of the current Solr cluster in the performance level information reads and writes the index and the system load of each host generates performance for expanding the capacity of the Solr cluster at the performance level Level strategy.
  • Step S105 The server expands the Solr cluster according to the security layer strategy and the performance layer strategy.
  • expanding the Solr cluster according to the security level strategy and the performance level strategy may be a process in which the server acquires the security level expansion strategy and the performance level expansion strategy in sequence.
  • the acquiring the security level information of the current Solr cluster includes: acquiring the copy information of the current Solr cluster; the generating security for expanding the security level of the Solr cluster based on the security level information
  • the level strategy includes: generating a security level strategy that increases the number of copies of the Solr cluster based on the copy information of the current Solr cluster.
  • the server generates a security level strategy to increase the number of copies of the Solr cluster based on the copy information of the current Solr cluster.
  • the security level information is the copy information of the Solr cluster.
  • the server obtains the copy factor in the current Solr cluster copy information and analyzes the security level.
  • the server does not need to generate a strategy to increase the copy factor of the Solr cluster; conversely, if the copy factor is equal to 1, the Solr cluster is unstable, it may be that a copy of the index data is damaged, or where If the host is disconnected from the network, the copy will be unavailable, which will directly cause some services of the Solr cluster to be unavailable.
  • the server generates a strategy to increase the copy factor of the Solr cluster. That is, based on the copy information in the security level information, the copy is generated.
  • a security level strategy for expanding the security level of the Solr cluster is generated.
  • the acquiring the security level information of the current Solr cluster includes: acquiring the transmission speed of each host of the current Solr cluster during data transmission; the generating, based on the security level information, for the Solr
  • the security level strategy for the cluster to expand at the security level includes: generating a security level strategy for transferring a copy of the Solr cluster on the host to the secure host based on the transmission speed of each host of the current Solr cluster when data is transferred.
  • the server generates a security level policy for transferring a copy of the Solr cluster on the host to a secure host based on the transmission speed when each host of the current Solr cluster performs data transmission.
  • the security level information is the transmission speed when each host of the Solr cluster performs data transmission. By periodically sampling the network transmission data for each Solr host, each of the current Solr clusters is obtained. The transmission speed of the host during data transmission, and the security weight of each Solr host is counted. You can read the latest N data sampling records of each Solr host for network transmission.
  • the security weight of each Solr host is calculated according to the rules, for example, the total Good weight is 1, Bad above M% has a weight of 0, M% is a tolerance, and can be set according to the actual situation of network transmission; finally, a copy of the Solr cluster on a host with a security weight of 0 is moved to a host with a security weight of 1
  • the above strategy that is, the copy transfer security host strategy, or for Solr hosts with frequent network disconnection or slow network transmission speed and other poor network stability, try not to deploy Solr copies. If there are existing copies on such hosts, you can move To other hosts with good network stability. Therefore, based on the copy information in the security layer information, a security layer policy for expanding the security level of the Solr cluster is generated.
  • FIG. 2 is a schematic flowchart of an implementation of another server capacity expansion method according to an embodiment of the present invention.
  • the method includes: step S201, the server obtains security level information of the current Solr cluster; step S202, the server is based on the Security level information to generate a security level strategy for expanding the Solr cluster at the security level; Step S203, the server obtains the speed at which each host of the current Solr cluster reads and writes the index, and the system of each host Load; here, the speed of each host of the current Solr cluster to read and write the index, and, the system load of each host can be stored in the relational database, the server can obtain each of the current Solr cluster from the relational database The speed at which the host reads and writes the index, and the system load of each host.
  • Step S204 the server generates a performance level strategy for expanding the performance level of the Solr cluster based on the speed of reading and writing the index of each host of the current Solr cluster and the system load of each host ;
  • the server generates a performance level strategy for expanding the performance level of the Solr cluster based on the speed at which each host of the current Solr cluster reads and writes the index and the system load of each host
  • the server can analyze the speed of each host of the current Solr cluster to read and write the index, and the system load of each host, and use the analysis results to generate a performance level for expanding the Solr cluster at the performance level Strategy.
  • Step S205 the server expands the Solr cluster according to the security level policy and the performance level policy.
  • FIG. 3A is a schematic flowchart of an implementation of another server capacity expansion method according to an embodiment of the present invention.
  • the method includes: step S301, the server obtains security level information of the current Solr cluster; step S302, the server is based on the security Level information, generate a security level strategy for expanding the Solr cluster at the security level; Step S303, the server obtains the speed at which each host of the current Solr cluster reads and writes the index, and the system load of each host Step S304, the server calculates the overall performance weight value of each host of the current Solr cluster according to the sum of the first weight value, the second weight value, and the third weight value of each host; here, the first weight value may By selecting the index reading time of one Solr host as the benchmark weight, the ratio of the index reading time of other hosts to the benchmark execution time is calculated.
  • the second weight value can be calculated as the reference weight by selecting the time taken by one Solr host to write the index as a reference weight, and calculating the ratio of the time spent by another host in reading the index test to the reference execution time.
  • the third weight value can be calculated by reading the system resource usage of each Solr host and the status of co-located components, such as CPU usage, memory usage, number of task queues, co-located components, etc. The weight value of each host system load.
  • the overall performance weight value of each host in the current Solr cluster can be calculated by combining the first weight value and the second weight of each host The value and the third weight value are added, and the result of the addition is the comprehensive performance weight value of each host in the current Solr cluster.
  • Step S305 the server generates a performance level strategy for expanding the performance level of the Solr cluster according to the comprehensive performance weight value of each host of the current Solr cluster;
  • the server can sort the overall performance weight value of each host in the current Solr cluster.
  • the sorting can be that the comprehensive performance weight value is ranked from high to low or from low to high.
  • the Solr hosts are grouped from the first Starting from a host, consecutive r hosts are a group, r can be the number of copies of each slice, and the overall performance weight value of each group of hosts is accumulated as the overall performance weight value of the group, and the comprehensive performance weight value is used to generate A performance-level strategy for expanding the capacity of the Solr cluster at the performance level.
  • Step S306 the server expands the Solr cluster according to the security level policy and the performance level policy.
  • FIG. 3B is a schematic flowchart of a method for determining a weight value.
  • the method further includes: Step S311, the server determines that a host in the current Solr cluster is a reference host; here, it determines the current A host in the Solr cluster is the base host. You can choose any host in the current Solr cluster as the base host.
  • Step S312 the server determines the first weight value of the host according to the ratio of the index reading speed of each host in the current Solr cluster to the index reading speed of the reference host; here, each of the current Solr clusters
  • the index reading speed of the host can be to perform busy hour performance data sampling for each Solr host, analyze the read index, and the time-consuming data uses the reference read speed of the reference host as a reference, and read each host in the current Solr cluster.
  • the speed of the index is compared with the speed of the reference host reading the index, and the obtained ratio is the first weight value of the host.
  • Step S313 the server determines the second weight value of the host according to the ratio of the index writing speed of each host in the current Solr cluster to the index writing speed of the reference host; here, each of the current Solr clusters
  • the speed at which the host writes the index can be a busy hour performance data sampling for each Solr host, and the speed of writing and querying the index for each Solr host is tested separately, and the time-consuming data refers to the speed at which the benchmark host writes the index as a reference
  • the index writing speed of each host in the current Solr cluster is compared with the index writing speed of the reference host, and the obtained ratio is the second weight value of the host.
  • Step S314 the server determines the third weight value of the host according to the ratio of the system load of each host in the current Solr cluster to the system load of the reference host.
  • the system load of each host in the current Solr cluster may be the system resource usage of each Solr host, and the status of co-located components, such as CPU usage, memory usage, number of task queues, co-located components, etc.
  • the third weight value of the host can be determined by comparing the system load of each host in the current Solr cluster with all Comparing the system load of the reference host, the obtained ratio is the third weight value of the host.
  • the method includes: step S401, the server obtains security level information of the current Solr cluster; step S402, the server is based on the security Level information, generate a security level strategy for expanding the security level of the Solr cluster; step S403, the server obtains the speed of reading and writing the index of each host of the current Solr cluster, and the system load of each host Step S404, the server calculates the overall performance weight value of each host of the current Solr cluster according to the sum of the first weight value, the second weight value, and the third weight value of each host; step S405, the server converts the current Solr
  • the overall performance weight value of each host in the cluster is sorted; here, the sorting may be that the overall performance weight value is ranked from high to low or from low to high.
  • Step S406 the server groups each host according to the result of the sorting; here, the grouping may start from the first host, a group of consecutive r hosts, and r may be a copy of each slice Number or copy factor, the number of copies and the copy factor can be the same or different.
  • Step S407 the server accumulates the comprehensive performance weight value of each host in each group as the group performance weight value of the group of hosts; here, the group performance weight value of the group of hosts may be the value of each host in the group The comprehensive performance weight values are added.
  • Step S408 the server assigns addresses to each group of hosts according to the group performance weight value of each group of hosts; here, the addresses 0x00-0xffffffff can be allocated in proportion to each group's comprehensive performance weight value, and the allocated address range is each Shard index range; then, for each shard, set the corresponding primary node (Leader) and secondary node (Replica) in each group of hosts.
  • Step S409 the server generates a performance level strategy for expanding the performance level of the Solr cluster according to the address of each group of hosts; step S410, the server compares the security level strategy with the performance level strategy Solr cluster expansion.
  • the generating a performance-level strategy for expanding the performance level of the Solr cluster based on the addresses of each group includes: Step S411, the server converts all the hosts according to the addresses of each group of hosts
  • the shards of the Solr cluster are allocated to each group of hosts; here, the server allocates the shards of the Solr cluster to each group of hosts according to the address of each group of hosts by assigning the original of the Solr cluster Shard 1 (Shard1) and original shard 2 (Shard2) are split and merged into new shard 1 (Shard1'), new shard 2 (Shard2'), new shard 3 (Shard3'), specifically, Set the front of the original shard 1 (Shard1) as the new shard 1 (Shard1'), and merge the back of the original shard 1 (Shard1) and the front of the original shard 2 (Shard2) into the new shard 2 (Shard2'), set the back of the original shard 2 (Shard2) as the
  • Step S412 the server generates a performance level strategy of the expanded Solr cluster according to the sharding of each group of hosts.
  • the server generates a shard split and merge performance level strategy of the expanded Solr cluster according to the shards of each group of hosts.
  • An embodiment of the present invention provides a method and server for expanding a Solr cluster server based on an adaptive balancing strategy, and automatically generates a security-level expansion strategy based on the Solr cluster copy factor and host network transmission and other security factors; at the same time, each host in the Solr cluster is combined The efficiency of reading and writing indexes, as well as system load and other performance factors, proportionally distribute the hash range of Solr cluster fragments on each host, automatically generate a performance-level expansion strategy, and finally automatically execute the expansion strategy to make the Solr cluster generate the expansion strategy.
  • the system resources are more balanced and stable, which further improves the security and performance of the Solr cluster.
  • the technical solution adopted by the embodiments of the present invention includes a security level and performance level expansion strategy.
  • FIG. 5A is a schematic diagram of an implementation process of a method for generating a security level expansion strategy according to an embodiment of the present invention.
  • the method includes the following steps: step S501, the server obtains a copy factor of the current Solr cluster; step S502, the server The current Solr cluster security level is analyzed according to the replication factor; here, the server analyzes its security level based on the replication factor, whether the replication factor is greater than 1, if it is greater than 1, the cluster security is high, otherwise, if the replication factor is equal to 1, The Solr cluster is unstable. When a copy of the index data is damaged or the host where the copy is located is disconnected, the copy will be unavailable, which will directly cause some services of the Solr cluster to be unavailable.
  • the server periodically performs network transmission data sampling for each Solr host; here, the network transmission data sampling may be the latest N data sampling records of each Solr host performing network transmission, and N may be determined according to actual conditions.
  • step S504 the server calculates the security weight of each Solr host according to the sampling; here, the security weight of each Solr host can be counted by rules.
  • the rule can be that the network transmission good weight is 1, and the network transmission difference weight is 0.
  • step S505 the server analyzes the current security level of the Solr cluster according to the security weight; here, the security weight can be used to determine whether the network transmission is good or bad.
  • Step S506 The server obtains a security expansion strategy based on the security level analysis.
  • the expansion strategy of the security layer may be that Solr hosts with poor network stability, such as frequent network disconnection or slow network transmission speed, are not deployed as much as possible. If there are existing copies on such hosts, you can move to On other hosts with good network stability.
  • FIG. 5B is a schematic diagram of an implementation process of a method for generating a performance-level expansion strategy according to an embodiment of the present invention.
  • the method includes the following steps: Step S511: The server obtains the current Solr cluster fragment number; here, the server Get the number of shards in the current Solr cluster, the hash range of each shard, and the distribution of replica hosts corresponding to each shard.
  • step S512 the server analyzes the current Solr cluster performance level according to the distribution of the hosts corresponding to the number of shards; here, the server determines whether the current Solr cluster has multiple copies on one Solr host because there are too many copies Coexisting a host may cause a performance bottleneck; the server determines whether there are shards with a large hash range in the current Solr cluster, because the read and write efficiency of shards with a large hash range will be relatively slow.
  • step S513 the server periodically samples busy current performance data for each current Solr cluster host
  • Step S514 the server tests the speed of writing and querying the index of each current Solr cluster host separately;
  • Step S515 the server counts the read-write performance weight of each current Solr cluster host; here, the busy hour performance data is sampled periodically for each Solr host, and each Solr host is tested for its speed of writing and querying the index, and then statistics The weight of read and write performance of each Solr host.
  • a host with a large read-write performance weight indicates that its available performance space is large, and a copy corresponding to a shard with a large hash range can be deployed. Conversely, a host with a smaller read-write performance weight indicates that its available performance space is smaller, and only copies corresponding to shards with a smaller hash range can be deployed.
  • step S5166 the server analyzes the current host system load of the Solr cluster according to the performance level; here, the system resource status of each host in the current cluster and the condition of the co-located components are obtained, and the weight of each host system load is calculated therefrom.
  • Step S517 the server obtains the current resource status of each host system of the Solr cluster and the status of co-located components
  • Step S518, the server calculates the weight of each host system load according to the resource situation and the condition of the co-located components.
  • Step S519 the server obtains the available performance weight of each host of the cluster according to the read-write performance weight and load weight of each host system;
  • Step S520 The server derives the allocable hash range of each host according to the available performance weight.
  • Step S521 The server obtains a capacity expansion strategy based on the hash range and the Solr cluster distribution before capacity expansion.
  • the solutions of the embodiments of the present invention are mainly: based on the characteristics of the Solr cluster, the network transmission speed of the Solr host in the cluster, the performance of reading and writing index, and the analysis and calculation of the system load, the Solr cluster expansion at the security level and performance level is proposed Strategy.
  • the default security level expansion strategy has a higher priority than the performance level expansion strategy. That is, the security level expansion strategy is first combined with the current Solr cluster copy factor and the host network stability, and then based on the security level expansion strategy, combined The efficiency of each Solr cluster host to read and write indexes, as well as the system load, formulate a capacity expansion strategy.
  • An embodiment of the present invention proposes a schematic diagram of a system for expanding a Solr cluster server based on an adaptive balancing strategy.
  • the system formed by a collection of servers in a Solr cluster is shown in FIG. 6A.
  • An embodiment of the present invention proposes a system based on adaptive balancing Schematic diagram of the system for expanding the Solr cluster server strategy.
  • the system 60 includes: an information collection module 61, which is used to periodically collect the number of Solr hosts, basic information of the Solr cluster, and data sampling of each Solr host network transmission. The time-consuming data sampling of each Solr host read and write index, the system resources of each enterprise search service application server host, and the situation of co-located components.
  • the strategy configuration module 62 is used to store the expansion strategy at the security level and the performance level.
  • the strategy analysis module 63 is used to analyze the security data and performance data of the Solr cluster.
  • the policy execution module 64 is used to obtain the security level expansion strategy and the performance level expansion strategy from the policy configuration module, and execute them in sequence.
  • FIG. 6B is a schematic diagram of a server capacity expansion process provided by an embodiment of the present invention.
  • the method includes the following steps: Step S601, the system starts a data collection timer; the preset condition of the capacity expansion system is: in a Solr cluster One Solr host corresponds to one copy of one slice. The number of copies (ie, copy factor) of each slice is r. One slice corresponds to the range of one segment of the hash ring 0x00-0xffffffffff. The number of Solr hosts before expansion is X, and Solr is added after expansion. Number of hosts Y.
  • a data collection timer is started inside the information collection management module to periodically collect the number of Solr hosts, the basic information of the Solr cluster, the data sampling of each Solr host network transmission, and the consumption of read and write indexes of each Solr host Time data sampling, the system resources of each enterprise search service application server host, and the situation of co-located components, and will be stored in the relational database management server (Mysql) database as the basic data for subsequent analysis.
  • Mysql relational database management server
  • Step S602 the system starts the strategy analysis function; here, the strategy analysis module listens to the broadcast of the newly added Y Solr host, and then starts the strategy analysis function.
  • Step S603 the system backs up the Solr cluster;
  • the strategy analysis module first starts the backup operation to back up the metadata and index data of the entire Solr cluster;
  • Step S604 the system analyzes the security data; here, the strategy analysis module reads the basic data obtained in the first step and analyzes its security.
  • step S605 the system generates a security-level expansion strategy; here, the security-level expansion strategy is generated according to the security and written into the policy configuration module.
  • step S606 the system analyzes the performance data; the system strategy analysis module analyzes the index writing time-consuming data, and generates the write index performance weight value of each Solr host.
  • a Index1 Index2 Index3 Host1 T1_Host1 T2_Host1 T3_Host1 ... Host2 T1_Host2 T2_Host2 T3_Host2 ... Host3 T1_Host3 T1_Host3 T1_Host3 ... ... ... ... ... ... ... ... ... ...
  • the strategy analysis module analyzes the index reading time-consuming data and generates the reading index performance weight value of each Solr host.
  • the strategy analysis module reads the system resource usage of each Solr host and the condition of co-located components, and calculates the weight of the system load of each host; here, the strategy analysis module reads the system resource usage of each Solr host and co-located Components, such as CPU usage, memory usage, number of task queues, number of co-located components, etc., and use this to calculate the weight of each host system load, as shown in Table 5.
  • the strategy analysis module combines the write index weight, the read index weight and the system load weight to calculate the overall performance weight value of each Solr host; here, the strategy analysis module combines the write index weight, read index weight and system load weight to calculate the comprehensiveness of each Solr host Performance weight values, as shown in Table 6.
  • the strategy analysis module generates the expanded Solr cluster according to the comprehensive performance weight value of each Solr host.
  • FIG. 6C is a schematic diagram of a principle for generating a expanded Solr cluster according to an embodiment of the present invention.
  • the original Solr host 621 before expansion is performed.
  • Solr host 622 needs to be added. All Solr hosts in the original Solr host 621 and the added Solr host 622 are sorted and grouped according to the order of weight value from high to low, and the proportion of each group's comprehensive performance weight value is proportionally allocated. Ultimately, you get the first shard 623, the second shard 624, and the third shard 625.
  • the allocated hash range is the index range of each shard, and each shard is in each group of hosts Set the corresponding primary node (Leader) and secondary node (Replica), and generate an expanded Solr cluster.
  • (X+Y)/r (rounded) shards can be expanded; the overall performance weight value of each Solr host is sorted from high to low; then the sorted Solr host Grouping, starting from the first host, consecutive r hosts as a group; accumulate the comprehensive performance weight value of each group of hosts as the overall performance weight value of the group; according to the comprehensive performance weight value of each group, the hash ring 0x00 is proportionally assigned -0xfffffffff, the allocated hash range is the index range of each shard; then, for each shard, set the corresponding primary node (Leader) and secondary node (Replica) in each group of hosts, this is Solr cluster after expansion.
  • the strategy analysis module generates a performance-level expansion strategy for the Solr cluster before the expansion and the Solr cluster after the ninth step expansion, and writes to the strategy configuration module.
  • FIG. 6D is a schematic diagram of a principle for generating a performance-level expansion strategy according to an embodiment of the present invention. As shown in FIG. 6D, the host 631 before capacity expansion is split to obtain a shard 632 split by the host before capacity expansion, and a new shard 633 formed after the split is merged, and the new shard is allocated to the host 634 after capacity expansion .
  • Step S607 the system generates a capacity expansion strategy at the performance level
  • Step S608 the system executes the strategy; here, the strategy analysis module notifies the strategy execution module to obtain the security-level expansion strategy and the performance-level expansion strategy from the strategy configuration module to execute in sequence.
  • step S609 the system judges whether the execution is successful; here, if the execution fails, go to step S610; otherwise, if the execution is successful, notify the user that the expansion strategy is successfully generated.
  • Step S610 the system restores the Solr cluster backup
  • Step S611 the system sends a failure log notification; here, the system sends a failure log notification to the user.
  • step S612 the system sends a success notification for generating the expansion strategy; here, the system sends a failure log notification to the user.
  • Step S613 the system deletes the Solr cluster backup.
  • the embodiments of the present invention provide a server capacity expansion device.
  • the device includes each unit included, each module included in each unit, and each submodule included in each module, which can pass through the server.
  • the processor can be realized by the processor in the course; of course, it can also be realized by the logic circuit; in the implementation process, the processor can be the central processing unit (CPU), the microprocessor (MPU), the digital signal processor (DSP) or the field can Programming gate array (FPGA), etc.
  • the device 700 includes: a first obtaining unit 701 configured to obtain security level information of a current enterprise search application server Solr cluster
  • the first generating unit 702 is configured to generate a security level strategy for expanding the Solr cluster at the security level based on the security level information
  • the second obtaining unit 703 is configured to obtain performance level information of the current Solr cluster
  • the second generation unit 704 is configured to generate a performance-level strategy for expanding the Solr cluster at the performance level based on the performance-level information
  • the expansion unit 705 is configured to perform according to the security-level policy and the performance The layer strategy expands the Solr cluster.
  • the first obtaining unit is further configured to obtain copy information of the current Solr cluster; the first generating unit is further configured to generate increased Solr based on the copy information of the current Solr cluster Security level strategy for the number of cluster copies.
  • the first acquiring unit is further configured to acquire the transmission speed of each host of the current Solr cluster when performing data transmission; the first generating unit is also configured to be based on the current Solr The transmission speed of each host in the cluster during data transmission generates a security level strategy for transferring a copy of the Solr cluster on the host to a secure host.
  • the second acquiring unit is further configured to acquire the speed at which each host of the current Solr cluster reads and writes the index, and the system load of each host.
  • the second generating unit is further configured to: based on the speed at which each host of the current Solr cluster reads and writes the index, and the system load of each host, generate A performance-level strategy for expanding the capacity of the Solr cluster at the performance level.
  • the second generating unit includes: a calculation module configured to calculate each unit of the current Solr cluster based on the sum of the first weight value, the second weight value, and the third weight value of each host The overall performance weight value of the host; the generation module is configured to generate a performance level strategy for expanding the performance level of the Solr cluster based on the overall performance weight value of each host of the current Solr cluster.
  • the apparatus further includes: a first determining unit configured to determine a host in the current Solr cluster as a reference host; a second determining unit configured to determine each host in the current Solr cluster The ratio of the speed at which the host reads the index to the speed at which the reference host reads the index determines the first weight value of the host; the third determining unit is configured to determine the index writing speed and the index of each host in the current Solr cluster The ratio of the speed at which the reference host writes the index determines the second weight value of the host; the fourth determination unit is configured to be based on the ratio of the system load of each host in the current Solr cluster to the system load of the reference host To determine the third weight value of the host.
  • the generation module includes: a sorting sub-module configured to sort the overall performance weight value of each host of the current Solr cluster; a grouping sub-module configured to perform the sorting according to the result of the sorting, Group each of the hosts; the accumulation submodule is configured to accumulate the comprehensive performance weight value of each host in each group as the group performance weight value of the group of hosts; assign the submodule to be configured according to each group The group performance weight value of the host is an address assigned to each group of hosts; a submodule is generated, configured to generate a performance level strategy for expanding the performance level of the Solr cluster according to the address of each group of hosts.
  • the generation sub-module is further configured to: according to the address of each group of hosts, allocate the fragments of the Solr cluster to each group of hosts; according to the fragments of each group of hosts To generate the performance-level strategy of the expanded Solr cluster.
  • the above method is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
  • the computer software products are stored in a storage medium and include several instructions for A network-side switching device (which may be a terminal or a network, etc.) executes all or part of the methods described in the embodiments of the present invention.
  • the foregoing storage media include various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (Read Only Memory, ROM), a magnetic disk, or an optical disk.
  • program codes such as a USB flash drive, a mobile hard disk, a read-only memory (Read Only Memory, ROM), a magnetic disk, or an optical disk.
  • an embodiment of the present invention provides a server capacity expansion server, including a memory and a processor, the memory stores a computer program that can run on the processor, and the processor implements the program to implement the server Steps in the capacity expansion allocation method, or the above-mentioned server capacity expansion method.
  • an embodiment of the present invention provides a computer-readable storage medium in which a computer program is stored, and the computer program is configured to execute the steps in the above-mentioned server capacity expansion method when executed.
  • an embodiment of the present invention provides a computer program product
  • the computer program product includes a computer program stored on a non-transitory computer readable storage medium
  • the computer program includes program instructions, and when the program instructions are When executed, the computer is caused to execute the method described in the above aspects.
  • FIG. 8 is a schematic diagram of a hardware entity for generating an expansion strategy server in an embodiment of the present invention.
  • the hardware entity of the server 800 includes: a processor 801, a communication interface 802, and a memory 803, where The processor 801 generally controls the overall operation of the server 800.
  • the communication interface 802 can enable the server to communicate with other terminals or servers through the network.
  • the memory 803 is configured to store instructions and applications executable by the processor 801, and may also cache data to be processed or processed by the modules in the processor 801 and the server 800 (for example, image data, audio data, voice communication data, and video Communication data) can be achieved through flash memory (FLASH) or random access memory (RandomAccessMemory, RAM).
  • the disclosed device and method may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the units is only a division of logical functions.
  • the displayed or discussed components are coupled to each other, or directly coupled, or the communication connection may be through some interfaces, and the indirect coupling or communication connection of the device or unit may be electrical, mechanical, or other forms of.
  • the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units; they may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above integration
  • the unit can be implemented in the form of hardware, or in the form of hardware plus software functional units.
  • the foregoing program may be stored in a computer-readable storage medium.
  • the execution includes The steps of the foregoing method embodiments; and the foregoing storage medium includes various media that can store program codes, such as a mobile storage device, a read-only memory (Read Only Memory, ROM), a magnetic disk, or an optical disk.
  • ROM Read Only Memory
  • the above integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
  • the computer software products are stored in a storage medium and include several instructions for One server executes all or part of the methods described in various embodiments of the present invention.
  • the foregoing storage media include various media that can store program codes, such as mobile storage devices, ROM, magnetic disks, or optical disks.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明实施例公开了一种服务器的扩容方法及装置、服务器、存储介质，所述方法包括：获取当前企业级搜索应用服务器Solr集群的安全层面信息；基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略；获取当前Solr集群的性能层面信息；基于所述性能层面信息，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略；根据所述安全层面策略和所述性能层面策略对所述Solr集群进行扩容。

Description

服务器的扩容方法及装置、服务器、存储介质
交叉引用
本发明要求在2018年12月26日提交至中国专利局、申请号为201811604894.X、发明名称为“服务器的扩容方法及装置、服务器、存储介质”的中国专利申请的优先权，该申请的全部内容通过引用结合在本发明中。
技术领域
本发明实施例涉及分布式搜索引擎技术，尤其涉及服务器的扩容方法及装置、服务器、存储介质。
背景技术
企业级搜索应用服务器(Solr)是一款用计算机编程语言(Java)开发，基于全文搜索引擎(Lucene)的高性能全文搜索服务，在Lucene基础上对其进行了扩展，提供了比其更为丰富的查询语言，支持可配置和可扩展并对查询性能进行了优化，同时它提供了一个完善的功能管理界面，是一款非常优秀的全文搜索服务。
Solr集群是一种分布式系统，它具有一个或多个分片，每个分片拥有一个哈希范围，所有分片组成一个完整的哈希环，该哈希环的范围为0x00-0xffffffff，分片是一种虚拟的范围概念，每个分片会对应一个或多个副本，而副本则是实际存在主机上的物理节点。
随着索引数量越来越大，索引写入和搜索的响应时间会变得越来越慢，此时则需要对Solr集群进行扩容。而目前相关技术中还没有提出一种针对Solr集群的扩容方法。
发明内容
有鉴于此，本发明实施例的技术方案是这样实现的：
第一方面，本发明实施例提供一种服务器的扩容方法，所述方法包括：获取当前企业级搜索应用服务器Solr集群的安全层面信息；基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略；获取当前Solr集群的性能层面信息；基于所述性能层面信息，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略；根据所述安全层面策略和所述性能层面策略对所述Solr集群进行扩容。
第二方面，本发明实施例提供一种服务器的扩容装置，所述装置包括：第一获取单元，配置为获取当前企业级搜索应用服务器Solr集群的安全层面信息；第一生成单元，配置为基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略；第二获取单元，配置为获取当前Solr集群的性能层面信息；第二生成单元，配置为基于所述性能层面信息，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略；扩容单元，配置为根据所述安全层面策略和所述性能层面策略对所述Solr集群进行扩 容。
第三方面，本发明实施例提供一种服务器的扩容服务器，包括存储器和处理器，所述存储器存储有可在处理器上运行的计算机程序，所述处理器执行所述程序时实现上述所述生成扩容策略方法中的步骤。
第四方面，本发明实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，所述计算机程序配置为执行时实现上述服务器的扩容方法中的步骤。
第五方面，本发明实施例提供一种计算机程序产品，所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，当所述程序指令被计算机执行时，使所述计算机执行以上各个方面所述的方法。
附图说明
图1为本发明实施例提供一种服务器的扩容方法的实现流程示意图；
图2为本发明实施例提供又一种服务器的扩容方法的实现流程示意图；
图3A为本发明实施例又一种服务器的扩容方法的实现流程示意图；
图3B为确定权重值的方法实现流程示意图；
图4为本发明实施例又一种服务器的扩容方法的实现流程示意图；
图5A为本发明实施例提供一种生成安全层面扩容策略的方法的实现流程示意图；
图5B为本发明实施例提供一种生成性能层面扩容策略的方法的实现流程示意图；
图6A为本发明实施例提出了一种基于自适应平衡策略进行Solr集群服务器的扩容系统示意图；
图6B为本发明实施例提供一种服务器的扩容实现流程示意图；
图6C为本发明实施例提供一种生成扩容后的Solr集群的原理示意图；
图6D为本发明实施例提供一种生成性能层面扩容策略的原理示意图；
图7本发明实施例一种服务器的扩容装置的组成结构示意图；
图8为本发明实施例中生成扩容策略服务器的一种硬件实体示意图。
具体实施方式
目前扩容的方法仅仅是根据当前Solr集群的分片及副本情况，并结合新增Solr主机，进行手工切分。该扩容方法具有以下缺陷：
1、手工切分十分复杂且极易出错；
2、扩容未考虑Solr集群副本因子的安全性因素；
3、扩容未考虑当前各个Solr主机的网络传输情况；
4、扩容未考虑当前各个Solr主机的读写索引性能；
5、扩容未考虑当前各个Solr主机的系统负荷情况。
基于以上缺陷，本发明实施例提出了一种基于自适应平衡策略的服务器的扩容方法和系统，使得Solr集群遇到安全或性能瓶颈时，能够依据Solr 集群自身特点，所处集群环境自有特征，结合一定扩容规则，自动规划和执行扩容策略，从而使得Solr集群扩容后的安全和性能更优化。
下面结合附图和具体实施例对本发明的技术方案进一步详细阐述。
实施例一
本发明实施例提供一种服务器的扩容方法，该方法应用于服务器，该方法所实现的功能可以通过终端中的处理器调用程序代码来实现，当然程序代码可以保存在计算机存储介质中，可见，该服务器至少包括处理器和存储介质。
图1为本发明实施例提供一种服务器的扩容方法的实现流程示意图，如图1所示，该方法包括：步骤S101，服务器获取当前Solr集群的安全层面信息；这里，服务器这里不做限定，可以是一个服务器，也可以是多个服务器，所述安全层面信息包括但不限定为Solr集群的基本信息和每台Solr主机网络进行数据传输时的传输速度，所述Solr集群的基本信息可以为Solr集群的副本信息，Solr集群的副本因子，当前Solr集群的分片数，每个分片的哈希范围，每个分片对应副本在各个Solr主机上的分布情况。获取当前Solr集群的安全层面信息可以为获取当前Solr集群的副本信息或获取当前Solr集群的每台Solr主机网络进行数据传输时的传输速度。
步骤S102，服务器基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略；基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略，为了方便理解，这里举例说明，例如，所述安全层面信息为Solr集群的副本信息，服务器获取当前Solr集群的副本信息中的副本因子，分析其安全层面，副本因子是否大于1，如果大于1，则集群安全性高，这时服务器则不需要生成一条增加Solr集群副本因子的策略；反之，如果副本因子等于1，则Solr集群是不稳定的，可能是某一个副本出现索引数据损坏，或所在主机断网，该副本将不可用，会直接导致Solr集群的部分业务不可用，这时服务器则生成一条增加Solr集群副本因子的策略，即基于所述安全层面信息中的副本信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略。
基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略，为了方便理解，这里举例说明，例如，所述安全层面信息为Solr集群的每台主机进行数据传输时的传输速度，通过定时针对每台Solr主机进行网络传输数据采样，获取当前Solr集群的每台主机进行数据传输时的传输速度，统计每台Solr主机的安全权重。可以读取各个Solr主机进行网络传输的最近N条数据采样记录，用Good表示网络传输良好，Bad表示网络传输差；然后根据规则统计出每台Solr主机的安全权重，例如全Good权重为1，超过M％的Bad则权重为0，M％为容忍度，可以根据网络传输的实际情况进行设定；最后生成一条将安全权重为0的主机上的Solr集群副本移至安全权重为1的主机上的策略，即副本转移安全主机策略，或者对于经常断网，或者网络传输速度较慢等网络稳定性差的Solr主机，尽量不部署Solr副本，如已有副本在此类主机上，则可移至其他网络稳定性好的主机上。从 而基于所述安全层面信息中的副本信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略。
步骤S103，服务器获取当前Solr集群的性能层面信息；这里，所述性能层面信息包括但不限定为每台Solr主机读写索引的耗时数据采样，每台Solr主机的系统资源情况，以及合设组件的情况，当前Solr集群的分片数，每个分片的哈希范围，每个分片对应副本的主机分布情况。获取当前Solr集群的性能层面信息可以为获取当前Solr集群的每台主机对索引进行读和写的速度以及每台主机的系统负荷。
步骤S104，服务器基于所述性能层面信息，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略；这里，基于所述性能层面信息，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略，为了方便理解，这里举例说明，例如，基于所述性能层面信息可以为基于所述性能层面信息中的当前Solr集群的每台主机对索引进行读和写的速度以及每台主机的系统负荷，基于所述性能层面信息中的当前Solr集群的每台主机对索引进行读和写的速度以及每台主机的系统负荷生成用于对所述Solr集群在性能层面进行扩容的性能层面策略。
步骤S105，服务器根据所述安全层面策略和所述性能层面策略对所述Solr集群进行扩容。
这里，根据所述安全层面策略和所述性能层面策略对所述Solr集群进行扩容可以是服务器获取安全层面扩容策略和性能层面扩容策略依次执行的过程。
在其他实施例，所述获取当前Solr集群的安全层面信息，包括：获取当前Solr集群的副本信息；所述基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略，包括：基于所述当前Solr集群的副本信息，生成增加Solr集群的副本数量的安全层面策略。
这里，服务器基于所述当前Solr集群的副本信息，生成增加Solr集群的副本数量的安全层面策略。为了方便理解，这里举例说明，例如，所述安全层面信息为Solr集群的副本信息，服务器获取当前Solr集群的副本信息中的副本因子，分析其安全层面，副本因子是否大于1，如果大于1，则集群安全性高，这时服务器则不需要生成一条增加Solr集群副本因子的策略；反之，如果副本因子等于1，则Solr集群是不稳定的，可能是某一个副本出现索引数据损坏，或所在主机断网，该副本将不可用，会直接导致Solr集群的部分业务不可用，这时服务器则生成一条增加Solr集群副本因子的策略，即基于所述安全层面信息中的副本信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略。
在其他实施例，所述获取当前Solr集群的安全层面信息，包括：获取当前Solr集群的每台主机进行数据传输时的传输速度；所述基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略，包括：基于所述当前Solr集群的每台主机进行数据传输时的传输速度，生成将主机上的Solr集群的副本转移至安全主机的安全层面策略。
这里，服务器基于所述当前Solr集群的每台主机进行数据传输时的传输速度，生成将主机上的Solr集群的副本转移至安全主机的安全层面策略。为了方便理解，这里举例说明，例如，所述安全层面信息为Solr集群的每台主机进行数据传输时的传输速度，通过定时针对每台Solr主机进行网络传输数据采样，获取当前Solr集群的每台主机进行数据传输时的传输速度，统计每台Solr主机的安全权重。可以读取各个Solr主机进行网络传输的最近N条数据采样记录，用Good表示网络传输良好，Bad表示网络传输差；然后根据规则统计出每台Solr主机的安全权重，例如全Good权重为1，超过M％的Bad则权重为0，M％为容忍度，可以根据网络传输的实际情况进行设定；最后生成一条将安全权重为0的主机上的Solr集群副本移至安全权重为1的主机上的策略，即副本转移安全主机策略，或者对于经常断网，或者网络传输速度较慢等网络稳定性差的Solr主机，尽量不部署Solr副本，如已有副本在此类主机上，则可移至其他网络稳定性好的主机上。从而基于所述安全层面信息中的副本信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略。
实施例二
图2为本发明实施例提供又一种服务器的扩容方法的实现流程示意图，如图2所示，该方法包括：步骤S201，服务器获取当前Solr集群的安全层面信息；步骤S202，服务器基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略；步骤S203，服务器获取当前Solr集群的每台主机对索引进行读和写的速度，和，每台主机的系统负荷；这里，当前Solr集群的每台主机对索引进行读和写的速度，和，每台主机的系统负荷可以都存在关系型数据库中，服务器可以从关系型数据库中获取当前Solr集群的每台主机对索引进行读和写的速度，和，每台主机的系统负荷。
步骤S204，服务器基于所述当前Solr集群的每台主机对索引进行读和写的速度，以及所述每台主机的系统负荷，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略；这里，服务器基于所述当前Solr集群的每台主机对索引进行读和写的速度，以及所述每台主机的系统负荷，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略可以通过服务器分析当前Solr集群的每台主机对索引进行读和写的速度，以及所述每台主机的系统负荷，利用分析的结果生成用于对所述Solr集群在性能层面进行扩容的性能层面策略。
步骤S205，服务器根据所述安全层面策略和所述性能层面策略对所述Solr集群进行扩容。
实施例三
图3A为本发明实施例又一种服务器的扩容方法的实现流程示意图，如图3A所示，该方法包括：步骤S301，服务器获取当前Solr集群的安全层面信息；步骤S302，服务器基于所述安全层面信息，生成用于对所述Solr集群 在安全层面进行扩容的安全层面策略；步骤S303，服务器获取当前Solr集群的每台主机对索引进行读和写的速度，和，每台主机的系统负荷；步骤S304，服务器根据每台主机的第一权重值、第二权重值和第三权重值的总和，计算当前Solr集群的每台主机的综合性能权重值；这里，所述第一权重值可以通过选择一台Solr主机读索引的耗时作为基准权重，计算其他主机的读索引测试耗时与基准执行的时间比。
所述第二权重值可以通过选择一台Solr主机写索引的耗时作为基准权重，计算其他主机的读索引测试耗时与基准执行的时间比。
所述第三权重值可以通过读取每台Solr主机的系统资源使用情况，以及合设组件的情况，例如CPU使用、内存使用、任务队列数、合设组件数等等，并以此计算得到各个主机系统负荷的权重值。
根据每台主机的第一权重值、第二权重值和第三权重值的总和，计算当前Solr集群的每台主机的综合性能权重值可以通过将每台主机的第一权重值、第二权重值和第三权重值进行相加，所述相加结果即为当前Solr集群的每台主机的综合性能权重值。
步骤S305，服务器根据所述当前Solr集群的每台主机的综合性能权重值，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略；
这里，服务器可以通过对当前Solr集群的每台主机的综合性能权重值进行排序，排序可以是综合性能权重值按照从高到低或从低到高排，排序后对Solr主机分组，从第一台主机开始，连续r台主机为一组，r可以为每个切片的副本数，累加每组主机的综合性能权重值作为该组的综合性能权重值，利用所述综合性能权重值生成用于对所述Solr集群在性能层面进行扩容的性能层面策略。
步骤S306，服务器根据所述安全层面策略和所述性能层面策略对所述Solr集群进行扩容。
在其他实施例，图3B为确定权重值的方法实现流程示意图，如图3B所示，所述方法还包括：步骤S311，服务器确定当前Solr集群中的一台主机为基准主机；这里，确定当前Solr集群中的一台主机为基准主机可以在当前Solr集群任意选择一台为基准主机。
步骤S312，服务器根据所述当前Solr集群中的每台主机读索引的速度与所述基准主机读索引的速度的比值，确定该主机的第一权重值；这里，所述当前Solr集群中的每台主机读索引的速度可以是针对每台Solr主机进行忙时性能数据采样，分析读索引，耗时数据将所述基准主机读索引的速度为参照物，将当前Solr集群中的每台主机读索引的速度与所述基准主机读索引的速度进行比较，得到的比值就是该主机的第一权重值。
步骤S313，服务器根据所述当前Solr集群中的每台主机写索引的速度与所述基准主机写索引的速度的比值，确定该主机的第二权重值；这里，所述当前Solr集群中的每台主机写索引的速度可以是针对每台Solr主机进行忙时性能数据采样，分别对每台Solr主机测试其写入和查询索引的速度，耗时数据将所述基准主机写索引的速度为参照物，将当前Solr集群中的每台主机写 索引的速度与所述基准主机写索引的速度进行比较，得到的比值就是该主机的第二权重值。
步骤S314，服务器根据所述当前Solr集群中的每台主机的系统负荷与所述基准主机的系统负荷的比值，确定该主机的第三权重值。
这里，所述当前Solr集群中的每台主机的系统负荷可以是每台Solr主机的系统资源使用情况，以及合设组件的情况，例如CPU使用、内存使用、任务队列数、合设组件数等等，根据所述当前Solr集群中的每台主机的系统负荷与所述基准主机的系统负荷的比值，确定该主机的第三权重值可以通过将当前Solr集群中的每台主机系统负荷与所述基准主机系统负荷进行比较，得到的比值就是该主机的第三权重值。
实施例四
图4为本发明实施例又一种服务器的扩容方法的实现流程示意图，如图4所示，该方法包括：步骤S401，服务器获取当前Solr集群的安全层面信息；步骤S402，服务器基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略；步骤S403，服务器获取当前Solr集群的每台主机对索引进行读和写的速度，和，每台主机的系统负荷；步骤S404，服务器根据每台主机的第一权重值、第二权重值和第三权重值的总和，计算当前Solr集群的每台主机的综合性能权重值；步骤S405，服务器将所述当前Solr集群的每台主机的综合性能权重值进行排序；这里，所述排序可以是综合性能权重值按照从高到低或从低到高排。
步骤S406，服务器根据所述排序的结果，将所述每台主机进行分组；这里，所述分组可以是从第一台主机开始，连续r台主机为一组，r可以为每个切片的副本数或副本因子，副本数与副本因子可以相同，也可以不同。
步骤S407，服务器将每组中的每台主机的综合性能权重值累加作为该组主机的组性能权重值；这里，所述该组主机的组性能权重值可以是将组中的每台主机的综合性能权重值进行相加。
步骤S408，服务器根据所述每组主机的组性能权重值为每组主机分配地址；这里，可以根据每组的综合性能权重值按比例分配地址0x00-0xffffffff，分配后的地址范围即为各个分片(Shard)的索引范围；然后，针对每个分片在每组主机中设置对应的主节点(Leader)和副节点(Replica)。
步骤S409，服务器根据每组主机的所述地址，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略；步骤S410，服务器根据所述安全层面策略和所述性能层面策略对所述Solr集群进行扩容。
在其他实施例，所述根据每组的所述地址，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略，包括：步骤S411，服务器根据每组主机的所述地址，将所述Solr集群的分片分配至所述每组主机；这里，服务器根据每组主机的所述地址，将所述Solr集群的分片分配至所述每组主机可以通过将所述Solr集群的原始分片1(Shard1)和原始分片2(Shard2)经过拆分 合并成新分片1(Shard1’)，新分片2(Shard2’)，新分片3(Shard3’)，具体来说，将原始分片1(Shard1)的前部设置为新分片1(Shard1’)，将原始分片1(Shard1)的后部和原始分片2(Shard2)的前部合并为新分片2(Shard2’)，将原始分片2(Shard2)的后部设置为新分片3(Shard3’)，再根据每组主机的所述地址，将所述Solr集群的新分片分配至所述每组主机。
步骤S412，服务器根据所述每组主机的分片，生成所述扩容后的Solr集群的性能层面策略。
这里，服务器根据所述每组主机的分片，生成所述扩容后的Solr集群的分片拆分合并性能层面策略。
实施例五
本发明实施例提供了一种基于自适应平衡策略进行Solr集群服务器的扩容方法和服务器，根据Solr集群副本因子，以及主机网络传输等安全因素，自动生成安全层面扩容策略；同时结合Solr集群各主机读写索引的效率，以及系统负荷情况等性能因素，按比例分配Solr集群分片在各主机上的哈希范围，自动生成性能层面扩容策略，最终自动执行扩容策略，使得Solr集群生成扩容策略后的系统资源更加平衡和稳定，进一步提升了Solr集群的安全和性能。本发明实施例采用的技术方案包括安全层面和性能层面的扩容策略。
图5A为本发明实施例提供一种生成安全层面扩容策略的方法的实现流程示意图，如图5A所示，该方法包括以下步骤：步骤S501，服务器获取当前Solr集群的副本因子；步骤S502，服务器依据所述副本因子分析当前Solr集群安全层面；这里，服务器依据所述副本因子分析其安全层面，副本因子是否大于1，如果大于1，则集群安全性高，反之，如果副本因子等于1，则Solr集群是不稳定的，当某一个副本出现索引数据损坏，或该副本所在主机断网时，该副本将不可用，会直接导致Solr集群的部分业务不可用。
步骤S503，服务器定时针对每台Solr主机进行网络传输数据采样；这里，所述网络传输数据采样可以为各个Solr主机进行网络传输的最近N条数据采样记录，N可以根据实际情况进行确定。
步骤S504，服务器依据所述采样统计每台Solr主机的安全权重；这里，统计每台Solr主机的安全权重可以通过规则统计，规则可以是网络传输良好权重为1，网络传输差权重为0。
步骤S505，服务器依据所述安全权重分析当前Solr集群安全层面；这里，可以通过所述安全权重确定网络传输是好还是差。
步骤S506，服务器依据所述安全层面的分析，得出安全层面的扩容策略。
这里，所述安全层面的扩容策略可以是对于经常断网，或者网络传输速度较慢等网络稳定性差的Solr主机，尽量不部署Solr副本，如已有副本在此类主机上，则可移至其他网络稳定性好的主机上。
图5B为本发明实施例提供一种生成性能层面扩容策略的方法的实现流程示意图，如图5B所示，该方法包括以下步骤：步骤S511，服务器获取当 前Solr集群的分片数；这里，服务器获取当前Solr集群的分片数，每个分片的哈希范围，每个分片对应副本的主机分布情况。
步骤S512，服务器依据所述分片数对应副本的主机分布情况，分析当前Solr集群性能层面；这里，服务器判断当前Solr集群是否存在一台Solr主机上具备多个副本的情况，因为过多的副本并存一台主机可能导致性能瓶颈；服务器判断当前Solr集群是否存在哈希范围较大的分片，因为较大哈希范围的分片读写效率都会相对较慢。
步骤S513，服务器定时针对每台当前Solr集群主机进行忙时性能数据采样；
步骤S514，服务器分别对每台当前Solr集群主机测试其写入和查询索引的速度；
步骤S515，服务器统计每台当前Solr集群主机的读写性能权重；这里，定时针对每台Solr主机进行忙时性能数据采样，分别对每台Solr主机测试其写入和查询索引的速度，然后统计每台Solr主机的读写性能权重。
读写性能权重较大的主机，说明其可用性能空间较大，可以部署哈希范围较大的分片对应的副本。反之，读写性能权重较小的主机，说明其可用性能空间较小，只能部署哈希范围较小的分片对应的副本。
步骤S516，服务器依据所述性能层面分析当前Solr集群的主机系统负荷；这里，获取当前集群各台主机的系统资源情况，以及合设组件的情况，并以此计算各个主机系统负荷的权重。
有较多资源空余和合设组件较少的主机，来部署哈希范围较大的分片对应的副本。反之，则只能部署哈希范围较小的分片对应的副本。
步骤S517，服务器获取当前Solr集群各台主机系统的资源情况和合设组件的情况；
步骤S518，服务器依据资源情况和合设组件的情况计算各个主机系统负荷的权重。
步骤S519，服务器依据各个主机系统读写性能权重和负荷的权重，得到集群各个主机的可用性能权重；
步骤S520，服务器依据所述可用性能权重推出各个主机的可分配哈希范围。
步骤S521，服务器依据所述哈希范围和扩容前的Solr集群分布，得出性能层面的扩容策略。
实施例六
本发明实施例的解决方案主要是：基于Solr集群自身特点，所处集群Solr主机的网络传输速度，读写索引性能，以及系统负荷进行分析和计算，提出了安全层面和性能层面的Solr集群扩容策略。考虑到索引数据的重要性，默认安全层面扩容策略的优先级高于性能层面扩容策略，即首先结合当前Solr集群副本因子和主机网络稳定性制定安全层面扩容策略，然后基于安全层面扩容策略，结合各个Solr集群主机读写索引的效率，以及系统负荷情况，制 定性能层面的扩容策略。
本发明实施例提出了一种基于自适应平衡策略进行Solr集群服务器的扩容系统示意图，该系统Solr集群中各服务器的集合形成的系统，图6A为本发明实施例提出了一种基于自适应平衡策略进行Solr集群服务器的扩容系统示意图，如图6A所示，该系统60包括：信息采集模块61，用于定期采集Solr主机数、Solr集群的基本信息、每台Solr主机网络传输的数据采样、每台Solr主机读写索引的耗时数据采样、每台企业级搜索服务应用服务器主机的系统资源情况,以及合设组件的情况。
策略配置模62，用于存储安全层面和性能层面扩容策略。
策略分析模块63，用于分析Solr集群的安全类数据和性能类数据。
策略执行模块64，用于从策略配置模块中获取安全层面扩容策略和性能层面扩容策略，并依次执行。
图6B为本发明实施例提供一种服务器的扩容实现流程示意图，如图6B所示，该方法包括以下步骤：步骤S601，系统启动数据采集定时器；扩容系统的预置条件为：Solr集群中一台Solr主机对应一个切片的一个副本，每个切片的副本数(即副本因子)为r，一个切片对应哈希环0x00-0xffffffff一个片断的范围，扩容前Solr主机数X，扩容后增加Solr主机数Y。
这里，系统启动时，信息采集管理模块内部会启动一个数据采集定时器，定期采集Solr主机数、Solr集群的基本信息、每台Solr主机网络传输的数据采样、每台Solr主机读写索引的耗时数据采样、每台企业级搜索服务应用服务器主机的系统资源情况,以及合设组件的情况，并将存入关系型数据库管理服务器(Mysql)数据库，作为后续分析的基础数据。
步骤S602，系统启动策略分析功能；这里，策略分析模块监听到新增了Y台Solr主机的广播，则开始启动策略分析功能。
步骤S603，系统备份Solr集群；这里，策略分析模块先启动备份操作，备份整个Solr集群的元数据和索引数据；
步骤S604，系统分析安全类数据；这里，策略分析模块读取第一步获取的基础数据，分析其安全性。
步骤S605，系统生成安全层面的扩容策略；这里，依据安全性生成安全层面的扩容策略，写入策略配置模块。
首先，获取Solr集群的副本因子，如果为1，则生成一条增加Solr集群副本因子的策略。
接着，读取各个Solr主机进行网络传输的最近N条数据采样记录，如表1，其中Good表示网络传输良好，Bad表示网络传输差；然后根据规则统计出每台Solr主机的安全权重，例如全Good权重为1，超过M％的Bad则权重为0，如表2；最后生成一条将安全权重为0的主机上的Solr集群副本移至安全权重为1的主机上的策略，即副本转移安全主机策略。
表1各Solr主机网络传输采样表
  Host1 Host2 Host3 …
Time1 Good Bad Good …
Time2 Good Good Good …
Time3 Good Bad Good …
… … … … …
TimeN Good Good Good …
表2各Solr主机网络传输的安全权重表
  Host1 Host2 Host3 …
Safe 1 0 1 …
步骤S606，系统分析性能类数据；系统策略分析模块分析写索引耗时数据，并生成各Solr主机的写索引性能权重值。
首先，读取每台Solr主机写不同索引数据(Index1,Index2,Index3...)的耗时数据采样，如表3。
表3各Solr主机执行基准写索引测试的耗时表
  Index1 Index2 Index3 ...
Host1 T1_Host1 T2_Host1 T3_Host1 ...
Host2 T1_Host2 T2_Host2 T3_Host2 ...
Host3 T1_Host3 T1_Host3 T1_Host3 ...
... ... ... ... ...
接着，选择一台Solr主机写索引的耗时作为基准权重，计算其他主机的写索引测试耗时与基准执行的时间比，完成写索引权重表，如表4。
表4各Solr主机写索引的性能权重表
  Write_Weight1 Write_Weigh2 Write_Weight3 …
Host1 1 1 1 …
Host2 T1_Host2/T1_Host1 T2_Host2/T2_Host1 T3_Host2/T3_Host1 …
Host3 T1_Host3/T1_Host1 T2_Host3/T2_Host1 T3_Host3/T3_Host1 …
… … … … …
策略分析模块分析读索引耗时数据，并生成各Solr主机的读索引性能权重值。
首先，读取每台Solr主机读不同索引数据(Index1,Index2,Index3...)的耗时数据采样；接着，选择一台Solr主机读索引的耗时作为基准权重，计算其他主机的读索引测试耗时与基准执行的时间比，完成读索引权重表。
策略分析模块读取每台Solr主机的系统资源使用情况，以及合设组件的情况，计算各个主机系统负荷的权重；这里，策略分析模块读取每台Solr主 机的系统资源使用情况，以及合设组件的情况，例如CPU使用、内存使用、任务队列数、合设组件数等等，并以此计算各个主机系统负荷的权重，如表5。
表5各Solr主机资源使用和合设组件判断
  Cpu_used Mem_used Num_task Num_Comp ...
Host1 Cpu1 Mem1 Task1 Num_Comp1 ...
Host2 Cpu2 Mem2 Task2 Num_Comp2 ...
Host3 Cpu3 Mem3 Task3 Num_Comp3 ...
... ... ... ... ... ...
策略分析模块结合写索引权重，读索引权重和系统负荷权重，计算各Solr主机的综合性能权重值；这里，策略分析模块结合写索引权重，读索引权重和系统负荷权重，计算各Solr主机的综合性能权重值，如表6。
表6各Solr主机的综合性能权重表
  Host1 Host2 Host3 ...
Performance W1 W2 W3 ...
策略分析模块根据各Solr主机的综合性能权重值，生成扩容后的Solr集群。
这里，策略分析模块根据各Solr主机的综合性能权重值进行计算，图6C为本发明实施例提供一种生成扩容后的Solr集群的原理示意图，如图6C所示，对扩容前原Solr主机621进行扩容后需要增加Solr主机622，将原Solr主机621和增加的Solr主机622中的全部Solr主机按照权重值从高到低的顺序进行排序并分组，按每组的综合性能权重值按比例分配哈希环，得到第一分片623、第二分片624和第三分片625，分配后的哈希范围即为各个分片(Shard)的索引范围，针对每个分片在每组主机中设置对应的主节点(Leader)和副节点(Replica)，并生成扩容后的Solr集群。
根据预置条件计算新增主机后能扩容出(X+Y)/r(取整)个分片；将各Solr主机的综合性能权重值按照从高到低排序；然后对排序后的Solr主机分组，从第一台主机开始，连续r台主机为一组；累加每组主机的综合性能权重值作为该组的综合性能权重值；根据每组的综合性能权重值按比例分配哈希环0x00-0xffffffff，分配后的哈希范围即为各个分片(Shard)的索引范围；然后，针对每个分片在每组主机中设置对应的主节点(Leader)和副节点(Replica)，这就是扩容后的Solr集群。
策略分析模块针对扩容前的Solr集群和第九步扩容后的Solr集群，生成性能层面的扩容策略，写入策略配置模块，图6D为本发明实施例提供一种生成性能层面扩容策略的原理示意图，如图6D所示，对扩容前的主机631进行拆分，得到扩容前主机拆分的分片632，拆分后合并形成的新分片633，将新分片分配给扩容后的主机634。
将原始分片1(Shard1)和原始分片2(Shard2)经过拆分合并成新分片1(Shard1’)，新分片2(Shard2’)，新分片3(Shard3’)，具体来说，将原始分片1(Shard1)的前部设置为新分片1(Shard1’)，将原始分片1(Shard1)的后部和原始分片2(Shard2)的前部合并为新分片2(Shard2’)，将原始分片2(Shard2)的后部设置为新分片3(Shard3’)，即生成Solr集群分片拆分合并策略；
步骤S607，系统生成性能层面的扩容策略；
步骤S608，系统执行策略；这里，策略分析模块通知策略执行模块，从策略配置模块中获取安全层面扩容策略和性能层面扩容策略依次执行。
步骤S609，系统判断是否执行成功；这里，若执行失败，则转至步骤S610；反之，若执行成功，则通知用户生成扩容策略成功。
步骤S610，系统恢复Solr集群备份；
步骤S611，系统发送失败日志通知；这里，系统发送失败日志通知给用户。
步骤S612，系统发送生成扩容策略成功通知；这里，系统发送失败日志通知给用户。
步骤S613，系统删除Solr集群备份。
基于前述的实施例，本发明实施例提供一种服务器的扩容装置，该装置包括所包括的各单元、以及各单元所包括的各模块，以及各模块所包括的各子模块，都可以通过服务器中的处理器来实现；当然也可通过的逻辑电路实现；在实施的过程中，处理器可以为中央处理器(CPU)、微处理器(MPU)、数字信号处理器(DSP)或现场可编程门阵列(FPGA)等。
图7本发明实施例一种服务器的扩容装置的组成结构示意图，如图7所示，所述装置700包括：第一获取单元701，配置为获取当前企业级搜索应用服务器Solr集群的安全层面信息；第一生成单元702，配置为基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略；第二获取单元703，配置为获取当前Solr集群的性能层面信息；第二生成单元704，配置为基于所述性能层面信息，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略；扩容单元705，配置为根据所述安全层面策略和所述性能层面策略对所述Solr集群进行扩容。
在其他的实施例中，所述第一获取单元，还配置为：获取当前Solr集群的副本信息；所述第一生成单元，还配置为：基于所述当前Solr集群的副本信息，生成增加Solr集群的副本数量的安全层面策略。
在其他的实施例中，所述第一获取单元，还配置为：获取当前Solr集群的每台主机进行数据传输时的传输速度；所述第一生成单元，还配置为：基于所述当前Solr集群的每台主机进行数据传输时的传输速度，生成将主机上的Solr集群的副本转移至安全主机的安全层面策略。
在其他的实施例中，所述第二获取单元，还配置为：获取当前Solr集群的每台主机对索引进行读和写的速度，和，每台主机的系统负荷。
在其他的实施例中，所述第二生成单元，还配置为：基于所述当前Solr 集群的每台主机对索引进行读和写的速度，以及所述每台主机的系统负荷，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略。
在其他的实施例中，所述第二生成单元，包括：计算模块，配置为根据每台主机的第一权重值、第二权重值和第三权重值的总和，计算当前Solr集群的每台主机的综合性能权重值；生成模块，配置为根据所述当前Solr集群的每台主机的综合性能权重值，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略。
在其他的实施例中，所述装置还包括：第一确定单元，配置为确定当前Solr集群中的一台主机为基准主机；第二确定单元，配置为根据所述当前Solr集群中的每台主机读索引的速度与所述基准主机读索引的速度的比值，确定该主机的第一权重值；第三确定单元，配置为根据所述当前Solr集群中的每台主机写索引的速度与所述基准主机写索引的速度的比值，确定该主机的第二权重值；第四确定单元，配置为根据所述当前Solr集群中的每台主机的系统负荷与所述基准主机的系统负荷的比值，确定该主机的第三权重值。
在其他的实施例中，所述生成模块包括：排序子模块，配置为将所述当前Solr集群的每台主机的综合性能权重值进行排序；分组子模块，配置为根据所述排序的结果，将所述每台主机进行分组；累加子模块，配置为将每组中的每台主机的综合性能权重值累加作为该组主机的组性能权重值；分配子模块，配置为根据所述每组主机的组性能权重值为每组主机分配地址；生成子模块，配置为根据每组主机的所述地址，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略。
在其他的实施例中，所述生成子模块还配置为：根据每组主机的所述地址，将所述Solr集群的分片分配至所述每组主机；根据所述每组主机的分片，生成所述扩容后的Solr集群的性能层面策略。
以上装置实施例的描述，与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本发明装置实施例中未披露的技术细节，请参照本发明方法实施例的描述而理解。
需要说明的是，本发明实施例中，如果以软件功能模块的形式实现上述的方法，并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台网络侧切换设备(可以是终端或者网络等)执行本发明各个实施例所述方法的全部或部分。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read Only Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。这样，本发明实施例不限制于任何特定的硬件和软件结合。
对应地，本发明实施例提供一种服务器的扩容服务器，包括存储器和处理器，所述存储器存储有可在处理器上运行的计算机程序，所述处理器执行所述程序时实现上述的服务器的扩容分配方法，或者上述的服务器的扩容方法中的步骤。
对应地，本发明实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，所述计算机程序配置为执行时实现上述服务器的扩容方法中的步骤。
对应地，本发明实施例提供一种计算机程序产品，所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，当所述程序指令被计算机执行时，使所述计算机执行以上各个方面所述的方法。
这里需要指出的是：以上存储介质和服务器实施例的描述，与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本发明存储介质和设备实施例中未披露的技术细节，请参照本发明方法实施例的描述而理解。
需要说明的是，图8为本发明实施例中生成扩容策略服务器的一种硬件实体示意图，如图8所示，该服务器800的硬件实体包括：处理器801、通信接口802和存储器803，其中处理器801通常控制服务器800的总体操作。
通信接口802可以使服务器通过网络与其他终端或服务器通信。
存储器803配置为存储由处理器801可执行的指令和应用，还可以缓存待处理器801以及服务器800中各模块待处理或已经处理的数据(例如，图像数据、音频数据、语音通信数据和视频通信数据)，可以通过闪存(FLASH)或随机访问存储器(RandomAccessMemory，RAM)实现。
在本申请所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，如：多个单元或组件可以结合，或可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口，设备或单元的间接耦合或通信连接，可以是电性的、机械的或其它形式的。
上述作为分离部件说明的单元可以是、或也可以不是物理上分开的，作为单元显示的部件可以是、或也可以不是物理单元；既可以位于一个地方，也可以分布到多个网络单元上；可以根据实际的需要选择其中的部分或全部单元来实现本实施例方案的目的。
另外，在本发明各实施例中的各功能单元可以全部集成在一个处理单元中，也可以是各单元分别单独作为一个单元，也可以两个或两个以上单元集成在一个单元中；上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。
本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：移动存储设备、只读存储器(ReadOnly Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。
或者，本发明上述集成的单元如果以软件功能模块的形式实现并作为独 立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台服务器执行本发明各个实施例所述方法的全部或部分。而前述的存储介质包括：移动存储设备、ROM、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以所述权利要求的保护范围为准。

Claims (12)

  1. 一种服务器的扩容方法，其中，所述方法包括：
    获取当前企业级搜索应用服务器Solr集群的安全层面信息；
    基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略；
    获取当前Solr集群的性能层面信息；
    基于所述性能层面信息，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略；
    根据所述安全层面策略和所述性能层面策略对所述Solr集群进行扩容。
  2. 根据权利要求1所述的方法，其中，所述获取当前Solr集群的安全层面信息，包括：获取当前Solr集群的副本信息；
    所述基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略，包括：基于所述当前Solr集群的副本信息，生成增加Solr集群的副本数量的安全层面策略。
  3. 根据权利要求1所述的方法，其中，所述获取当前Solr集群的安全层面信息，包括：获取当前Solr集群的每台主机进行数据传输时的传输速度；
    所述基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略，包括：基于所述当前Solr集群的每台主机进行数据传输时的传输速度，生成将主机上的Solr集群的副本转移至安全主机的安全层面策略。
  4. 根据权利要求1所述的方法，其中，所述获取当前Solr集群的性能层面信息，包括：
    获取当前Solr集群的每台主机对索引进行读和写的速度，和，每台主机的系统负荷。
  5. 根据权利要求4所述的方法，其中，所述基于所述性能层面信息，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略，包括：
    基于所述当前Solr集群的每台主机对索引进行读和写的速度，以及所述每台主机的系统负荷，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略。
  6. 根据权利要求5所述的方法，其中，所述基于所述性能层面信息，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略，包括：
    根据每台主机的第一权重值、第二权重值和第三权重值的总和，计算当前Solr集群的每台主机的综合性能权重值；
    根据所述当前Solr集群的每台主机的综合性能权重值，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略。
  7. 根据权利要求6所述的方法，其中，所述方法还包括：
    确定当前Solr集群中的一台主机为基准主机；
    根据所述当前Solr集群中的每台主机读索引的速度与所述基准主机读索引的速度的比值，确定该主机的第一权重值；
    根据所述当前Solr集群中的每台主机写索引的速度与所述基准主机写索引的速度的比值，确定该主机的第二权重值；
    根据所述当前Solr集群中的每台主机的系统负荷与所述基准主机的系统负荷的比值，确定该主机的第三权重值。
  8. 根据权利要求6所述的方法，其中，所述根据所述当前Solr集群的每台主机的综合性能权重值，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略，包括：
    将所述当前Solr集群的每台主机的综合性能权重值进行排序；
    根据所述排序的结果，将所述每台主机进行分组；
    将每组中的每台主机的综合性能权重值累加作为该组主机的组性能权重值；
    根据所述每组主机的组性能权重值为每组主机分配地址；
    根据每组主机的所述地址，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略。
  9. 根据权利要求8所述的方法，其中，所述根据每组的所述地址，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略，包括：
    根据每组主机的所述地址，将所述Solr集群的分片分配至所述每组主机；
    根据所述每组主机的分片，生成所述扩容后的Solr集群的性能层面策略。
  10. 一种服务器的扩容装置，其中，所述装置包括：
    第一获取单元，配置为获取当前企业级搜索应用服务器Solr集群的安全层面信息；
    第一生成单元，配置为基于所述安全层面信息，生成用于对所述Solr集群在安全层面进行扩容的安全层面策略；
    第二获取单元，配置为获取当前Solr集群的性能层面信息；
    第二生成单元，配置为基于所述性能层面信息，生成用于对所述Solr集群在性能层面进行扩容的性能层面策略；
    扩容单元，配置为根据所述安全层面策略和所述性能层面策略对所述Solr集群进行扩容。
  11. 一种服务器的扩容服务器，包括存储器和处理器，所述存储器存储有可在处理器上运行的计算机程序，其中，所述处理器执行所述程序时实现权利 要求1至9任一项所述生成扩容策略中的步骤。
  12. 一种存储介质，其中，所述存储介质中存储有计算机可执行指令，所述计算机可执行指令配置为执行上述权利要求1至9任一项所提供的服务器的扩容方法。
PCT/CN2019/120668 2018-12-26 2019-11-25 服务器的扩容方法及装置、服务器、存储介质 WO2020134786A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811604894.XA CN111371583B (zh) 2018-12-26 2018-12-26 服务器的扩容方法及装置、服务器、存储介质
CN201811604894.X 2018-12-26

Publications (1)

Publication Number Publication Date
WO2020134786A1 true WO2020134786A1 (zh) 2020-07-02

Family

ID=71128320

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/120668 WO2020134786A1 (zh) 2018-12-26 2019-11-25 服务器的扩容方法及装置、服务器、存储介质

Country Status (2)

Country Link
CN (1) CN111371583B (zh)
WO (1) WO2020134786A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114168071A (zh) * 2021-10-29 2022-03-11 济南浪潮数据技术有限公司 一种分布式集群扩容方法、分布式集群扩容装置及介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113609245B (zh) * 2021-06-24 2023-12-22 济南浪潮数据技术有限公司 一种索引的分片扩容方法及系统
CN113505176A (zh) * 2021-07-08 2021-10-15 中国工商银行股份有限公司 分布式系统集群在线分片扩容方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104156367A (zh) * 2013-05-14 2014-11-19 阿里巴巴集团控股有限公司 一种搜索引擎的扩容方法及搜索服务系统
CN105024842A (zh) * 2014-04-25 2015-11-04 深圳市腾讯计算机系统有限公司 服务器的扩容方法及装置
CN106254470A (zh) * 2016-08-08 2016-12-21 广州唯品会信息科技有限公司 分布式作业分片分配方法和装置
CN107302444A (zh) * 2016-04-15 2017-10-27 中兴通讯股份有限公司 企业级搜索应用服务器集群自动扩容方法及装置
CN108469989A (zh) * 2018-03-13 2018-08-31 广州西麦科技股份有限公司 一种基于集群性能的反馈式自动扩缩容方法及系统

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8750199B2 (en) * 2008-11-28 2014-06-10 Via Telecom Co., Ltd. Cell selection handoff for CDMA2000 1X advance
KR101540631B1 (ko) * 2012-12-28 2015-07-30 삼성에스디에스 주식회사 가상 클러스터의 동적 확장 시스템, 방법 및 그 프로그램이 저장된 기록매체
CN103973759A (zh) * 2013-02-06 2014-08-06 腾讯科技（深圳）有限公司 负载调节的方法及装置
CN105262640A (zh) * 2015-09-17 2016-01-20 北京汉柏科技有限公司 一种提升云平台服务器可靠性的系统、方法及其部署框架
CN106020934A (zh) * 2016-05-24 2016-10-12 浪潮电子信息产业股份有限公司 一种基于虚拟集群在线迁移的优化部署方法
CN107562531B (zh) * 2016-06-30 2020-10-09 华为技术有限公司 一种数据均衡方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104156367A (zh) * 2013-05-14 2014-11-19 阿里巴巴集团控股有限公司 一种搜索引擎的扩容方法及搜索服务系统
CN105024842A (zh) * 2014-04-25 2015-11-04 深圳市腾讯计算机系统有限公司 服务器的扩容方法及装置
CN107302444A (zh) * 2016-04-15 2017-10-27 中兴通讯股份有限公司 企业级搜索应用服务器集群自动扩容方法及装置
CN106254470A (zh) * 2016-08-08 2016-12-21 广州唯品会信息科技有限公司 分布式作业分片分配方法和装置
CN108469989A (zh) * 2018-03-13 2018-08-31 广州西麦科技股份有限公司 一种基于集群性能的反馈式自动扩缩容方法及系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114168071A (zh) * 2021-10-29 2022-03-11 济南浪潮数据技术有限公司 一种分布式集群扩容方法、分布式集群扩容装置及介质
CN114168071B (zh) * 2021-10-29 2023-11-03 济南浪潮数据技术有限公司 一种分布式集群扩容方法、分布式集群扩容装置及介质

Also Published As

Publication number Publication date
CN111371583A (zh) 2020-07-03
CN111371583B (zh) 2022-09-23

Similar Documents

Publication Publication Date Title
US20220335034A1 (en) Multi-master architectures for distributed databases
WO2020134786A1 (zh) 服务器的扩容方法及装置、服务器、存储介质
US11262916B2 (en) Distributed storage system, data processing method, and storage node
CN103136110B (zh) 内存管理方法、内存管理装置及numa系统
EP2863310B1 (en) Data processing method and apparatus, and shared storage device
JP6086463B2 (ja) ピアツーピアデータ複製用の方法、デバイス、およびシステム、ならびにマスタノード切替え用の方法、デバイス、およびシステム
US10963353B2 (en) Systems and methods for cross-regional back up of distributed databases on a cloud service
WO2023216571A1 (zh) 弹性搜索集群的资源调度方法、装置及系统
US20240061712A1 (en) Method, apparatus, and system for creating training task on ai training platform, and medium
WO2015039569A1 (zh) 副本存储装置及副本存储方法
WO2020019779A1 (zh) 区块链数据的压缩处理方法和装置
US20240143456A1 (en) Log replay methods and apparatuses, data recovery methods and apparatuses, and electronic devices
CN115878046A (zh) 数据处理方法、系统、装置、存储介质及电子设备
US20240205292A1 (en) Data processing method and apparatus, computer device, and computer-readable storage medium
EP4012573A1 (en) Graph reconstruction method and apparatus
CN111046004B (zh) 一种数据文件存储方法、装置、设备及存储介质
CN111475279B (zh) 用于备份的智能数据负载平衡的系统和方法
US11010410B1 (en) Processing data groupings belonging to data grouping containers
CN112347036B (zh) 一种云存储系统的云间迁移方法及装置
CN111782634B (zh) 数据分布式存储方法、装置、电子设备及存储介质
JP2015095246A (ja) 情報処理システム、管理装置、サーバ装置及びキー割当プログラム
CN111666338B (zh) 数据复制方法、控制节点及电子设备
CN114647697A (zh) 访问数据库的方法、装置、计算设备和存储介质
CN115729982A (zh) 数据处理方法以及装置
CN113590311A (zh) 云设备分配方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19902924

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 16/11/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19902924

Country of ref document: EP

Kind code of ref document: A1